Peeling the stinky onion that expresses the beautiful simplicity of cloud

With 1 line we can have a postgresql 10 image running. We stand on the shoulders of giants, and are efficient. Maybe we should take an early and long lunch after that exertion? I mean, why bother to poke under the covers? But what risks did we just introduce to our organisation?

Lets take a look at the GoogleCloudPlatform docker container, designed for use in Kubernetes etc. First, the Dockerfile.

FROM gcr.io/google-appengine/debian9

OK, this is giving is a starting ‘layer’. So we have to jump there first.  At first its a pretty vanilla debian 9 build, and debian has a good track record of implementing upstream security issues.

A few items that might raise an eyebrow. If we look at the build, it uses http rather than https. So this means that, during the build of this image, if we can spoof or poison DNS, we own this image. E.g. the debootstrap is:

debootstrap --variant="$VARIANT" "$DIST" "$WORKDIR" http://snapshot.debian.org/archive/debian/"$SNAPSHOT"

OK, but presumably this is build on Google Cloud Builder and they have some protection against this? But, what bombs does it lay for us the user later? If we look at the sources.list it contains:

deb http://snapshot.debian.org/archive/debian/$SNAPSHOT $DIST main

It also installs the ca-certificates (the bundle by which the OS and all derived layers will verify HTTPS) in the same fashion. This means that later when I use this image, and do an ‘apt update’ e.g. in my Dockerfile, or other, I am exposed to spoofery.

OK, switching back to Postgresql and continuing through the Dockerfile:

# grab gosu for easy step-down from root
ENV GOSU_VERSION 1.10
RUN set -x \
	&& apt-get update && apt-get install -y --no-install-recommends ca-certificates wget && rm -rf /var/lib/apt/lists/* \
	&& wget -q -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture)" \
	&& wget -q -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture).asc" \
# copy source code
	&& wget -q -O /usr/local/src/gosu.tar.gz "https://github.com/tianon/gosu/archive/$GOSU_VERSION.tar.gz" \
# extract gosu binary and check signature
	&& export GNUPGHOME="$(mktemp -d)" \
	&& found='' \
	&& for server in \
		pool.sks-keyservers.net \
		ha.pool.sks-keyservers.net \
		pgp.mit.edu \
		na.pool.sks-keyservers.net \
		eu.pool.sks-keyservers.net \
		oc.pool.sks-keyservers.net \
		ha.pool.sks-keyservers.net \
		hkp://p80.pool.sks-keyservers.net:80 \
		hkp://keyserver.ubuntu.com:80 \
	; do \
		gpg --keyserver $server --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 \
			&& found="yes" && break; \
	done; test -n "$found" \
	&& gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu \
	&& rm -rf "$GNUPGHOME" /usr/local/bin/gosu.asc \
	&& chmod +x /usr/local/bin/gosu \
	&& gosu nobody true \
	&& apt-get purge -y --auto-remove ca-certificates wget

OK, that was a bit of a mouthful. At first blush it looks terrible, its doing a wget -O /usr/local/bin/gosu from a 3rd-party git repo. Who owns that, do they still own it, how was it built, etc. scrolling down that paragraph we see that it does verify the GPG signing of it. So, the risk we introduce is:

1. Who is tianon, do we trust him/her
2. How was this built, was it secure?
3. How secure is that signing key?
4. What upstreams does that pull in?

So lets go look there. The author warns us specifically about using this more broadly than intended (and about CVE-2016-2779 in this discussion). But, this is written in Go, we introduce the entire upstream risk of that, its runtime, garbage collector etc. So, well, a fair bit of risk is introduced here. Now, its likely we intend to use this to *drop* privilege rather than increase (and there is no setuid chmod above), so perhaps not a big deal. Hmm..

Lets move on.

# make the "en_US.UTF-8" locale so postgres will be utf-8 enabled by default
RUN apt-get update && apt-get install -y locales && rm -rf /var/lib/apt/lists/* \
	&& localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
ENV LANG en_US.utf8

Here we are updating packages, and using that ‘http://’ repo from above, so the risk now shifts to our DNS and caching and spoofing etc. We could end up pulling something we don’t want (e.g. a ‘localedef’ from a hacked ‘locales’ package could modify our system and we would not kn0w).

RUN mkdir /docker-entrypoint-initdb.d

RUN set -ex; \
	key='B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8'; \
	export GNUPGHOME="$(mktemp -d)"; \
	found=''; \
	for server in \
		pool.sks-keyservers.net \
		ha.pool.sks-keyservers.net \
		pgp.mit.edu \
		na.pool.sks-keyservers.net \
		eu.pool.sks-keyservers.net \
		oc.pool.sks-keyservers.net \
		ha.pool.sks-keyservers.net \
		hkp://p80.pool.sks-keyservers.net:80 \
		hkp://keyserver.ubuntu.com:80 \
	; do \
		gpg --keyserver ${server} --recv-keys "$key" \
			&& found="yes" && break; \
	done; \
	test -n "$found"; \
	gpg --export "$key" > /etc/apt/trusted.gpg.d/postgres.gpg; \
	rm -rf "$GNUPGHOME"; \
	apt-key list

ENV PG_MAJOR 10
ENV PG_VERSION 10.3-1.pgdg90+1
RUN echo 'deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main' $PG_MAJOR > /etc/apt/sources.list.d/pgdg.list

RUN apt-get update \
	&& apt-get install -y postgresql-common \
	&& sed -ri 's/#(create_main_cluster) .*$/\1 = false/' /etc/postgresql-common/createcluster.conf \
	&& apt-get install -y \
		postgresql-$PG_MAJOR=$PG_VERSION \
		postgresql-contrib \
	&& rm -rf /var/lib/apt/lists/*

OK, similar to the gosu, we are using a 3rd party upstream (with a decent track record) and installing their binaries. We are checking the gpg keys, we are introducing some risk around using http:// repo. And we are importing all of the build tools and choices of postgresql.

So, in summary, my 1 line command imported the business risk of the supply chain of several ‘vendors’, asserted by either nothing or their gpg keys. And then I build upon it. The ripples in the pond. The layers of the onion.

Hmm.