Most recently, there was a
release of minimalistic Alpine Linux 3.8. Very often, this linux image is used in the docker, collecting very compact environments for runtime.
Today's article will be discussed in the context of using the runtime system in the docker for Python 3.6.X versions, with different composition of pip packages. And also we will assemble the newest Python 3.7 in Alpine.
At the end of the article will be presented the size of the image image occupied on the disk, depending on the composition of the pip packages, and a comparison between the distributions of Alpine 3.8, Debian 9, Fedora 28 is made.
So let's get started: distributions are selected for testing. We will collect the following docker images:
- System, its update. And Python3 with updated pip (version 10)
- p.1 + tornado cython
- p.2 + numpy-scipy
- p.3 + pillow bokeh pandas websocket-client
As a result of these fills, we will get different versions: Python without packages, Python with a web server, Python with packages for processing multi-threaded mathematical calculations, Python with a “graphical” stack, and working with data.
So, the resulting files for Debian and Fedora will look like this for us:
Debian
FROM debian RUN apt-get update -y && apt-get install python3-pip -y && pip3 install pip
Fedora
FROM fedora RUN dnf update -y && dnf install libstdc++ -y && dnf clean all && pip3 install
But with Alpine 3.8 yet a hitch. Officially, at the time of this writing, he still came out, but I want to see :-). Therefore, we need their image of the system:
dl-cdn.alpinelinux.org/alpine/v3.8/releases/x86_64And we will build your Alpine from Sratch:
github.com/gliderlabs/docker-alpine/tree/master/versions/library-3.8/x86_64We download the file system Alpine 3.8:
curl
dl-cdn.alpinelinux.org/alpine/v3.8/releases/x86_64/alpine-minirootfs-3.8.0_rc10-x86_64.tar.gz > alpine3.8.tar.gz
Create your docker file:
FROM scratch ADD alpine3.8.tar.gz / ENV RELEASE="v3.8" ENV MIRROR="http://dl-cdn.alpinelinux.org/alpine" ENV PACKAGES="alpine-baselayout,busybox,alpine-keys,apk-tools,libc-utils" ENV TAGS=(alpine:3.8)
Then copy-paste and add Python 3.6 backups to this file from
github.com/docker-library/python/blob/master/3.6/alpine3.7/Dockerfiledo not forget to delete or comment out the line FROM alpine: 3.7And try to create an image with Alpine 3.8 and Python on board:
$docker build -t alpine3.8 .
The results of the first step are installing only Python (docker images - all):
- Debian 9/513 MB
- Fedora 28/387 MB
- Alpine 3.8 / 82.2 MB
Step 2. Installing cython and tornado
Start adding pip packages. First install cython and tornado. For Debian and Fedora, the packages are installed without errors, but Alpine crashes:
unable to execute 'gcc': No such file or directory error: command 'gcc' failed with exit status 1
You have to google and then add the build libraries to Alpine so that pip successfully compiles them from the source text. Then run the docker build again, then look for dependencies again, read the stackoverflow and issues forums in github and wait and wait and wait.
Since in the following steps we will start adding math and graphics libraries to our Python runtime image, and in order not to increase the text of this article too much, I will give the final dependencies for Alpine linux:
apk add --no-cache --virtual .build-deps \ gfortran \ build-base \ openblas-dev \ bzip2-dev \ coreutils \ dpkg-dev dpkg \ expat-dev \ gcc \ gdbm-dev \ libc-dev \ libffi-dev \ libnsl-dev \ libressl \ libressl-dev \ libtirpc-dev \ linux-headers \ make \ ncurses-dev \ pax-utils \ readline-dev \ sqlite-dev \ tcl-dev \ tk \ tk-dev \ xz-dev \ zlib-dev \ libxml2-dev \ libxslt-dev \ musl-dev \ libgcc \ curl \ jpeg-dev \ zlib-dev \ freetype-dev \ lcms2-dev \ openjpeg-dev \ tiff-dev \ tk-dev \ tcl-dev \ && ln -s /usr/include/locale.h /usr/include/xlocale.h
- Debian 9/534 MB
- Fedora 28/407 MB
- Alpine 3.8 / 144 MB
Step 3. Add the math numpy scipy
- Debian 9/763 MB
- Fedora 28/626 MB
- Alpine 3.8 / 404 MB MB
Step 4. Add a websocket-client graphic stack pytest pandas bokeh pillow
Dockerfile-alpine FROM scratch ADD alpine3.8.tar.gz / CMD ["/bin/sh"] ENV RELEASE="v3.8" ENV MIRROR="http://dl-cdn.alpinelinux.org/alpine" ENV PACKAGES="alpine-baselayout,busybox,alpine-keys,apk-tools,libc-utils" #ENV BUILD_OPTIONS=(-b -s -t UTC -r $RELEASE -m $MIRROR -p $PACKAGES) ENV TAGS=(alpine:3.8) # # NOTE: THIS DOCKERFILE IS GENERATED VIA "update.sh" # # PLEASE DO NOT EDIT IT DIRECTLY. # #https://github.com/docker-library/python/blob/master/3.6/alpine3.7/Dockerfile # ensure local python is preferred over distribution python ENV PATH /usr/local/bin:$PATH # http://bugs.python.org/issue19846 # > At the moment, setting "LANG=C" on a Linux system *fundamentally breaks Python 3*, and that's not OK. ENV LANG C.UTF-8 # install ca-certificates so that HTTPS works consistently # the other runtime dependencies for Python are installed later RUN apk add --no-cache ca-certificates ENV GPG_KEY 0D96DF4D4110E5C43FBFB17F2D347EA6AA65421D ENV PYTHON_VERSION 3.6.6 RUN set -ex \ && apk add --no-cache --virtual .fetch-deps \ gnupg \ libressl \ tar \ xz \ \ && wget -O python.tar.xz "https://www.python.org/ftp/python/${PYTHON_VERSION%%[az]*}/Python-$PYTHON_VERSION.tar.xz" \ && wget -O python.tar.xz.asc "https://www.python.org/ftp/python/${PYTHON_VERSION%%[az]*}/Python-$PYTHON_VERSION.tar.xz.asc" \ && export GNUPGHOME="$(mktemp -d)" \ && gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$GPG_KEY" \ && gpg --batch --verify python.tar.xz.asc python.tar.xz \ && rm -rf "$GNUPGHOME" python.tar.xz.asc \ && mkdir -p /usr/src/python \ && tar -xJC /usr/src/python --strip-components=1 -f python.tar.xz \ && rm python.tar.xz \ \ && apk add --no-cache --virtual .build-deps \ bzip2-dev \ coreutils \ dpkg-dev dpkg \ expat-dev \ gcc \ gdbm-dev \ libc-dev \ libffi-dev \ libnsl-dev \ libressl \ libressl-dev \ libtirpc-dev \ linux-headers \ make \ ncurses-dev \ pax-utils \ readline-dev \ sqlite-dev \ tcl-dev \ tk \ tk-dev \ xz-dev \ zlib-dev \ # add build deps before removing fetch deps in case there's overlap && apk del .fetch-deps \ \ && cd /usr/src/python \ && gnuArch="$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)" \ && ./configure \ --build="$gnuArch" \ --enable-loadable-sqlite-extensions \ --enable-shared \ --with-system-expat \ --with-system-ffi \ --without-ensurepip \ && make -j "$(nproc)" \ # set thread stack size to 1MB so we don't segfault before we hit sys.getrecursionlimit() # https://github.com/alpinelinux/aports/commit/2026e1259422d4e0cf92391ca2d3844356c649d0 EXTRA_CFLAGS="-DTHREAD_STACK_SIZE=0x100000" \ && make install \ \ && runDeps="$( \ scanelf --needed --nobanner --format '%n#p' --recursive /usr/local \ | tr ',' '\n' \ | sort -u \ | awk 'system("[ -e /usr/local/lib/" $1 " ]") == 0 { next } { print "so:" $1 }' \ )" \ && apk add --virtual .python-rundeps $runDeps \ && apk del .build-deps \ \ && find /usr/local -depth \ \( \ \( -type d -a \( -name test -o -name tests \) \) \ -o \ \( -type f -a \( -name '*.pyc' -o -name '*.pyo' \) \) \ \) -exec rm -rf '{}' + \ && rm -rf /usr/src/python # make some useful symlinks that are expected to exist RUN cd /usr/local/bin \ && ln -s idle3 idle \ && ln -s pydoc3 pydoc \ && ln -s python3 python \ && ln -s python3-config python-config # if this is called "PIP_VERSION", pip explodes with "ValueError: invalid truth value '<VERSION>'" ENV PYTHON_PIP_VERSION 10.0.1 RUN set -ex; \ \ apk add --no-cache --virtual .fetch-deps libressl; \ \ wget -O get-pip.py 'https://bootstrap.pypa.io/get-pip.py'; \ \ apk del .fetch-deps; \ \ python get-pip.py \ --disable-pip-version-check \ --no-cache-dir \ "pip==$PYTHON_PIP_VERSION" \ ; \ pip --version; \ \ find /usr/local -depth \ \( \ \( -type d -a \( -name test -o -name tests \) \) \ -o \ \( -type f -a \( -name '*.pyc' -o -name '*.pyo' \) \) \ \) -exec rm -rf '{}' +; \ rm -f get-pip.py apk add --no-cache --virtual .build-deps \ gfortran \ build-base \ openblas-dev \ bzip2-dev \ coreutils \ dpkg-dev dpkg \ expat-dev \ gcc \ gdbm-dev \ libc-dev \ libffi-dev \ libnsl-dev \ libressl \ libressl-dev \ libtirpc-dev \ linux-headers \ make \ ncurses-dev \ pax-utils \ readline-dev \ sqlite-dev \ tcl-dev \ tk \ tk-dev \ xz-dev \ zlib-dev \ libxml2-dev \ libxslt-dev \ musl-dev \ libgcc \ curl \ jpeg-dev \ zlib-dev \ freetype-dev \ lcms2-dev \ openjpeg-dev \ tiff-dev \ tk-dev \ tcl-dev \ && ln -s /usr/include/locale.h /usr/include/xlocale.h \ && pip install cython tornado websocket-client pytest numpy pandas scipy bokeh pillow \ && apk del .build-deps \ && apk add --no-cache libstdc++ openblas zlib jpeg openjpeg tiff tk tcl musl libxml2 libxslt xz zlib libstdc++ openblas \ && pip list CMD ["python3"]
- Debian 9/905 MB
- Fedora 28/760 MB
- Alpine 3.8 / 650 MB
As a bonus, we will try to compile in Alpine 3.8, which has not yet been released for the docker, Python 3.7.
New version of Python 3.7 presented June 27, 2018Compile code is taken from:
github.com/docker-library/python/tree/bbbc37fff3411a34deef30dd9b34dc938fe7b134/3.7-rc/alpine3.7Compile Completion 3.7 Package Version ---------------- ------- atomicwrites 1.1.5 attrs 18.1.0 bokeh 0.13.0 Cython 0.28.3 Jinja2 2.10 MarkupSafe 1.0 more-itertools 4.2.0 numpy 1.14.5 packaging 17.1 pandas 0.23.1 Pillow 5.1.0 pip 10.0.1 pluggy 0.6.0 py 1.5.4 pyparsing 2.2.0 pytest 3.6.2 python-dateutil 2.7.3 pytz 2018.4 PyYAML 4.1 scipy 1.1.0 setuptools 39.2.0 six 1.11.0 tornado 5.0.2 websocket-client 0.48.0 wheel 0.31.1 ---> 3d18b8c27cd9 Removing intermediate container f546e004b79f Step 18/18 : CMD python3 ---> Running in 23c5aea50a0d ---> d5385e425064 Removing intermediate container 23c5aea50a0d Successfully built d5385e425064 real 41m17,619s
Alpine 3.8 size with Python 3.7 with current list of pip 656 MB packages
Results
Python
- Debian 9 / more at 6.24x / +430 Mb
- Fedora 28 / more in 4,7x / +304 Mb
Python tornado cython
- Debian 9 / more in 3.71x / +390 Mb
- Fedora 28 / more at 2.82x / +263 Mb
Python tornado cython numpy scipy
- Debian 9 / more than 1.88 times / +359 Mb
- Fedora 28 / more 1.54 times / +222 Mb
Python tornado cython numpy scipy websocket-client pytest pandas bokeh pillow
- Debian 9 / more 1.39 times / +255 Mb
- Fedora 28 / more 1.16 times / +110 Mb
When using an empty Python runtime , the
Alpine linux distribution is the smallest
leader . By increasing the number of pip libraries to tornado + cython + numpy + scipy, Alpine still provides significant savings in size on the hard disk. However,
as soon as
graphical utilities for working with data for Python
appear in packages, the
difference practically disappears.With a large number of graphics packages, it is better to choose the Fedora distribution kit than to compile the packages in Alpine (the compilation can take 1-2 hours), and as a result, you can save one or two ten percent of the hard disk space.
UPDATE1: Testing was conducted on Fedora Atomic Host: release 28 (Twenty Eight), Version: 2018.5
UPDATE2: Checking the occupied disk space, was carried out on this build
hub.docker.com/r/flytrue/python-runtime-docker/tags $docker pull flytrue/python-runtime-docker:alpine-full $docker save flytrue/python-runtime-docker:alpine-full -o alpine-full.tar $ls -lh alpine-full.tar -rw-------. 1 fedora fedora 631M 29 08:38 alpine-full.tar $ docker images --all|grep alpine docker.io/flytrue/python-runtime-docker alpine-full f37154658671 19 hours ago 650 MB