Day 9: a chain (or Russian doll) of containers

If you’re in the business, you’ve probably by now heard about containers. They can be described as executables on steroids, or also, as its namesake, a great way of shipping applications anywhere, or have then stored and ready to use whenever you need them. These kinda-executables are called images, and you can find them in a number of places called registries, starting with Docker Hub, joined lately by the GitHub Container Registry and other places like Quay.io or RedHat container catalog. These last are up and coming, you and have to add them to your default configuration. Most containers are registered in Docker Hub anyway.

Also, since they are kinda-executables, they are architecture and operating system specific. Docker hub marks their architecture and operating system, with Linux being the most common ones. You can, however, run Linux images everywhere, as long as the Docker daemon is running in a Linux virtual machine; that is also the default configuration in Macs.

Of course, there’s a slew of containers you can use with Raku, even if we don’t really have one we can call official. I’m going to go with my own, since, well, I’m more familiar with them. But there’re these nightly images by tyil, for instance, or the kinda-official Rakudo Star images, which have not been updated since, well, Rakudo Star itself was updated last March.

Let’s start with the basic image, the tiny Russian Doll with Nicky the tsar. Since it’s going to be inside, we need to make it real tiny. Here it is, jjmerelo/alpine-perl6:

FROM alpine:latest
LABEL version="2.2" maintainer="JJMerelo@GMail.com" perl6version="2019.11"

# Environment
ENV PATH="/root/.rakudobrew/versions/moar-2019.11/install/bin:/root/.rakudobrew/versions/moar-2019.11/install/share/perl6/site/bin:/root/.rakudobrew/bin:${PATH}" \
    PKGS="curl git perl" \
    PKGS_TMP="curl-dev linux-headers make gcc musl-dev wget" \
    ENV="/root/.profile" \
    VER="2019.11"

# Basic setup, programs and init
RUN mkdir /home/raku \
    apk update && apk upgrade \
    && apk add --no-cache $PKGS $PKGS_TMP \
    && git clone https://github.com/tadzik/rakudobrew ~/.rakudobrew \
    && echo 'eval "$(~/.rakudobrew/bin/rakudobrew init Sh)"' >> ~/.profile \
    && eval "$(~/.rakudobrew/bin/rakudobrew init Sh)"\
    && rakudobrew build moar $VER \
    && rakudobrew global moar-$VER \
    && rakudobrew build-zef\
    && zef install Linenoise App::Prove6\
    && apk del $PKGS_TMP \
    && RAKUDO_VERSION=`sed "s/\n//" /root/.rakudobrew/CURRENT` \
       rm -rf /root/.rakudobrew/${RAKUDO_VERSION}/src /root/zef \
       /root/.rakudobrew/git_reference

# Runtime
WORKDIR /home/raku
ENTRYPOINT ["raku"]

This image was created just last week, after the release of Raku 2019.11, the first one to actually be called Raku and the one that calls its executable raku too.

First thing you see is that FROM which declares the teeny container that’s inside this one. We’re using Alpine Linux, a distribution little known outside the containerer community, that uses a couple of tricks to avoid bloating the number of files, and thus the size, of the container. This image will add up to less than 300 MBs, while an equivalent image with Debian or Ubuntu will be twice as much. That means that downloading it will take half as much, which is what we’re looking for.

Because there’s this thing, too: real-life containers, when empty, can be Russian-dolled and put inside one another so that they don’t occupy so much space. Something similar happens to containers. They are built putting layers on top of each other, the inner layer usually an operating system. Let’s check out the rest.

The next LABELs are simply tags or metadata that can be extracted from the image by inspection. Not really that important.

But the ENV block kinda is, over all the first one, which defines the PATH that is going to be used across the Russian doll buildup. The rest of the variables are mainly used while building the image. They will also help to make it somewhat generic, so that we can just change the value of a variable and get a new version; we put that into VER.

So far, no building has taken place, but in this humongous RUN statement is where we download rakudobrew, put it to work building the version contained in VER, set that version as the default one, install zef and a couple of modules we are going to need, and then delete what we will no longer be needing in the rest of the outer dolls to keep the whole thing small.

Finally, after setting up a working directory, we define an entry point, which is the real executable-within-the-executable. The container can be used in place of this command, so that anything that can be done with raku, can be done with this this executable. For instance, let’s run this program:

my @arr;
my ($a, $b) = (1,1);
for ^5 {
    ($a,$b) = ($b, $a+$b);
    @arr.push: ($a.item, $b.item);
    say @arr
};
say @arr;

We will give our containerized Raku an alias:

alias raku-do='docker run --rm -t -v `pwd`:/home/raku  jjmerelo/alpine-perl6'

We can run the program above with:

raku-do itemizer-with-container.p6

But you can take it a step further. Create this shell script and put it in the path:

#!/bin/bash

docker run --rm -t -v `pwd`:/home/raku  jjmerelo/alpine-perl6 $@

You can then use this in the shebang line: !/usr/bin/env raku-do.sh. This will create a throwaway image, that will be ephemerally created to run the script, and then thrown away (that’s the --rm in the line). The current directory (pwd) will be aliased to /home/raku, remember, our working directory, which means that the raku inside the container will see it right there. You see? With this you can have raku run wherever docker is installed. Pretty much everywhere, nowadays.

But let’s build up on this. Containers are extensively used for testing, since instead of building and installing, you can put everything in a single container, and download and use it for testing straight away. That’s is actually what made a containerer out of me, the long 20 minutes it took to brew rakudo for a few seconds of testing for every module. After that base container, I created this one, jjmerelo/test-perl6. Here it is:

FROM jjmerelo/alpine-perl6:latest
LABEL version="4.0.2" maintainer="JJ Merelo <jjmerelo@GMail.com>"

# Set up dirs
RUN mkdir /test
VOLUME /test
WORKDIR /test


# Will run this
ENTRYPOINT perl6 -v && zef install --deps-only . && zef test .

This is actually simplicity itself: the only thing that changes is the entrypoint and the working dir. Instead of running directly the raku compiler, it does a couple of things: install dependencies needed to run the tests, and then issue zef test . to run the tests.

That really speeds things up when testing. Put it in your .travis.yml file this way:

language: minimal

services:
  - docker

install: docker pull jjmerelo/test-perl6

script: docker run -t -v  $TRAVIS_BUILD_DIR:/test jjmerelo/test-perl6

And you’re good to go. Takes all of a minute and a half, as opposed to more than 20 minutes if you use the official Travis image, which is based in rakudobrew.

The Russian doll does not stop there: jjmerelo/perl6-test-openssl includes additional Alpine packages which are needed to install OpenSSL. And, based on that one, jjmerelo/perl6-doccer, which packs everything that’s needed to test the Raku documentation.

You should really try this yourself. If you’ve got even just a few additional modules to download when testing your module, just build up from the test-perl6 image and get your own! You’ll save time, and also save computing time, thus saving energy.