Building in Docker Containers on GitLab CE

In this article I will show how I added a GitLab runner with a Docker executor to my original GitLab CE setup. This type of GitLab runner will allow me to run GitLab jobs in containers created from Docker images.

Almost a year ago I wrote an article about running GitLab CE in a Docker container. At that time I promised that I would return with more GitLab related articles and now the time has come.
Before continuing, I have to note that this article is more in the line of a personal notebook. There are far better articles out there if you want a configuration that is more appropriate for a production environment with multiple users, full-blown security etc.

Motivation

Some of the reason for me wanting to run GitLab jobs in Docker containers are:

• Allows to run steps in a build in parallel without the risk of one build step affecting another.
  In my case this means for example being able to run tests in one container while running, for instance, static code analysis in another.
• Controlled and reproducible build environment.
  The build environment, that is the container in which the build job is performed, is created immediately prior to the job being run and discarded immediately afterwards.
• Easy configuration of the build environment.
  In my case this means allowing me to use different versions of Java and Maven when building. It is even possible to build against multiple Java versions in one and the same GitLab CI/CD pipeline.
  In an organization with multiple developers, this would mean letting the developers decide which Docker image to use for a job and not having to maintain a build server to any larger extent than maintaining a GitLab runner with a Docker executor.

Prerequisites

The Docker host I am using is running Ubuntu 18.04 and have Docker version 18.09 as well as Docker Compose version 1.25 installed.
The GitLab Runner documentation states that the minimum Docker version required is version 1.5.0.

Updated Docker Compose Configuration

I started out with the Docker Compose configuration from the previous article and added a service for the GitLab Runner, a volume for the GitLab Runner’s configuration and a network to be shared by GitLab, the GiLab Runner(s) and the job containers allowing the GitLab Runner and job containers to communicate with GitLab.
I recommend creating a directory for the GitLab stack and then creating a docker-compose.yml in the directory with the following contents:

# GitLab CE deployment with one GitLab Runner.
#
version: '3'

services:
# GitLab service.
# Change the hostname value in the configuration below to match the name of
# your server/computer on which GitLab is to be run.
# Once started, access GitLab using the URL http://[hostname]:8880.
# The logs directory can be mapped to a logs directory in the same director
# as the docker-compose file using the following entry under volumes:
# - ./logs:/var/log/gitlab
  gitlab:
    image: gitlab/gitlab-ce:latest
    hostname: localhost
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        gitlab_rails['gitlab_shell_ssh_port'] = 8822
# The following ports are exposed by GitLab:
# 8880 (HTTP)
# 443 (if HTTPS configured)
# 8080 (used by Unicorn)
# 8822 (used by the SSH daemon)
    ports:
      - "443:443"
      - "8880:80"
      - "8822:22"
# Volumes to store GitLab's data and configuration on.
    volumes:
      - gitlab-config:/etc/gitlab
      - gitlab-data:/var/opt/gitlab
    restart: always
    networks:
      gitlabnetwork: {}

# GitLab Runner service.
  gitlab-runner:
    image: gitlab/gitlab-runner:latest
    ports:
      - "8093:8093"
# Volume to store the GitLab Runner's configuration on and a
# volume that allows the runner to access the host's Docker socket.
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - gitlab-runner-config:/etc/gitlab-runner
    restart: always
    networks:
      gitlabnetwork: {}

# Use an attachable network shared by GitLab, the GitLab runner and
# all the job containers.
networks:
  gitlabnetwork:
    driver: overlay

# Volumes for GitLab's data and configuration as well as
# a volume for the GitLab runner's configuration.
volumes:
  gitlab-data:
  gitlab-config:
  gitlab-runner-config:

Comments:

• The first part is the GitLab service as described in the previous article with the addition of a network configuration.
  The network is named gitlabnetwork. Details on the network below!
• The second service is the GitLab Runner service (named gitlab-runner).
• The GitLab Runner service exposes port 8093.
  This port is used by the GitLab Runner’s session server that allows a user to interact with jobs created by the runner.
• The GitLab Runner service has two volumes.
  The first volume exposes the host’s Docker socket to the GitLab Runner’s container, allowing the runner to interact with the host’s Docker service.
  The second volume is used to store the GitLab Runner’s configuration on.
• The GitLab Runner service joins the same network as GitLab.
  The network is named gitlabnetwork. Details on the network below!
• A network named gitlabnetwork is declared.
  This network will be used for communication between the GitLab instance, the GitLab Runner and the job containers that the GitLab Runner will start.
  The network is an overlay network. This is in order to ensure that the network is attachable.
  The network need to be attachable, which is the default for overlay networks, in order for the job containers to be able to join the network when they are started.
• Three volumes are declared.
  The first two volumes are for GitLab data and configuration, as seen in my previous article. The third volume is for the GitLab Runner’s configuration.

Starting the GitLab Stack

We are now ready to start the GitLab stack.

• Open a terminal window.
• Go to the GitLab stack directory.
• Start the GitLab stack using the following command:
  sudo docker-compose up -d
  Output similar to the following should appear in the terminal window:

Creating network "gitlabce_gitlabnetwork" with driver "overlay"
Creating volume "gitlabce_gitlab-data" with default driver
Creating volume "gitlabce_gitlab-config" with default driver
Creating volume "gitlabce_gitlab-runner-config" with default driver
Creating gitlabce_gitlab_1        ... done
Creating gitlabce_gitlab-runner_1 ... done

• Wait a few minutes and then issue the following command to check the status of the GitLab stack:
  sudo docker ps -a
  Output similar to the following should appear in the terminal window:

CONTAINER ID        IMAGE                         COMMAND                  CREATED             STATUS                   PORTS                                                              NAMES
169de81569dc        gitlab/gitlab-ce:latest       "/assets/wrapper"        6 minutes ago       Up 6 minutes (healthy)   0.0.0.0:443->443/tcp, 0.0.0.0:8822->22/tcp, 0.0.0.0:8880->80/tcp   gitlabce_gitlab_1
ccdf9b6763b9        gitlab/gitlab-runner:latest   "/usr/bin/dumb-init …"   6 minutes ago       Up 6 minutes             0.0.0.0:8093->8093/tcp                                             gitlabce_gitlab-runner_1

• Note the name of the GitLab Runner container.
  In the above example, it is gitlabce_gitlab-runner_1.

If all goes according to plan, both the GitLab and the GitLab Runner containers should be up. We can now check that the GitLab web UI is up and running:

• Open the URL http://localhost:8880 in a browser.
  You should be presented with a request to create a new password for your new account.
  I will enter “password” in both the fields for the sake of this example.
  Having successfully created the new password, you will then be directed to the login page.

Obtaining a GitLab Runner Registration Token

GitLab will not automatically be aware of new GitLab Runners until they have been registered. Prior to registering a GitLab Runner, a runner registration token must be obtained from GitLab. As of writing this article, there is no good way to automatically obtain such a token known to me

• Log in using the user name “root” and the password “password” (or the password you entered when creating a new password).
  Upon successful login, the GitLab Welcome page should be shown.

• Click the wrench in the GitLab menu bar.
  This will take you to the Admin Area landing page.

• Click the Runners entry in the Admin Area menu on the left.
  The Runners page on which all runners known to this GitLab instance are listed.
  For obvious reasons there are no runners listed on this page, since this is a new GitLab instance.

• On the right, under the “Set up a shared Runner manually” title copy the registration token in red.
  This is the token needed to register our GitLab Runner with GitLab.

Registering and Configuring the GitLab Runner

With the name of the container in which the GitLab Runner is running (gitlabce_gitlab-runner_1 in my case) and the registration token obtained in the previous step the GitLab Runner can now be registered. If you add more GitLab Runners then this registration procedure has to be repeated once for each runner.

• Open a terminal window or re-use an already open terminal window.
  This window has to be opened on the Docker host on which the GitLab Runner container is located.
• If you do not know the name of the container in which the GitLab Runner is running, issue the following command to obtain the container name:

sudo docker ps -a

• Enter the container by executing the following command in the terminal window, replacing the container name if it is different:

sudo docker exec -it gitlabce_gitlab-runner_1 bash

• While in the GitLab Runner container, try the following commands which provide valuable help when configuring GitLab Runners:

gitlab-runner --help
gitlab-runner register --help

• In the container execute the following command to register and configure the GitLab Runner. Replace the registration token value with the registration token value you obtained earlier.

gitlab-runner register \
--executor="docker" \
--custom_build_dir-enabled="true" \
--docker-image="maven:3.6.1-jdk-11" \
--url="http://gitlab:80" \
--clone-url="http://gitlab:80" \
--registration-token="vWtwwQgdPSEzTPNTGZnq" \
--description="docker-runner" \
--tag-list="docker" \
--run-untagged="true" \
--locked="false" \
--docker-network-mode="gitlabce_gitlabnetwork" \
--docker-disable-cache="true" \
--docker-privileged="true" \
--cache-dir="/cache" \
--builds-dir="/builds" \
--docker-volumes="gitlab-runner-builds:/builds" \
--docker-volumes="gitlab-runner-cache:/cache"

Comments on the configuration options:

• –executor=”docker”
  Configures GitLab Runner jobs to be run in containers.
• –custom_build_dir-enabled
  Enables job-specific build directories.
• –docker-image=”maven:3.6.1-jdk-11″
  The default Docker image to use when creating a container to run a job in if none is specified in the GitLab CI/CD configuration. I have chosen a Maven Docker image, since I will mainly build my projects with Maven.
• –url=”http://gitlab:80″
  The GitLab URL. Will be used as a base URL for the GitLab API. Note that the “address” is gitlab, which is the name of the service in the Docker Compose file. The port is 80 which is the port on which GitLab is listening inside the container and thus also in the gitlabnetwork network (and not the port in the host to which port 80 is mapped, which is 8880).
• –clone-url=”http://gitlab:80″
  Base URL used when checking out projects from GitLab job containers. Uses the “gitlab” service name that is only reachable within the gitlabnetwork Docker network.
• –registration-token=”vWtwwQgdPSEzTPNTGZnq”
  GitLab Runner registration token as obtained from GitLab.
• –description=”docker-runner”
  Arbitrary description of the GitLab Runner.
• –tag-list=”docker”
  One or more tags that can be used to associate a GitLab project with one or more GitLab Runners so that the project will be built by the runners with the matching tag(s).
• –run-untagged=”true”
  Allow the runner to run jobs without tags.
• –locked=”false”
  Do not lock the GitLab Runner to the current project only.
• –docker-network-mode=”gitlabce_gitlabnetwork”
  Ensures that GitLab job containers join the same network as GitLab in order to be able to check out projects to build from GitLab.
• –docker-disable-cache=”true”
  Causes GitLab job containers to be removed immediately after job finished executing.
  Given that my system is a laptop and thus have limited disk resources, I have chosen to enable this option. You may want to remove this if you want job containers to remain in existence after job execution has completed, in which case you may want to consider some other alternative to clean up old containers.
• –docker-privileged=”true”
  Give extended privileges to job containers. This may incur security risks. Please refer to the Docker documentation for details.
• –cache-dir=”/cache”
  Directory where build cache is stored in the job containers. Set to ensure that the builds cache is written to a Docker volume – see below.
• –builds-dir=”/builds”
  Directory where builds are stored in the job containers. Set to ensure that the builds directory is written to a Docker volume – see below.
• –docker-volumes=”gitlab-runner-builds:/builds”
  Maps the builds directory in the job containers spawned from this runner to the Docker volume “gitlab-runner-builds”.
  The volume will be created first time it is used, if it does not already exist.
• –docker-volumes=”gitlab-runner-cache:/cache”
  Maps the builds cache directory in the job containers spawned from this runner to the Docker volume “gitlab-runner-cache”. The volume will be created first time it is used, if it does not already exist.
  Using Docker volumes to store cache and build directories in ensures that caching will be preserved even though job containers are removed. In addition, Docker volumes are more visible than shared directories on the host computer and can be easily managed.

Having executed the command you will be asked a number of questions regarding the configuration of the GitLab Runner. The default values will be taken from the provided parameters, so just press enter on each of the questions. The console output will look similar to this if the registration was successful:

Runtime platform                                    arch=amd64 os=linux pid=39 revision=ac2a293c version=11.11.2
Running in system-mode.                            
                                                   
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
[http://gitlab:80]: 
Please enter the gitlab-ci token for this runner:
[vWtwwQgdPSEzTPNTGZnq]: 
Please enter the gitlab-ci description for this runner:
[docker-runner]: 
Please enter the gitlab-ci tags for this runner (comma separated):
[docker]: 
Registering runner... succeeded                     runner=vWtwwQgd
Please enter the executor: docker, parallels, docker-ssh+machine, virtualbox, docker+machine, kubernetes, docker-ssh, shell, ssh:
[docker]: 
Please enter the default Docker image (e.g. ruby:2.1):
[maven:3.6.1-jdk-11]: 
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! 

• Exit the container using the exit command.
• Verify that contact has been established between GitLab and the GitLab Runner by going back to the Runners page in the GitLab Admin Area and refresh it.
  An entry should appear in the list of runners:

Testing the GitLab Runner

This article would feel unfinished if I hadn’t proved that the GitLab Runner just added indeed does what it is supposed to do, so here we go!

• If you are not logged in to GitLab, then log in.
  If you are already logged in, click the GitLab icon to the left in the GitLab menu bar.
  Both these actions should take you to the GitLab Welcome screen.
• In the + item in the menu bar, select New Project.

• Create a blank project and give it a name.
  I’ll call my project gitlabrunnertest.
• In the new project, click the green button “New file”.

• Name the file .gitlab-ci.yml
  This is the name used for GitLab CI/CD configuration files.
• Paste the following contents into the new file:

image: maven:3.6.1-jdk-11

stages:
  - build

test:
  stage: build
  script:
    - echo "***** Starting build..."
    - echo "*** Checking Maven and Java version:"
    - mvn -version
    - echo "*** Checking environment variables:"
    - env
- echo "***** Build finished!"

• Click the green button Commit changes in the lower left corner of the page.
• In the project’s vertical menu on the left, select CI/CD and then select Pipelines.
  There should already be a pipeline running or perhaps even having finished.

• Click the status button that indicate that the pipeline has passed.

• Click the test step in the pipeline, the one that has a green check-mark next to it.
• There should be a console showed that displays the console output generated during the execution of the step in the job.
  The output should look like this:

Running with gitlab-runner 11.11.2 (ac2a293c)
  on docker-runner eptGNuNk
Using Docker executor with image maven:3.6.1-jdk-11 ...
WARNING: Container based cache volumes creation is disabled. Will not create volume for "/cache"
Pulling docker image maven:3.6.1-jdk-11 ...
Using docker image sha256:067b65acf67aa56715c50b55fd2a590a14e339174a9858f2bbd3e20f0619e94b for maven:3.6.1-jdk-11 ...
Running on runner-eptGNuNk-project-1-concurrent-0 via ccdf9b6763b9...
Initialized empty Git repository in /builds/root/gitlabrunnertest/.git/
Fetching changes...
Created fresh repository.
From http://gitlab:80/root/gitlabrunnertest
 * [new branch]      master     -> origin/master
Checking out 80463704 as master...

Skipping Git submodules setup
$ echo "***** Starting build..."
***** Starting build...
$ echo "*** Checking Maven and Java version:"
*** Checking Maven and Java version:
$ mvn -version
Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 2019-04-04T19:00:29Z)
Maven home: /usr/share/maven
Java version: 11.0.3, vendor: Oracle Corporation, runtime: /usr/local/openjdk-11
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "4.15.0-23-generic", arch: "amd64", family: "unix"
$ echo "*** Checking environment variables:"
*** Checking environment variables:
$ env
CI_COMMIT_SHORT_SHA=80463704
CI_RUNNER_EXECUTABLE_ARCH=linux/amd64
CI_COMMIT_TITLE=Add new file
CI_JOB_TOKEN=[MASKED]
CI_BUILD_REF_NAME=master
CI_REGISTRY_PASSWORD=[MASKED]
CI_RUNNER_TAGS=docker
CI_PIPELINE_URL=http://localhost/root/gitlabrunnertest/pipelines/1
CI_JOB_NAME=test
CI_COMMIT_REF_PROTECTED=true
CI_SERVER_VERSION=11.11.3
CI_PAGES_DOMAIN=example.com
LANG=C.UTF-8
GITLAB_CI=true
CI_SERVER_REVISION=e3eeb779d72
HOSTNAME=runner-eptGNuNk-project-1-concurrent-0
CI_PROJECT_VISIBILITY=private
OLDPWD=/
CI_DISPOSABLE_ENVIRONMENT=true
MAVEN_HOME=/usr/share/maven
CI_COMMIT_SHA=804637044b4af8aa7c497e69e158271f110dfad1
CI_COMMIT_MESSAGE=Add new file
FF_USE_LEGACY_VOLUMES_MOUNTING_ORDER=false
CI_BUILD_STAGE=build
JAVA_HOME=/usr/local/openjdk-11
CI_PROJECT_URL=http://localhost/root/gitlabrunnertest
CI_COMMIT_REF_SLUG=master
CI_SERVER_NAME=GitLab
CI_RUNNER_VERSION=11.11.2
CI_BUILD_NAME=test
CI=true
CI_REGISTRY_USER=gitlab-ci-token
CI_PROJECT_ID=1
CI_PIPELINE_ID=1
JAVA_VERSION=11.0.3
CI_JOB_URL=http://localhost/root/gitlabrunnertest/-/jobs/1
CI_COMMIT_DESCRIPTION=
PWD=/builds/root/gitlabrunnertest
GITLAB_FEATURES=
FF_CMD_DISABLE_DELAYED_ERROR_LEVEL_EXPANSION=false
HOME=/root
CI_CONCURRENT_ID=0
CI_BUILD_TOKEN=[MASKED]
CI_BUILD_ID=1
GITLAB_USER_NAME=Administrator
CI_COMMIT_BEFORE_SHA=0000000000000000000000000000000000000000
CI_PROJECT_PATH_SLUG=root-gitlabrunnertest
CI_CONCURRENT_PROJECT_ID=0
CI_API_V4_URL=http://localhost/api/v4
GITLAB_USER_EMAIL=admin@example.com
CI_COMMIT_REF_NAME=master
CI_SERVER_VERSION_PATCH=3
CI_PIPELINE_IID=1
CI_RUNNER_ID=1
CI_BUILDS_DIR=/builds
CI_SERVER=yes
CI_JOB_ID=1
CI_REPOSITORY_URL=http://gitlab-ci-token:[MASKED]@localhost/root/gitlabrunnertest.git
FF_USE_LEGACY_BUILDS_DIR_FOR_DOCKER=false
GITLAB_USER_LOGIN=root
CI_RUNNER_REVISION=ac2a293c
CI_CONFIG_PATH=.gitlab-ci.yml
CI_PROJECT_NAME=gitlabrunnertest
JAVA_BASE_URL=https://github.com/AdoptOpenJDK/openjdk11-upstream-binaries/releases/download/jdk-11.0.3%2B7/OpenJDK11U-
CI_SERVER_VERSION_MINOR=11
CI_SERVER_VERSION_MAJOR=11
FF_K8S_USE_ENTRYPOINT_OVER_COMMAND=true
CI_NODE_TOTAL=1
FF_DOCKER_HELPER_IMAGE_V2=false
SHLVL=1
JAVA_URL_VERSION=11.0.3_7
CI_RUNNER_DESCRIPTION=docker-runner
CI_PAGES_URL=http://root.example.com/gitlabrunnertest
CI_PROJECT_PATH=root/gitlabrunnertest
GITLAB_USER_ID=1
FF_USE_LEGACY_GIT_CLEAN_STRATEGY=false
CI_BUILD_BEFORE_SHA=0000000000000000000000000000000000000000
CI_BUILD_REF=804637044b4af8aa7c497e69e158271f110dfad1
PATH=/usr/local/openjdk-11/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
CI_PIPELINE_SOURCE=push
CI_PROJECT_NAMESPACE=root
CI_PROJECT_DIR=/builds/root/gitlabrunnertest
CI_JOB_STAGE=build
CI_BUILD_REF_SLUG=master
_=/usr/bin/env
$ echo "***** Build finished!"
***** Build finished!
Job succeeded

Note:

• The job was run on the GitLab Runner with the name docker-runner, using the Docker executor in a container created from the Docker image maven:3.6.1-jdk-11.
• The project was checked out from GitLab.
• The script in the CI/CD configuration file was executed.
• The Maven and Java versions in the container were 3.6.1 and 11.0.3 respectively.
• The environment variables in the job container and their values are listed.

If we finally list the Docker containers on the host using the command sudo docker ps -a we can see that the job container created by the GitLab Runner has been removed, which is what the GitLab Runner was configured to do.

This concludes this article. In the next GitLab-related article, I will set up a CI/CD pipeline to build a Maven-based Java application and a Docker image containing the application.

Happy coding!