In this article I will show how I added a GitLab runner with a Docker executor to my original GitLab CE setup. This type of GitLab runner will allow me to run GitLab jobs in containers created from Docker images.
Almost a year ago I
wrote an article about running GitLab
CE in a Docker container. At that time I promised that I would
return with more GitLab related articles and now the time has come.
Before continuing, I have to note that this article is more in
the line of a personal notebook. There are far better articles out
there if you want a configuration that is more appropriate for a
production environment with multiple users, full-blown security etc.
Motivation
Some of the reason for me wanting to run GitLab jobs in Docker containers are:
- Allows to run steps in a build in parallel without the risk of one build step affecting another.
In my case this means for example being able to run tests in one container while running, for instance, static code analysis in another. - Controlled and reproducible build environment.
The build environment, that is the container in which the build job is performed, is created immediately prior to the job being run and discarded immediately afterwards. - Easy configuration of the build environment.
In my case this means allowing me to use different versions of Java and Maven when building. It is even possible to build against multiple Java versions in one and the same GitLab CI/CD pipeline.
In an organization with multiple developers, this would mean letting the developers decide which Docker image to use for a job and not having to maintain a build server to any larger extent than maintaining a GitLab runner with a Docker executor.
Prerequisites
The Docker host I am
using is running Ubuntu 18.04 and have Docker version 18.09 as well
as Docker Compose version 1.25 installed.
The GitLab
Runner documentation states that the minimum Docker version
required is version 1.5.0.
Updated Docker Compose Configuration
I started out with
the Docker Compose configuration from the previous article and added
a service for the GitLab Runner, a volume for the GitLab Runner’s
configuration and a network to be shared by GitLab, the GiLab
Runner(s) and the job containers allowing the GitLab Runner and job
containers to communicate with GitLab.
I recommend creating a
directory for the GitLab stack and then creating a docker-compose.yml
in the directory with the following contents:
# GitLab CE deployment with one GitLab Runner. # version: '3' services: # GitLab service. # Change the hostname value in the configuration below to match the name of # your server/computer on which GitLab is to be run. # Once started, access GitLab using the URL http://[hostname]:8880. # The logs directory can be mapped to a logs directory in the same director # as the docker-compose file using the following entry under volumes: # - ./logs:/var/log/gitlab gitlab: image: gitlab/gitlab-ce:latest hostname: localhost environment: GITLAB_OMNIBUS_CONFIG: | gitlab_rails['gitlab_shell_ssh_port'] = 8822 # The following ports are exposed by GitLab: # 8880 (HTTP) # 443 (if HTTPS configured) # 8080 (used by Unicorn) # 8822 (used by the SSH daemon) ports: - "443:443" - "8880:80" - "8822:22" # Volumes to store GitLab's data and configuration on. volumes: - gitlab-config:/etc/gitlab - gitlab-data:/var/opt/gitlab restart: always networks: gitlabnetwork: {} # GitLab Runner service. gitlab-runner: image: gitlab/gitlab-runner:latest ports: - "8093:8093" # Volume to store the GitLab Runner's configuration on and a # volume that allows the runner to access the host's Docker socket. volumes: - /var/run/docker.sock:/var/run/docker.sock - gitlab-runner-config:/etc/gitlab-runner restart: always networks: gitlabnetwork: {} # Use an attachable network shared by GitLab, the GitLab runner and # all the job containers. networks: gitlabnetwork: driver: overlay # Volumes for GitLab's data and configuration as well as # a volume for the GitLab runner's configuration. volumes: gitlab-data: gitlab-config: gitlab-runner-config:
Comments:
- The first part is the GitLab service as described in the previous article with the addition of a network configuration.
The network is named gitlabnetwork. Details on the network below! - The second service is the GitLab Runner service (named gitlab-runner).
- The GitLab Runner service exposes port 8093.
This port is used by the GitLab Runner’s session server that allows a user to interact with jobs created by the runner. - The GitLab Runner service has two volumes.
The first volume exposes the host’s Docker socket to the GitLab Runner’s container, allowing the runner to interact with the host’s Docker service.
The second volume is used to store the GitLab Runner’s configuration on. - The GitLab Runner service joins the same network as GitLab.
The network is named gitlabnetwork. Details on the network below! - A network named gitlabnetwork is declared.
This network will be used for communication between the GitLab instance, the GitLab Runner and the job containers that the GitLab Runner will start.
The network is an overlay network. This is in order to ensure that the network is attachable.
The network need to be attachable, which is the default for overlay networks, in order for the job containers to be able to join the network when they are started. - Three volumes are declared.
The first two volumes are for GitLab data and configuration, as seen in my previous article. The third volume is for the GitLab Runner’s configuration.
Starting the GitLab Stack
We are now ready to start the GitLab stack.
- Open a terminal window.
- Go to the GitLab stack directory.
-
Start the GitLab stack using the following command:
sudo docker-compose up -d
Output similar to the following should appear in the terminal window:
Creating network "gitlabce_gitlabnetwork" with driver "overlay" Creating volume "gitlabce_gitlab-data" with default driver Creating volume "gitlabce_gitlab-config" with default driver Creating volume "gitlabce_gitlab-runner-config" with default driver Creating gitlabce_gitlab_1 ... done Creating gitlabce_gitlab-runner_1 ... done
-
Wait a few minutes and then issue the following command to
check the status of the GitLab stack:
sudo docker ps -a
Output similar to the following should appear in the terminal window:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 169de81569dc gitlab/gitlab-ce:latest "/assets/wrapper" 6 minutes ago Up 6 minutes (healthy) 0.0.0.0:443->443/tcp, 0.0.0.0:8822->22/tcp, 0.0.0.0:8880->80/tcp gitlabce_gitlab_1 ccdf9b6763b9 gitlab/gitlab-runner:latest "/usr/bin/dumb-init …" 6 minutes ago Up 6 minutes 0.0.0.0:8093->8093/tcp gitlabce_gitlab-runner_1
-
Note the name of the GitLab Runner container.
In the above example, it is gitlabce_gitlab-runner_1.
If all goes according to plan, both the GitLab and the GitLab Runner containers should be up. We can now check that the GitLab web UI is up and running:
- Open the URL http://localhost:8880 in a browser.
You should be presented with a request to create a new password for your new account.
I will enter “password” in both the fields for the sake of this example.
Having successfully created the new password, you will then be directed to the login page.
Obtaining a GitLab Runner Registration Token
GitLab will not automatically be aware of new GitLab Runners until they have been registered. Prior to registering a GitLab Runner, a runner registration token must be obtained from GitLab. As of writing this article, there is no good way to automatically obtain such a token known to me
-
Log in using the user name “root” and the password
“password” (or the password you entered when creating a new
password).
Upon successful login, the GitLab Welcome page should be shown.

-
Click the wrench in the GitLab menu bar.
This will take you to the Admin Area landing page.

-
Click the Runners entry in the Admin Area menu on the
left.
The Runners page on which all runners known to this GitLab instance are listed.
For obvious reasons there are no runners listed on this page, since this is a new GitLab instance.

-
On the right, under the “Set up a shared Runner manually”
title copy the registration token in red.
This is the token needed to register our GitLab Runner with GitLab.
Registering and Configuring the GitLab Runner
With the name of the container in which the GitLab Runner is running (gitlabce_gitlab-runner_1 in my case) and the registration token obtained in the previous step the GitLab Runner can now be registered. If you add more GitLab Runners then this registration procedure has to be repeated once for each runner.
- Open a terminal window or re-use an already open terminal window.
This window has to be opened on the Docker host on which the GitLab Runner container is located. - If you do not know the name of the container in which the GitLab Runner is running, issue the following command to obtain the container name:
sudo docker ps -a
- Enter the container by executing the following command in the terminal window, replacing the container name if it is different:
sudo docker exec -it gitlabce_gitlab-runner_1 bash
- While in the GitLab Runner container, try the following commands which provide valuable help when configuring GitLab Runners:
gitlab-runner --help gitlab-runner register --help
- In the container execute the following command to register and configure the GitLab Runner. Replace the registration token value with the registration token value you obtained earlier.
gitlab-runner register \ --executor="docker" \ --custom_build_dir-enabled="true" \ --docker-image="maven:3.6.1-jdk-11" \ --url="http://gitlab:80" \ --clone-url="http://gitlab:80" \ --registration-token="vWtwwQgdPSEzTPNTGZnq" \ --description="docker-runner" \ --tag-list="docker" \ --run-untagged="true" \ --locked="false" \ --docker-network-mode="gitlabce_gitlabnetwork" \ --docker-disable-cache="true" \ --docker-privileged="true" \ --cache-dir="/cache" \ --builds-dir="/builds" \ --docker-volumes="gitlab-runner-builds:/builds" \ --docker-volumes="gitlab-runner-cache:/cache"
Comments on the configuration options:
- –executor=”docker”
Configures GitLab Runner jobs to be run in containers. - –custom_build_dir-enabled
Enables job-specific build directories. - –docker-image=”maven:3.6.1-jdk-11″
The default Docker image to use when creating a container to run a job in if none is specified in the GitLab CI/CD configuration. I have chosen a Maven Docker image, since I will mainly build my projects with Maven. - –url=”http://gitlab:80″
The GitLab URL. Will be used as a base URL for the GitLab API. Note that the “address” is gitlab, which is the name of the service in the Docker Compose file. The port is 80 which is the port on which GitLab is listening inside the container and thus also in the gitlabnetwork network (and not the port in the host to which port 80 is mapped, which is 8880). - –clone-url=”http://gitlab:80″
Base URL used when checking out projects from GitLab job containers. Uses the “gitlab” service name that is only reachable within the gitlabnetwork Docker network. - –registration-token=”vWtwwQgdPSEzTPNTGZnq”
GitLab Runner registration token as obtained from GitLab. - –description=”docker-runner”
Arbitrary description of the GitLab Runner. - –tag-list=”docker”
One or more tags that can be used to associate a GitLab project with one or more GitLab Runners so that the project will be built by the runners with the matching tag(s). - –run-untagged=”true”
Allow the runner to run jobs without tags. - –locked=”false”
Do not lock the GitLab Runner to the current project only. - –docker-network-mode=”gitlabce_gitlabnetwork”
Ensures that GitLab job containers join the same network as GitLab in order to be able to check out projects to build from GitLab. - –docker-disable-cache=”true”
Causes GitLab job containers to be removed immediately after job finished executing.
Given that my system is a laptop and thus have limited disk resources, I have chosen to enable this option. You may want to remove this if you want job containers to remain in existence after job execution has completed, in which case you may want to consider some other alternative to clean up old containers. - –docker-privileged=”true”
Give extended privileges to job containers. This may incur security risks. Please refer to the Docker documentation for details. - –cache-dir=”/cache”
Directory where build cache is stored in the job containers. Set to ensure that the builds cache is written to a Docker volume – see below. - –builds-dir=”/builds”
Directory where builds are stored in the job containers. Set to ensure that the builds directory is written to a Docker volume – see below. - –docker-volumes=”gitlab-runner-builds:/builds”
Maps the builds directory in the job containers spawned from this runner to the Docker volume “gitlab-runner-builds”.
The volume will be created first time it is used, if it does not already exist. - –docker-volumes=”gitlab-runner-cache:/cache”
Maps the builds cache directory in the job containers spawned from this runner to the Docker volume “gitlab-runner-cache”. The volume will be created first time it is used, if it does not already exist.
Using Docker volumes to store cache and build directories in ensures that caching will be preserved even though job containers are removed. In addition, Docker volumes are more visible than shared directories on the host computer and can be easily managed.
Having executed the command you will be asked a number of questions regarding the configuration of the GitLab Runner. The default values will be taken from the provided parameters, so just press enter on each of the questions. The console output will look similar to this if the registration was successful:
Runtime platform arch=amd64 os=linux pid=39 revision=ac2a293c version=11.11.2 Running in system-mode. Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/): [http://gitlab:80]: Please enter the gitlab-ci token for this runner: [vWtwwQgdPSEzTPNTGZnq]: Please enter the gitlab-ci description for this runner: [docker-runner]: Please enter the gitlab-ci tags for this runner (comma separated): [docker]: Registering runner... succeeded runner=vWtwwQgd Please enter the executor: docker, parallels, docker-ssh+machine, virtualbox, docker+machine, kubernetes, docker-ssh, shell, ssh: [docker]: Please enter the default Docker image (e.g. ruby:2.1): [maven:3.6.1-jdk-11]: Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!
- Exit the container using the exit command.
-
Verify that contact has been established between GitLab and
the GitLab Runner by going back to the Runners page in the GitLab
Admin Area and refresh it.
An entry should appear in the list of runners:

Testing the GitLab Runner
This article would feel unfinished if I hadn’t proved that the GitLab Runner just added indeed does what it is supposed to do, so here we go!
-
If you are not logged in to GitLab, then log in.
If you are already logged in, click the GitLab icon to the left in the GitLab menu bar.
Both these actions should take you to the GitLab Welcome screen. - In the + item in the menu bar, select New Project.

-
Create a blank project and give it a name.
I’ll call my project gitlabrunnertest. - In the new project, click the green button “New file”.

-
Name the file .gitlab-ci.yml
This is the name used for GitLab CI/CD configuration files. - Paste the following contents into the new file:
image: maven:3.6.1-jdk-11 stages: - build test: stage: build script: - echo "***** Starting build..." - echo "*** Checking Maven and Java version:" - mvn -version - echo "*** Checking environment variables:" - env - echo "***** Build finished!"
- Click the green button Commit changes in the lower left corner of the page.
-
In the project’s vertical menu on the left, select CI/CD
and then select Pipelines.
There should already be a pipeline running or perhaps even having finished.

- Click the status button that indicate that the pipeline has passed.

- Click the test step in the pipeline, the one that has a green check-mark next to it.
- There should be a console showed that displays the console output generated during the execution of the step in the job.
The output should look like this:
Running with gitlab-runner 11.11.2 (ac2a293c) on docker-runner eptGNuNk Using Docker executor with image maven:3.6.1-jdk-11 ... WARNING: Container based cache volumes creation is disabled. Will not create volume for "/cache" Pulling docker image maven:3.6.1-jdk-11 ... Using docker image sha256:067b65acf67aa56715c50b55fd2a590a14e339174a9858f2bbd3e20f0619e94b for maven:3.6.1-jdk-11 ... Running on runner-eptGNuNk-project-1-concurrent-0 via ccdf9b6763b9... Initialized empty Git repository in /builds/root/gitlabrunnertest/.git/ Fetching changes... Created fresh repository. From http://gitlab:80/root/gitlabrunnertest * [new branch] master -> origin/master Checking out 80463704 as master... Skipping Git submodules setup $ echo "***** Starting build..." ***** Starting build... $ echo "*** Checking Maven and Java version:" *** Checking Maven and Java version: $ mvn -version Apache Maven 3.6.1 (d66c9c0b3152b2e69ee9bac180bb8fcc8e6af555; 2019-04-04T19:00:29Z) Maven home: /usr/share/maven Java version: 11.0.3, vendor: Oracle Corporation, runtime: /usr/local/openjdk-11 Default locale: en, platform encoding: UTF-8 OS name: "linux", version: "4.15.0-23-generic", arch: "amd64", family: "unix" $ echo "*** Checking environment variables:" *** Checking environment variables: $ env CI_COMMIT_SHORT_SHA=80463704 CI_RUNNER_EXECUTABLE_ARCH=linux/amd64 CI_COMMIT_TITLE=Add new file CI_JOB_TOKEN=[MASKED] CI_BUILD_REF_NAME=master CI_REGISTRY_PASSWORD=[MASKED] CI_RUNNER_TAGS=docker CI_PIPELINE_URL=http://localhost/root/gitlabrunnertest/pipelines/1 CI_JOB_NAME=test CI_COMMIT_REF_PROTECTED=true CI_SERVER_VERSION=11.11.3 CI_PAGES_DOMAIN=example.com LANG=C.UTF-8 GITLAB_CI=true CI_SERVER_REVISION=e3eeb779d72 HOSTNAME=runner-eptGNuNk-project-1-concurrent-0 CI_PROJECT_VISIBILITY=private OLDPWD=/ CI_DISPOSABLE_ENVIRONMENT=true MAVEN_HOME=/usr/share/maven CI_COMMIT_SHA=804637044b4af8aa7c497e69e158271f110dfad1 CI_COMMIT_MESSAGE=Add new file FF_USE_LEGACY_VOLUMES_MOUNTING_ORDER=false CI_BUILD_STAGE=build JAVA_HOME=/usr/local/openjdk-11 CI_PROJECT_URL=http://localhost/root/gitlabrunnertest CI_COMMIT_REF_SLUG=master CI_SERVER_NAME=GitLab CI_RUNNER_VERSION=11.11.2 CI_BUILD_NAME=test CI=true CI_REGISTRY_USER=gitlab-ci-token CI_PROJECT_ID=1 CI_PIPELINE_ID=1 JAVA_VERSION=11.0.3 CI_JOB_URL=http://localhost/root/gitlabrunnertest/-/jobs/1 CI_COMMIT_DESCRIPTION= PWD=/builds/root/gitlabrunnertest GITLAB_FEATURES= FF_CMD_DISABLE_DELAYED_ERROR_LEVEL_EXPANSION=false HOME=/root CI_CONCURRENT_ID=0 CI_BUILD_TOKEN=[MASKED] CI_BUILD_ID=1 GITLAB_USER_NAME=Administrator CI_COMMIT_BEFORE_SHA=0000000000000000000000000000000000000000 CI_PROJECT_PATH_SLUG=root-gitlabrunnertest CI_CONCURRENT_PROJECT_ID=0 CI_API_V4_URL=http://localhost/api/v4 GITLAB_USER_EMAIL=admin@example.com CI_COMMIT_REF_NAME=master CI_SERVER_VERSION_PATCH=3 CI_PIPELINE_IID=1 CI_RUNNER_ID=1 CI_BUILDS_DIR=/builds CI_SERVER=yes CI_JOB_ID=1 CI_REPOSITORY_URL=http://gitlab-ci-token:[MASKED]@localhost/root/gitlabrunnertest.git FF_USE_LEGACY_BUILDS_DIR_FOR_DOCKER=false GITLAB_USER_LOGIN=root CI_RUNNER_REVISION=ac2a293c CI_CONFIG_PATH=.gitlab-ci.yml CI_PROJECT_NAME=gitlabrunnertest JAVA_BASE_URL=https://github.com/AdoptOpenJDK/openjdk11-upstream-binaries/releases/download/jdk-11.0.3%2B7/OpenJDK11U- CI_SERVER_VERSION_MINOR=11 CI_SERVER_VERSION_MAJOR=11 FF_K8S_USE_ENTRYPOINT_OVER_COMMAND=true CI_NODE_TOTAL=1 FF_DOCKER_HELPER_IMAGE_V2=false SHLVL=1 JAVA_URL_VERSION=11.0.3_7 CI_RUNNER_DESCRIPTION=docker-runner CI_PAGES_URL=http://root.example.com/gitlabrunnertest CI_PROJECT_PATH=root/gitlabrunnertest GITLAB_USER_ID=1 FF_USE_LEGACY_GIT_CLEAN_STRATEGY=false CI_BUILD_BEFORE_SHA=0000000000000000000000000000000000000000 CI_BUILD_REF=804637044b4af8aa7c497e69e158271f110dfad1 PATH=/usr/local/openjdk-11/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin CI_PIPELINE_SOURCE=push CI_PROJECT_NAMESPACE=root CI_PROJECT_DIR=/builds/root/gitlabrunnertest CI_JOB_STAGE=build CI_BUILD_REF_SLUG=master _=/usr/bin/env $ echo "***** Build finished!" ***** Build finished! Job succeeded
Note:
- The job was run on the GitLab Runner with the name docker-runner, using the Docker executor in a container created from the Docker image maven:3.6.1-jdk-11.
- The project was checked out from GitLab.
- The script in the CI/CD configuration file was executed.
- The Maven and Java versions in the container were 3.6.1 and 11.0.3 respectively.
- The environment variables in the job container and their values are listed.
If we finally list the Docker containers on the host using the command sudo docker ps -a we can see that the job container created by the GitLab Runner has been removed, which is what the GitLab Runner was configured to do.
This concludes this article. In the next GitLab-related article, I will set up a CI/CD pipeline to build a Maven-based Java application and a Docker image containing the application.
Happy coding!
Just wanted to say thanks for such a brilliant article. I’ve been tearing my hair out all night trying to get a docker in docker runner/executor going, annoyingly I’d very almost solved it before I stumbled upon your article which would have solved it for me in about 5 minutes! Nonetheless, really good – thanks for sharing.