Skip to content
Snippets Groups Projects

Gitlab Runner for DUNE

Images

The current images are:

image description
duneci/docker-builder Docker container for building images (privileged)
duneci/proxy HTTP proxy server for restricted internet access

Updating gitlab-runner

To update gitlab-runner on the VM:

./bin/duneci-runner
./bin/duneci-runner-gc

The current version can be shown by running

docker exec gitlab-runner gitlab-runner -v

See the gitlab-runner changelog for a list of changes.

Installing gitlab-runner

To initially install gitlab-runner:

apt install docker.io
mkdir -p /srv/gitlab-runner/config

then follow the steps from Updating gitlab-runner.

Register the runner with GitLab CI:

docker exec -it gitlab-runner gitlab-runner register

Finally edit /srv/gitlab-runner/config/config.toml:

concurrent = 4
check_interval = 0

[[runners]]
  name = "shared-ci@tu-dresden.de"
  url = "https://gitlab.dune-project.org"
  id = 0
  token = "[...]"
  token_obtained_at = 0001-01-01T00:00:00Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "docker"

  # Set proxy variables if needed:
  environment = ["DUNECI_PARALLEL=4", "ftp_proxy=http://dune-proxy:3128", "http_proxy=http://dune-proxy:3128", "https_proxy=http://dune-proxy:3128", "no_proxy=127.0.0.1, localhost"]

  [runners.docker]
    tls_verify = false
    image = "duneci/dune:latest"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    security_opt = ["no-new-privileges"]
    disable_cache = true
    # See [Proxy setup](#proxy-setup) below:
    network_mode = "gitlab-ci-dune"
    allowed_images = ["duneci/*", "docker.io/duneci/*", "docker.io/fsfe/reuse:*", "registry.dune-project.org/docker/ci/*"]
    pull_policy = ["always"]
    shm_size = 10737418240
    network_mtu = 0
    cpus = "6"
    memory = "32G"

    # No longer used:
    # OpenMPI-2 is unhappy with the (too long) default hostnames:
    #hostname = "ci"
    #allowed_services = []

See the documentation of GitLab runner's configuration for details. Please also keep the security considerations in mind.

An encrypted version of the live configuration can be found in config/gitlab-runner.

Proxy setup

Initial setup and updates:

./bin/duneci-proxy gitlab-ci-dune dune-proxy
./bin/duneci-proxy gitlab-ci-fu fu-proxy

In gitlab-runner's config.toml:

[[runners]]
  [runners.docker]
    network_mode = "gitlab-ci-dune"

This sets up a container dune-proxy which is part of two networks (the default bridge and gitlab-ci-dune) running a squid proxy configured to filter requests. The actual builds are only in the gitlab-ci-dune network and can only access the internet via the filtering proxy.

See the script duneci-proxy for details of the setup, and config/dune-proxy/squid.conf and config/fu-proxy/squid.conf for the proxy configuration.