README.md 3.14 KB
Newer Older
Ansgar Burchardt's avatar
Ansgar Burchardt committed
1
2
Gitlab Runner for DUNE
======================
Ansgar Burchardt's avatar
Ansgar Burchardt committed
3
4
5
6
7
8

Images
------

The current images are:

Ansgar Burchardt's avatar
Ansgar Burchardt committed
9
10
11
12
| image                    | description                                       |
|--------------------------|---------------------------------------------------|
| duneci/docker-builder    | Docker container for building images (privileged) |
| duneci/proxy             | HTTP proxy server for restricted internet access  |
13

14
15
16
17
18
Updating gitlab-runner
----------------------

To update `gitlab-runner` on the VM:

Ansgar Burchardt's avatar
Ansgar Burchardt committed
19
```shell
20
./bin/duneci-runner
Ansgar Burchardt's avatar
Ansgar Burchardt committed
21
./bin/duneci-runner-gc
22
```
23
24
25

The current version can be shown by running

Ansgar Burchardt's avatar
Ansgar Burchardt committed
26
```shell
27
docker exec gitlab-runner gitlab-runner -v
28
```
29

30
31
See the [gitlab-runner changelog][] for a list of changes.

32
  [gitlab-runner changelog]: https://gitlab.com/gitlab-org/gitlab-runner/blob/master/CHANGELOG.md
33

34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
Installing gitlab-runner
------------------------

To initially install gitlab-runner:

```shell
apt install docker.io
mkdir -p /srv/gitlab-runner/config
```
then follow the steps from [Updating gitlab-runner](#updating-gitlab-runner).

Register the runner with GitLab CI:
```shell
docker exec -it gitlab-runner gitlab-runner register
```

Finally edit `/srv/gitlab-runner/config/config.toml`:
```TOML
concurrent = 4
check_interval = 0

[[runners]]
  name = "<name>"
  url = "https://gitlab.dune-project.org/ci"
  token = "<private token from registration>"
  executor = "docker"

  # Set proxy variables if needed:
Ansgar Burchardt's avatar
Ansgar Burchardt committed
62
  environment = ["ftp_proxy=http://dune-proxy:3128", "http_proxy=http://dune-proxy:3128", "https_proxy=http://dune-proxy:3128", "no_proxy=127.0.0.1, localhost"]
63
64
65
66
67
68
69
  [runners.docker]
    # tls_verify = false
    image = "duneci/dune:latest"
    privileged = false
    security_opt = ["no-new-privileges"]
    disable_cache = false
    volumes = ["/cache"]
70
    allowed_images = ["docker.io/duneci/*", "duneci/*"]
71
    allowed_services = []
72
    pull_policy = "if-not-present"
Ansgar Burchardt's avatar
Ansgar Burchardt committed
73
74
    # See [Proxy setup](#proxy-setup) below:
    network_mode = "gitlab-ci-dune"
75
76
77
78
79
80

    # OpenMPI-2 is unhappy with the (too long) default hostnames:
    hostname = "ci"
```
See the [documentation of GitLab runner's configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html) for details.
Please also keep the [security considerations](https://docs.gitlab.com/runner/security/index.html) in mind.
Ansgar Burchardt's avatar
Ansgar Burchardt committed
81

82
83
84
An encrypted version of the live configuration can be found in
[config/gitlab-runner](config/gitlab-runner).

Ansgar Burchardt's avatar
Ansgar Burchardt committed
85
86
87
Proxy setup
-----------

88
Initial setup and updates:
Ansgar Burchardt's avatar
Ansgar Burchardt committed
89
```shell
90
91
./bin/duneci-proxy gitlab-ci-dune dune-proxy
./bin/duneci-proxy gitlab-ci-fu fu-proxy
Ansgar Burchardt's avatar
Ansgar Burchardt committed
92
93
94
95
96
97
98
99
100
```

In gitlab-runner's `config.toml`:

```TOML
[[runners]]
  [runners.docker]
    network_mode = "gitlab-ci-dune"
```
101
102
103
104
105
106

This sets up a container `dune-proxy` which is part of two networks
(the default bridge and `gitlab-ci-dune`) running a squid proxy
configured to filter requests.  The actual builds are only in the
`gitlab-ci-dune` network and can only access the internet via the
filtering proxy.
107

108
109
110
111
See the script [duneci-proxy](bin/duneci-proxy) for details of the
setup, and [config/dune-proxy/squid.conf](config/dune-proxy/squid.conf)
and [config/fu-proxy/squid.conf](config/fu-proxy/squid.conf) for the proxy
configuration.