README.md 5.12 KB
Newer Older
Ansgar Burchardt's avatar
Ansgar Burchardt committed
1 2 3 4 5 6 7 8
Docker images for GitLab CI
===========================

Images
------

The current images are:

9 10 11 12
| image                    | parent            | description                                                |
|--------------------------|-------------------|------------------------------------------------------------|
| duneci/base:8            | debian:8          | Debian 8 with gcc 4.9.2, clang 3.5, cmake 3.0.2            |
| duneci/base:8-backports  | duneci/base:8     | Debian 8 with gcc 4.9.2, clang 3.8 (backport), cmake 3.0.2 |
Ansgar Burchardt's avatar
Ansgar Burchardt committed
13
| duneci/base:9            | debian:9          | Debian 9 with gcc 6.3, clang 3.8, cmake 3.7                |
14
| duneci/base:10           | debian:10         | Debian 9 with gcc 7, clang 4.0, cmake 3.7                  |
15
| duneci/base:16.04        | ubuntu:16.04      | Ubuntu LTS 16.04 with gcc 5.4.0, clang 3.8.0, cmake 3.5.1  |
16 17
| duneci/dune:2.3          | duneci/base:8     | DUNE 2.3 core modules (Debian packages)                    |
| duneci/dune:2.4          | duneci/base:16.04 | DUNE 2.4 core modules (Debian packages)                    |
18
| duneci/dune:2.5          | duneci/base:9     | DUNE 2.5 core and staging modules (Debian packages)        |
19
| duneci/dune:2.6          | duneci/base:10    | DUNE 2.6 core and staging modules (Git 2.6 branch)         |
20
| duneci/dune:git          | duneci/base:9     | DUNE master core and staging modules (Git)                 |
21 22
| duneci/dune-fufem:2.4    | duneci/dune:2.4   | dune-{fufem,functions,solvers,typetree} (2.4 branch)       |
| duneci/dune-fufem:git    | duneci/dune:git   | dune-{elasticity,fufem,grid-glue,solvers} (master branch)  |
Ansgar Burchardt's avatar
Ansgar Burchardt committed
23 24 25 26

`.gitlab-ci.yml`
----------------

27
Installing dependencies:
28
```yaml
29 30 31 32 33
before_script:
  - duneci-install-module https://gitlab.dune-project.org/core/dune-common.git
  - duneci-install-module https://gitlab.dune-project.org/core/dune-geometry.git
```

Ansgar Burchardt's avatar
Ansgar Burchardt committed
34
To build with several images:
35
```yaml
Ansgar Burchardt's avatar
Ansgar Burchardt committed
36
---
37 38
dune:2.4--gcc:
  image: duneci/dune:2.4
39
  script: duneci-standard-test
Ansgar Burchardt's avatar
Ansgar Burchardt committed
40

41 42
dune:2.4--clang:
  image: duneci/dune:2.4
43
  script: duneci-standard-test --opts=/duneci/opts.clang
Ansgar Burchardt's avatar
Ansgar Burchardt committed
44 45 46 47
```

You can also specify a default image and use it in several jobs:

48
```yaml
49
---
Ansgar Burchardt's avatar
Ansgar Burchardt committed
50 51 52
image: duneci/dune:2.4

dune:2.4--gcc:
53
  script: duneci-standard-test
Ansgar Burchardt's avatar
Ansgar Burchardt committed
54 55

dune:2.4--clang:
56
  script: duneci-standard-test --opts=/duneci/opts.clang
Ansgar Burchardt's avatar
Ansgar Burchardt committed
57
```
58

59 60
For more information, take a look at the [GitLab documentation on `.gitlab-ci.yml`](https://docs.gitlab.com/ce/ci/yaml/README.html).

61 62 63 64 65
Updating gitlab-runner
----------------------

To update `gitlab-runner` on the VM:

66
```shell
67
./bin/duneci-runner
68
```
69 70 71

The current version can be shown by running

72
```shell
73
docker exec gitlab-runner gitlab-runner -v
74
```
75

76 77
See the [gitlab-runner changelog][] for a list of changes.

78
  [gitlab-runner changelog]: https://gitlab.com/gitlab-org/gitlab-runner/blob/master/CHANGELOG.md
79

80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107
Installing gitlab-runner
------------------------

To initially install gitlab-runner:

```shell
apt install docker.io
mkdir -p /srv/gitlab-runner/config
```
then follow the steps from [Updating gitlab-runner](#updating-gitlab-runner).

Register the runner with GitLab CI:
```shell
docker exec -it gitlab-runner gitlab-runner register
```

Finally edit `/srv/gitlab-runner/config/config.toml`:
```TOML
concurrent = 4
check_interval = 0

[[runners]]
  name = "<name>"
  url = "https://gitlab.dune-project.org/ci"
  token = "<private token from registration>"
  executor = "docker"

  # Set proxy variables if needed:
Ansgar Burchardt's avatar
Ansgar Burchardt committed
108
  environment = ["ftp_proxy=http://dune-proxy:3128", "http_proxy=http://dune-proxy:3128", "https_proxy=http://dune-proxy:3128", "no_proxy=127.0.0.1, localhost"]
109 110 111 112 113 114 115
  [runners.docker]
    # tls_verify = false
    image = "duneci/dune:latest"
    privileged = false
    security_opt = ["no-new-privileges"]
    disable_cache = false
    volumes = ["/cache"]
116
    allowed_images = ["docker.io/duneci/*", "duneci/*"]
117
    allowed_services = []
118
    pull_policy = "if-not-present"
Ansgar Burchardt's avatar
Ansgar Burchardt committed
119 120
    # See [Proxy setup](#proxy-setup) below:
    network_mode = "gitlab-ci-dune"
121 122 123 124 125 126

    # OpenMPI-2 is unhappy with the (too long) default hostnames:
    hostname = "ci"
```
See the [documentation of GitLab runner's configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html) for details.
Please also keep the [security considerations](https://docs.gitlab.com/runner/security/index.html) in mind.
Ansgar Burchardt's avatar
Ansgar Burchardt committed
127

128 129 130
An encrypted version of the live configuration can be found in
[config/gitlab-runner](config/gitlab-runner).

Ansgar Burchardt's avatar
Ansgar Burchardt committed
131 132 133
Proxy setup
-----------

134
Initial setup and updates:
Ansgar Burchardt's avatar
Ansgar Burchardt committed
135
```shell
136 137
./bin/duneci-proxy gitlab-ci-dune dune-proxy
./bin/duneci-proxy gitlab-ci-fu fu-proxy
Ansgar Burchardt's avatar
Ansgar Burchardt committed
138 139 140 141 142 143 144 145 146
```

In gitlab-runner's `config.toml`:

```TOML
[[runners]]
  [runners.docker]
    network_mode = "gitlab-ci-dune"
```
147 148 149 150 151 152

This sets up a container `dune-proxy` which is part of two networks
(the default bridge and `gitlab-ci-dune`) running a squid proxy
configured to filter requests.  The actual builds are only in the
`gitlab-ci-dune` network and can only access the internet via the
filtering proxy.
153

154 155 156 157
See the script [duneci-proxy](bin/duneci-proxy) for details of the
setup, and [config/dune-proxy/squid.conf](config/dune-proxy/squid.conf)
and [config/fu-proxy/squid.conf](config/fu-proxy/squid.conf) for the proxy
configuration.