From efa660abab54724115787ffc7a1a445a14d1231c Mon Sep 17 00:00:00 2001 From: julien Date: Mon, 4 May 2026 15:55:17 +0200 Subject: [PATCH] chore(infra): pin act_runner image pull policy (#10) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary `act_runner`'s default `container.force_pull: true` re-issues a `docker pull` at the start of every job, adding 10–30 s of registry round-trip even when every layer is already locally cached. With job images pinned to specific tags (`catthehacker/ubuntu:act-22.04` and `:full-22.04`), the implicit pull is pure overhead and contradicts the deliberate-upgrade policy ADR-0015 spells out for the runner image. - Add `infra/runner-config.yaml` with `container.force_pull: false`. - Mount it read-only into all three runners and point each at it via `CONFIG_FILE=/etc/runner/config.yaml`. - Document the pre-pull procedure and image-upgrade playbook in `infra/README.md` → "Job image pinning and pre-pull". - Fold the pre-pull into the "First-time registration" walkthrough so a fresh setup is correct end-to-end. The trade-off: the runner host must hold the images locally before the runner is asked to use them. Documented. ## Roll-out (manual, on the runner host) ```bash cd infra/ # 1. Pre-pull the job images (one-shot — pays the cold cost once). docker pull catthehacker/ubuntu:act-22.04 docker pull catthehacker/ubuntu:full-22.04 # 2. Recreate the runners so the new mount + env var take effect. docker compose -f ci-runners.compose.yml up -d --force-recreate --------- Co-authored-by: Julien Gautier Reviewed-on: https://git.unespace.com/julien/apf_portal/pulls/10 --- infra/README.md | 39 +++++++++++++++++++++++++++++++++--- infra/ci-runners.compose.yml | 13 ++++++++++++ infra/runner-config.yaml | 16 +++++++++++++++ 3 files changed, 65 insertions(+), 3 deletions(-) create mode 100644 infra/runner-config.yaml diff --git a/infra/README.md b/infra/README.md index ffa6410..4f4d7ac 100644 --- a/infra/README.md +++ b/infra/README.md @@ -5,6 +5,7 @@ Infrastructure-as-code artefacts for the project. Separate from application code | Subject | File / Folder | ADR / Reference | | -------------------------------------- | -------------------------------------------------- | ------------------------------------------------------------------- | | Self-hosted CI runners (Gitea Actions) | [`ci-runners.compose.yml`](ci-runners.compose.yml) | [ADR-0015 §"Runners"](../docs/decisions/0015-cicd-gitea-actions.md) | +| Shared `act_runner` configuration | [`runner-config.yaml`](runner-config.yaml) | [ADR-0015 §"Runners"](../docs/decisions/0015-cicd-gitea-actions.md) | | Runtime state of the runners | `data/` (git-ignored after `.gitignore`) | — | | Env-vars template for the runners | `.env.example` (`.env` is git-ignored) | — | @@ -35,10 +36,15 @@ $EDITOR .env # Set GITEA_INSTANCE_URL (https, no trailing slash) and # GITEA_RUNNER_REGISTRATION_TOKEN. -# 3. Bring the runners up. +# 3. Pre-pull the job images so the runner doesn't have to (see +# "Job image pinning and pre-pull" below for the rationale). +docker pull catthehacker/ubuntu:act-22.04 +docker pull catthehacker/ubuntu:full-22.04 + +# 4. Bring the runners up. docker compose -f ci-runners.compose.yml up -d -# 4. Verify in Gitea: the three runners appear as online with the +# 5. Verify in Gitea: the three runners appear as online with the # self-hosted, on-prem labels. If a runner doesn't come online, # inspect its logs: docker compose -f ci-runners.compose.yml logs runner-1 @@ -75,7 +81,7 @@ When this is reactivated, the right fix is one of: Either path needs a single-runner spike before being rolled out to the three. -### Image pinning +### `act_runner` image pinning The compose pins `gitea/act_runner:0.2.13`. Update the pin deliberately, not via `:latest`: @@ -86,6 +92,33 @@ The compose pins `gitea/act_runner:0.2.13`. Update the pin deliberately, not via The matching CI workflows refer to runner _labels_ (not images), so a runner-image upgrade does not affect `.gitea/workflows/*`. +### Job image pinning and pre-pull + +`act_runner` runs each job inside a container whose image is selected by the runner's _labels_. Two images are in use: + +| Label | Image | Used by | +| ---------------------- | -------------------------------- | ---------------------------------- | +| `self-hosted` | `catthehacker/ubuntu:act-22.04` | `check`, `scan`, `commits`, `a11y` | +| `on-prem` | `catthehacker/ubuntu:act-22.04` | (alias of `self-hosted`) | +| (per-job `container:`) | `catthehacker/ubuntu:full-22.04` | `perf` (Lighthouse needs Chrome) | + +[`runner-config.yaml`](runner-config.yaml) sets `container.force_pull: false`. Without that, act_runner re-issues a `docker pull` at the start of every single job (~10–30 s of registry round-trip even when every layer is already cached), which both wastes wall-clock and contradicts our policy of upgrading job images deliberately rather than implicitly via `:latest`. + +The trade-off: the host Docker daemon must already hold the images locally. Pre-pull them once after a fresh runner host install: + +```bash +docker pull catthehacker/ubuntu:act-22.04 +docker pull catthehacker/ubuntu:full-22.04 +``` + +Upgrading to a newer tag is a deliberate three-step process: + +1. Edit `GITEA_RUNNER_LABELS` (in [`ci-runners.compose.yml`](ci-runners.compose.yml)) and / or the per-job `container.image:` (in `.gitea/workflows/*`) to the new tag. +2. On the runner host, `docker pull ` so the image is locally available before the next CI job starts. +3. Commit on a feature branch with a `chore(deps):` Conventional Commits subject; one of `chore(deps): upgrade CI job image to ...`. + +Old, no-longer-referenced images can be reaped during the periodic `docker system prune -af` (see "Disk pressure" above). + --- ## Future infra concerns — placeholders diff --git a/infra/ci-runners.compose.yml b/infra/ci-runners.compose.yml index 7f83ee5..99b6c71 100644 --- a/infra/ci-runners.compose.yml +++ b/infra/ci-runners.compose.yml @@ -25,11 +25,16 @@ services: # act-compatible runners — bundles Node, Python, git, common build # tools, and the Docker CLI. GITEA_RUNNER_LABELS: self-hosted:docker://catthehacker/ubuntu:act-22.04,on-prem:docker://catthehacker/ubuntu:act-22.04 + # Path of the shared act_runner config file inside the container, + # mounted read-only via volumes below (see infra/runner-config.yaml). + CONFIG_FILE: /etc/runner/config.yaml volumes: # Docker socket — see security note in infra/README.md. - /var/run/docker.sock:/var/run/docker.sock # Persistent state: runner credentials live here after first registration. - ./data/runner-1:/data + # Shared act_runner configuration — see infra/runner-config.yaml. + - ./runner-config.yaml:/etc/runner/config.yaml:ro networks: - act-runners @@ -48,9 +53,13 @@ services: # act-compatible runners — bundles Node, Python, git, common build # tools, and the Docker CLI. GITEA_RUNNER_LABELS: self-hosted:docker://catthehacker/ubuntu:act-22.04,on-prem:docker://catthehacker/ubuntu:act-22.04 + # Path of the shared act_runner config file inside the container, + # mounted read-only via volumes below (see infra/runner-config.yaml). + CONFIG_FILE: /etc/runner/config.yaml volumes: - /var/run/docker.sock:/var/run/docker.sock - ./data/runner-2:/data + - ./runner-config.yaml:/etc/runner/config.yaml:ro networks: - act-runners @@ -69,9 +78,13 @@ services: # act-compatible runners — bundles Node, Python, git, common build # tools, and the Docker CLI. GITEA_RUNNER_LABELS: self-hosted:docker://catthehacker/ubuntu:act-22.04,on-prem:docker://catthehacker/ubuntu:act-22.04 + # Path of the shared act_runner config file inside the container, + # mounted read-only via volumes below (see infra/runner-config.yaml). + CONFIG_FILE: /etc/runner/config.yaml volumes: - /var/run/docker.sock:/var/run/docker.sock - ./data/runner-3:/data + - ./runner-config.yaml:/etc/runner/config.yaml:ro networks: - act-runners diff --git a/infra/runner-config.yaml b/infra/runner-config.yaml new file mode 100644 index 0000000..887cd72 --- /dev/null +++ b/infra/runner-config.yaml @@ -0,0 +1,16 @@ +# act_runner config — shared by all three runner instances defined +# in ci-runners.compose.yml. Mounted read-only at /etc/runner/config.yaml +# inside each container, selected via the CONFIG_FILE environment variable. +# +# Generated from `act_runner generate-config` and trimmed to the keys we +# actually need to override. Defaults for everything else. +# See: https://gitea.com/gitea/act_runner/src/branch/main/internal/pkg/config/config.example.yaml + +container: + # Skip the per-job `docker pull` round-trip when the image is already + # cached locally. The default (true) re-checks the registry on every + # job start, adding 10-30s of latency per job for no value once the + # tag is pinned. Image upgrades happen deliberately on the runner + # host (see infra/README.md "Updating the runner job images") — not + # implicitly via :latest. + force_pull: false