Julien Gautier
7aa2683750
Reduce the apps-profile mode switch (localhost / VSCode tunnel vs
HTTPS hostname) to a single file: infra/local/.env. Today the
toggle requires editing both apps/portal-bff/.env (the four
ENTRA_*_REDIRECT_URI) and infra/local/.env (NX_SERVE_CONFIGURATION).
After this PR, apps/portal-bff/.env stays at its localhost defaults
regardless — native nx serve outside Docker keeps working untouched
— and infra/local/.env carries the entire mode-specific block.
- dev.compose.yml portal-bff service: the four ENTRA_*_REDIRECT_URI
are added to the environment: block with
${VAR:-localhost-default} interpolation. Compose's environment:
wins over env_file:, so the BFF inside the container sees the
override when set and the localhost default otherwise.
- infra/local/.env.example: the trailing Apps block is restructured
as two clearly-labelled profiles (Mode A — Localhost / DEFAULT;
Mode B — HTTPS hostname, commented template).
- apps/portal-bff/.env.example: comment block above the redirect URI
defaults now explains that they stay at localhost regardless and
points at the compose-level override in infra/local/.env.
- infra/README.md: new "Switching between dev modes — localhost vs
hostname" subsection between "Dockerised app dev mode" and
"HTTPS dev-server setup" with a comparison table and per-mode
step-by-step.
483 lines
43 KiB
Markdown
483 lines
43 KiB
Markdown
# `infra/`
|
||
|
||
Infrastructure-as-code artefacts for the project. Separate from application code and from documentation: this folder contains the recipes and configs that the team and ops use to stand up running infrastructure (CI runners, future local-dev databases, future on-prem deploy assets).
|
||
|
||
| Subject | File / Folder | ADR / Reference |
|
||
| -------------------------------------- | --------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||
| Self-hosted CI runners (Gitea Actions) | [`ci-runners.compose.yml`](ci-runners.compose.yml) | [ADR-0015 §"Runners"](../docs/decisions/0015-cicd-gitea-actions.md) |
|
||
| Shared `act_runner` configuration | [`runner-config.yaml`](runner-config.yaml) | [ADR-0015 §"Runners"](../docs/decisions/0015-cicd-gitea-actions.md) |
|
||
| CI runners convenience script | [`ci-runners.sh`](ci-runners.sh) | See "Convenience script" below |
|
||
| Runtime state of the runners | `data/` (git-ignored after `.gitignore`) | — |
|
||
| Env-vars template for the runners | `.env.example` (`.env` is git-ignored) | — |
|
||
| Local-dev runtime stack | [`local/`](local/) | [ADR-0006](../docs/decisions/0006-persistence-postgresql-prisma.md), [ADR-0010](../docs/decisions/0010-session-management-redis.md), [ADR-0012](../docs/decisions/0012-observability-pino-opentelemetry.md), [ADR-0013](../docs/decisions/0013-audit-trail-separated-postgres-append-only.md) |
|
||
| Entra group GUID → role slug map | [`test-tenant.entra.example.json`](test-tenant.entra.example.json) (`*-tenant.entra.json` is git-ignored) | [ADR-0025 §"Sources of truth — Entra-side configuration"](../docs/decisions/0025-authorization-model-privileges-roles-scopes.md) |
|
||
|
||
Future folders / files that will land here as the corresponding ADRs ship:
|
||
|
||
- **`prod/`** — On-prem deploy manifests (HA Postgres, Redis Sentinel, OTel collector + backend, secret manager). Triggered by the on-prem infrastructure ADR (phase 3b).
|
||
|
||
---
|
||
|
||
## CI runners — `ci-runners.compose.yml`
|
||
|
||
Three self-hosted [`act_runner`](https://gitea.com/gitea/act_runner) instances, registered with the project's Gitea organisation, labelled `self-hosted` + `on-prem` (the labels referenced by every job in `.gitea/workflows/*`). Three matches the floor recommended by [ADR-0015 §"Runners"](../docs/decisions/0015-cicd-gitea-actions.md) — one runner is enough to validate the pipeline; two leave no slack; three keep CI flowing if one runner is down for upgrade or maintenance.
|
||
|
||
### First-time registration
|
||
|
||
```bash
|
||
cd infra/
|
||
|
||
# 1. Generate a registration token in Gitea.
|
||
# Site Administration → Actions → Runners → "Create new Runner"
|
||
# (or, for org-scoped runners: Organisation Settings → Actions → Runners).
|
||
# The token is one-time and short-lived; don't lose it.
|
||
|
||
# 2. Configure .env (which is git-ignored).
|
||
cp .env.example .env
|
||
$EDITOR .env
|
||
# Set GITEA_INSTANCE_URL (https, no trailing slash) and
|
||
# GITEA_RUNNER_REGISTRATION_TOKEN.
|
||
|
||
# 3. Pre-pull the job images and bring the runners up. The script
|
||
# chains the two — see "Job image pinning and pre-pull" below
|
||
# for the rationale.
|
||
./ci-runners.sh up --prepull
|
||
|
||
# 4. Verify in Gitea: the three runners appear as online with the
|
||
# self-hosted, on-prem labels. If a runner doesn't come online,
|
||
# inspect its logs:
|
||
./ci-runners.sh logs runner-1
|
||
```
|
||
|
||
After the first successful boot, each runner stores its credentials under `data/runner-N/.runner`. The registration token is no longer needed and **should be removed** from `.env`. Subsequent restarts (`./ci-runners.sh restart …` or direct `docker compose restart …`) authenticate from the persisted credential.
|
||
|
||
### Convenience script — `ci-runners.sh`
|
||
|
||
[`ci-runners.sh`](ci-runners.sh) is a thin wrapper around `docker compose -f ci-runners.compose.yml ...` for the everyday verbs. Two reasons to use it:
|
||
|
||
1. **Hides the compose-file path** on every command. `./ci-runners.sh up` instead of `docker compose -f ci-runners.compose.yml up -d`.
|
||
2. **`rotate` automates the rolling restart** the "Operational tips" below recommend: runner-1 → wait → runner-2 → wait → runner-3, so the CI pipeline always has at least N-1 runners online while you push a config change.
|
||
|
||
| Command | Effect |
|
||
| ---------------------------------- | ------------------------------------------------------------------------------ |
|
||
| `./ci-runners.sh up` | Bring the three runner containers up |
|
||
| `./ci-runners.sh up --prepull` | Pre-pull the job images (`act-22.04` + `:full-22.04`) on the host first |
|
||
| `./ci-runners.sh down` | Stop and remove the containers (preserves `data/runner-N/.runner` credentials) |
|
||
| `./ci-runners.sh restart <runner>` | Restart one runner |
|
||
| `./ci-runners.sh rotate` | Rolling restart of every runner with a 15 s pause between each |
|
||
| `./ci-runners.sh status` | `docker compose ps` for the runner services |
|
||
| `./ci-runners.sh logs [runner]` | Follow logs (one runner or all of them) |
|
||
| `./ci-runners.sh pull-images` | Pre-pull / refresh the job images (idempotent) |
|
||
|
||
Anything not matching one of the named verbs is passed through to `docker compose -f ci-runners.compose.yml ...`. Run `./ci-runners.sh help` for the full reference.
|
||
|
||
For the destructive `down -v` (wipes `data/`, forces re-registration with a fresh Gitea token), the script intentionally **doesn't** offer a verb — invoke `docker compose -f ci-runners.compose.yml down -v` directly so the path is explicit at the typing level.
|
||
|
||
### Operational tips
|
||
|
||
- **Rotation of one runner at a time** — to upgrade the image or change config, run `./ci-runners.sh rotate` (or restart manually one by one — `./ci-runners.sh restart runner-1`, wait, …) so the CI pipeline is never starved.
|
||
- **Logs** — `./ci-runners.sh logs runner-N` (or `docker compose logs -f --tail=100 runner-N`) for a single runner; jobs being executed appear here.
|
||
- **Disk pressure** — the runner caches each job's container image in `/var/lib/docker` on the host. On a small host, prune periodically (`docker system prune -af` while no job is running).
|
||
- **Adding a fourth runner** — copy any `runner-N` block in the compose file, increment the suffix in `container_name`, `GITEA_RUNNER_NAME`, and the `data/` mount path. Add the new name to the `RUNNERS=(…)` array at the top of `ci-runners.sh` so `rotate` and `restart` learn about it. Then `./ci-runners.sh up` (or `docker compose up -d`). The runner registers using the same `GITEA_RUNNER_REGISTRATION_TOKEN` (which must be regenerated if it has expired).
|
||
|
||
### Security — Docker socket exposure
|
||
|
||
The compose mounts `/var/run/docker.sock` into each runner so jobs can spawn containers. **This grants the runner root-equivalent access to the host's Docker daemon.** A malicious workflow could spawn arbitrary containers, mount host paths, escalate privileges. Mitigations:
|
||
|
||
- **Trust boundary:** only register the runners against repositories controlled by the org. Gitea's runner-registration UI lets you scope a runner to an organisation, a single repository, or instance-wide. Prefer the narrowest scope.
|
||
- **Dedicated host:** run these containers on a host that does not also run production services or hold sensitive data. The runner host is in the trust boundary of any developer who can push to a repo it serves.
|
||
- **No host filesystem mounts beyond the docker socket:** the compose intentionally does not mount `/`, `/etc`, or any project source. Workflows that need data on the host must do so via Docker volumes.
|
||
- **Future hardening (out of scope of v1):** migrate to **rootless Docker** on the runner host, or to a **DinD (Docker-in-Docker) sidecar** so the runner cannot escape into the host daemon. Decided when the org's RSSI confirms the security posture, or when the runner host is shared with anything else of value.
|
||
|
||
### Cache server
|
||
|
||
`act_runner` ships a built-in GitHub-Actions-cache-compatible server, used by `actions/setup-node@v6` (`cache: 'pnpm'`), `actions/cache`, and similar. The default behaviour does **not** work in our compose-based setup: the runner container is on the compose-defined `apf-portal-act-runners` bridge, while jobs spawned through the mounted `/var/run/docker.sock` come up on Docker's anonymous `bridge` network — the cache server binds inside the runner on a random port, advertises an IP on the runners' bridge, and the job can't reach it. The symptom is a ~2 min `ETIMEDOUT` at the start (restore) and end (save) of every job that opts into caching.
|
||
|
||
The fix is in [`runner-config.yaml`](runner-config.yaml): `container.network: apf-portal-act-runners` instructs `act_runner` to attach every job container to the same compose-defined bridge as the runners. Job → runner is now an internal-network DNS hop, the advertised cache URL is reachable, and `cache: 'pnpm'` works end-to-end. The `cache: 'pnpm'` flag is enabled on every `actions/setup-node` step in `.gitea/workflows/ci.yml` and `.gitea/workflows/security-scheduled.yml`.
|
||
|
||
The blast-radius trade-off is bounded: every container on `apf-portal-act-runners` is one of our runner containers (plus the jobs they spawn), all of which already have full docker-socket access. Sharing a network does not widen what a malicious workflow can already do; it just lets jobs reach the cache server.
|
||
|
||
If the cache ever needs to be disabled (debugging cache-hit issues, etc.), set `cache.enabled: false` in `runner-config.yaml` and `./ci-runners.sh rotate`.
|
||
|
||
### `act_runner` image pinning
|
||
|
||
The compose pins `gitea/act_runner:0.2.13`. Update the pin deliberately, not via `:latest`:
|
||
|
||
1. Read the act_runner [release notes](https://gitea.com/gitea/act_runner/releases) for breaking changes.
|
||
2. Edit the three image references (`runner-1`, `runner-2`, `runner-3`).
|
||
3. Commit on a feature branch with a `chore(deps):` Conventional Commits subject.
|
||
4. Roll one runner at a time (rotation tip above).
|
||
|
||
The matching CI workflows refer to runner _labels_ (not images), so a runner-image upgrade does not affect `.gitea/workflows/*`.
|
||
|
||
### Job image pinning and pre-pull
|
||
|
||
`act_runner` runs each job inside a container whose image is selected by the runner's _labels_. Two images are in use:
|
||
|
||
| Label | Image | Used by |
|
||
| ---------------------- | -------------------------------- | ---------------------------------- |
|
||
| `self-hosted` | `catthehacker/ubuntu:act-22.04` | `check`, `scan`, `commits`, `a11y` |
|
||
| `on-prem` | `catthehacker/ubuntu:act-22.04` | (alias of `self-hosted`) |
|
||
| (per-job `container:`) | `catthehacker/ubuntu:full-22.04` | `perf` (Lighthouse needs Chrome) |
|
||
|
||
[`runner-config.yaml`](runner-config.yaml) sets `container.force_pull: false`. Without that, act_runner re-issues a `docker pull` at the start of every single job (~10–30 s of registry round-trip even when every layer is already cached), which both wastes wall-clock and contradicts our policy of upgrading job images deliberately rather than implicitly via `:latest`.
|
||
|
||
The trade-off: the host Docker daemon must already hold the images locally. Pre-pull them once after a fresh runner host install:
|
||
|
||
```bash
|
||
docker pull catthehacker/ubuntu:act-22.04
|
||
docker pull catthehacker/ubuntu:full-22.04
|
||
```
|
||
|
||
Upgrading to a newer tag is a deliberate three-step process:
|
||
|
||
1. Edit `GITEA_RUNNER_LABELS` (in [`ci-runners.compose.yml`](ci-runners.compose.yml)) and / or the per-job `container.image:` (in `.gitea/workflows/*`) to the new tag.
|
||
2. On the runner host, `docker pull <new-tag>` so the image is locally available before the next CI job starts.
|
||
3. Commit on a feature branch with a `chore(deps):` Conventional Commits subject; one of `chore(deps): upgrade CI job image to ...`.
|
||
|
||
Old, no-longer-referenced images can be reaped during the periodic `docker system prune -af` (see "Disk pressure" above).
|
||
|
||
---
|
||
|
||
## Local-dev stack — `local/`
|
||
|
||
A Docker Compose recipe spinning up the runtime services the BFF and ADRs assume — Postgres, Redis, OpenTelemetry Collector — plus optional viewers / tooling (pgweb, Jaeger UI, Caddy serve-static) gated behind Compose profiles. Designed to start in a single command on a contributor's WSL2 / Linux / macOS host.
|
||
|
||
| File | Role |
|
||
| -------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
|
||
| [`local/dev.sh`](local/dev.sh) | Convenience wrapper around `docker compose` — see "Convenience script" below |
|
||
| [`local/dev.compose.yml`](local/dev.compose.yml) | Service definitions: postgres, redis, otel-collector, plus pgweb / jaeger / caddy / the `apps` dev servers behind profiles |
|
||
| [`local/Dockerfile.dev`](local/Dockerfile.dev) | Dev-only image (Node 24 + corepack) shared by the three `apps`-profile dev servers (ADR-0030) |
|
||
| [`local/dev-entrypoint.sh`](local/dev-entrypoint.sh) | Entrypoint for the `apps` services: BFF runs `prisma generate` + `migrate deploy`, then each runs `nx serve` |
|
||
| [`local/.env.example`](local/.env.example) | Credentials + ports template (copy to `.env`, which is git-ignored) |
|
||
| [`local/init/postgres/01-init.sql`](local/init/postgres/01-init.sql) | Bootstrap SQL for ADR-0013: audit roles + schema, applied on first boot only |
|
||
| [`local/otel-collector.yaml`](local/otel-collector.yaml) | Collector pipeline: OTLP receivers → batch → debug exporter (always) + forward to Jaeger when active |
|
||
| [`local/Caddyfile`](local/Caddyfile) | Reverse-proxy config for the `serve-static` profile — per-locale SPA fallback + smart `/` redirect (ADR-0019) |
|
||
|
||
### First-time setup
|
||
|
||
```bash
|
||
# 1. Configure local secrets (copy template, edit, do not commit).
|
||
cp infra/local/.env.example infra/local/.env
|
||
$EDITOR infra/local/.env
|
||
# Set strong dev values for POSTGRES_PASSWORD and REDIS_PASSWORD
|
||
# (defaults in the template are placeholders that the compose
|
||
# rejects with `must be set in infra/local/.env` if left as-is).
|
||
|
||
# 2. Bring up the core stack (postgres + redis + otel-collector).
|
||
./infra/local/dev.sh up
|
||
|
||
# 3. (Optional) Activate viewers / tooling when needed:
|
||
./infra/local/dev.sh up dbtools # adds pgweb
|
||
./infra/local/dev.sh up observability # adds Jaeger UI
|
||
./infra/local/dev.sh up serve-static # adds caddy serving the prod build
|
||
./infra/local/dev.sh up all # core + every profile
|
||
|
||
# 4. Verify health.
|
||
./infra/local/dev.sh status
|
||
```
|
||
|
||
### Convenience script — `dev.sh`
|
||
|
||
[`local/dev.sh`](local/dev.sh) is a thin wrapper around `docker compose -f dev.compose.yml ...` with two reasons to exist:
|
||
|
||
1. **Hides the Compose-profile gotcha.** `docker compose down` only operates on services whose profile is currently active — anything started under `--profile X` keeps running unless the same flag is on `down`. The script always passes every profile in scope on teardown / status / log commands, so profile-gated services (pgweb, Jaeger) are never accidentally orphaned.
|
||
2. **Ergonomic verbs** for the common workflows. `./dev.sh up all`, `./dev.sh stop pgweb`, `./dev.sh logs otel-collector`, etc.
|
||
|
||
Run `./infra/local/dev.sh help` for the full reference. Cheat-sheet:
|
||
|
||
| Command | Effect |
|
||
| ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
|
||
| `./infra/local/dev.sh up` | Core only (postgres + redis + otel-collector) |
|
||
| `./infra/local/dev.sh up all` | Core + dbtools + observability + apps (full dev stack). serve-static is excluded — it would collide with apps on port 4200 |
|
||
| `./infra/local/dev.sh up dbtools` | Core + pgweb |
|
||
| `./infra/local/dev.sh up observability` | Core + Jaeger |
|
||
| `./infra/local/dev.sh up serve-static` | Core + Caddy serving `dist/.../browser/` per ADR-0019 |
|
||
| `./infra/local/dev.sh up apps` | Core + the three Nx dev servers in Docker (ADR-0030) |
|
||
| `./infra/local/dev.sh down` | Tear down the whole stack (every profile in scope) |
|
||
| `./infra/local/dev.sh down -v` | Tear down + wipe named volumes (incl. audit-roles bootstrap) |
|
||
| `./infra/local/dev.sh stop pgweb` | Stop one service (containers stay around) |
|
||
| `./infra/local/dev.sh status` | `docker compose ps`, with every profile visible |
|
||
| `./infra/local/dev.sh logs otel-collector` | Follow logs |
|
||
| `./infra/local/dev.sh exec postgres psql -U "$POSTGRES_USER" -d "$POSTGRES_DB"` | Run a command inside a service |
|
||
|
||
Anything not matching one of the named verbs is passed through to `docker compose -f dev.compose.yml ...` (with every profile flagged in), so you keep the full Compose surface available — `./dev.sh config`, `./dev.sh top`, `./dev.sh inspect …`, etc.
|
||
|
||
If you prefer to call `docker compose` directly, every example below shows the raw command alongside the script form.
|
||
|
||
### Dockerised app dev mode — `apps` profile (ADR-0030)
|
||
|
||
The `apps` profile runs the three Nx dev servers **in Docker**, so a contributor can bring up the whole stack without installing Node / pnpm natively:
|
||
|
||
```bash
|
||
./infra/local/dev.sh up apps # infra + portal-bff:3000 + portal-shell:4200 + portal-admin:4300
|
||
```
|
||
|
||
How it works (see [ADR-0030](../docs/decisions/0030-dockerised-dev-mode.md)):
|
||
|
||
- A single [`Dockerfile.dev`](local/Dockerfile.dev) (Node 24 + corepack) backs all three services — one image, one install for the monorepo.
|
||
- The repo is bind-mounted for hot reload; `node_modules` and the Nx cache live in named volumes (`apf-portal-app-node-modules`, `apf-portal-app-nx-cache`) so the container's native modules are never shadowed by the host's.
|
||
- A one-shot `apps-deps` service runs `pnpm install` once into the shared volume; the three servers gate on its completion, avoiding a three-way install race.
|
||
- The BFF entrypoint runs `prisma generate` + `prisma migrate deploy` before serving.
|
||
|
||
**Prerequisite — the BFF still needs its secrets.** No native toolchain is required, but `apps/portal-bff/.env` (Entra / session / jwks config) must exist, same as native dev (`cp apps/portal-bff/.env.example apps/portal-bff/.env` then fill it). The host-specific URLs (`DATABASE_URL` / `REDIS_URL` / OTel endpoint) are overridden automatically to the Compose service names — you don't edit those for the container. SPA-only work (`up portal-shell`) doesn't need the BFF env.
|
||
|
||
**Port note.** The SPA dev servers default to 4200 / 4300 — 4200 is the same port the `serve-static` profile uses. Don't run `apps` and `serve-static` together, or set `SHELL_PORT` in `infra/local/.env`.
|
||
|
||
The three dev modes (native `nx serve`, devcontainer, this `apps` profile) and when to use each are summarised in [docs/setup/01-dev-debian-vm-setup.md](../docs/setup/01-dev-debian-vm-setup.md).
|
||
|
||
### Switching between dev modes — `localhost` vs hostname
|
||
|
||
Within the `apps` profile, there are **two access modes** for the SPA. They differ only in how the browser reaches the dev-servers — the BFF and the rest of the stack are unchanged. The toggle is a single file: [`infra/local/.env`](local/.env.example).
|
||
|
||
| Mode | When to use | What flips |
|
||
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
|
||
| **A — `localhost` (default)** | Solo dev, browser on the same workstation as the VSCode Remote-SSH client. No cert plumbing, no `hosts` file change. The fastest path to a working stack. | Nothing — `infra/local/.env` is the default. |
|
||
| **B — HTTPS hostname** | A teammate (or PM / QA) needs to browse YOUR VM from THEIR machine, or you have a shared VM. Requires the team mkcert CA already installed (see "Team mkcert CA on `vm-gitlab`" below) and `apf-portal.dev-XX.local` in their `hosts` file. | `NX_SERVE_CONFIGURATION=https` plus the four `ENTRA_*_REDIRECT_URI` lines in `infra/local/.env` (commented template in `.env.example`). |
|
||
|
||
The toggle works because Compose's `environment:` block on the `portal-bff` service interpolates each `ENTRA_*_REDIRECT_URI` from `infra/local/.env` with a `localhost` fallback, and wins over `env_file:`. As a result `apps/portal-bff/.env` keeps its native-friendly `localhost` defaults regardless of mode — native `nx serve` (no Docker) and Mode A both read the same `.env` cleanly, while Mode B sees the override only inside the container.
|
||
|
||
#### Mode A — `localhost` via VSCode port forwarding
|
||
|
||
1. Make sure `infra/local/.env` does NOT set `NX_SERVE_CONFIGURATION` (or sets it to `development`) and leaves the four `ENTRA_*_REDIRECT_URI` lines commented (this is the `.env.example` default).
|
||
2. `./infra/local/dev.sh up apps`.
|
||
3. VSCode Remote-SSH auto-discovers the published ports (panel **PORTS** at the bottom of VSCode) and forwards them to your workstation. If a port doesn't show up, "Forward a Port" manually (4200, 4300, 3000).
|
||
4. Open `http://localhost:4200/` on the workstation. The SPA loads, fetches `/api/...` (proxied to the BFF inside Compose), and the OIDC callback at `http://localhost:3000/api/auth/callback` (already registered in Entra) is reachable through the same VSCode tunnel.
|
||
|
||
Nothing else to configure. No mkcert. No `hosts` file. No cert warning.
|
||
|
||
#### Mode B — HTTPS hostname (`apf-portal.dev-XX.local`)
|
||
|
||
Detailed setup in the next two subsections ("HTTPS dev-server setup" + "Team mkcert CA on `vm-gitlab`"). Once the workstation has the team CA installed and the host file knows the hostname, switching is:
|
||
|
||
1. In `infra/local/.env`, uncomment the five Mode B lines (and replace `dev-jg` with the right hostname).
|
||
2. `./infra/local/dev.sh down && ./infra/local/dev.sh up apps`.
|
||
3. Browse `https://apf-portal.dev-XX.local:4200/`.
|
||
|
||
To switch back to Mode A: comment those five lines, `down && up apps`. No other file touched.
|
||
|
||
### HTTPS dev-server setup — remote-browser access via a hostname
|
||
|
||
By default the dev-servers serve plain HTTP — fine when the browser is on the same host as the BFF (`http://localhost:4200/`), which is also the only HTTP origin Entra accepts as a redirect URI. The moment you access the SPA over a **hostname** (e.g. `apf-portal.dev.local`, useful when the browser sits on a workstation and the stack runs on a shared / per-dev VM), Entra refuses the `http:` redirect URI and the dev-servers must terminate TLS.
|
||
|
||
The setup is one-time per dev:
|
||
|
||
1. **Install [mkcert](https://github.com/FiloSottile/mkcert)** on your workstation (the machine where the browser runs) and bootstrap its local CA:
|
||
|
||
```bash
|
||
# Debian / WSL Ubuntu:
|
||
sudo apt install -y libnss3-tools
|
||
# macOS:
|
||
# brew install mkcert nss
|
||
# Windows (PowerShell, choco):
|
||
# choco install mkcert
|
||
mkcert -install
|
||
```
|
||
|
||
2. **Generate the cert** for the hostname you registered in your `/etc/hosts` and in Entra. From the repo root on your workstation:
|
||
|
||
```bash
|
||
mkdir -p .secrets
|
||
mkcert -key-file .secrets/dev-tls.key -cert-file .secrets/dev-tls.pem \
|
||
apf-portal.dev-jg.local # ← replace with YOUR hostname
|
||
```
|
||
|
||
`.secrets/` is git-ignored; the bind-mount in the `apps` profile (the repo at `/workspace`) makes the files visible inside the containers at the path the `https` configuration expects.
|
||
|
||
3. **Update `apps/portal-bff/.env`** so the BFF tells Entra the matching HTTPS URIs — see the redirect-URI block in [`apps/portal-bff/.env.example`](../apps/portal-bff/.env.example) for the override pattern. The same URIs must be registered in your Entra app registration's "Redirect URIs" list (the BFF only sends one of them per auth request; Entra validates it is on the list).
|
||
|
||
4. **Enable the `https` Nx serve configuration** for the compose dev-servers by adding to `infra/local/.env`:
|
||
|
||
```env
|
||
NX_SERVE_CONFIGURATION=https
|
||
```
|
||
|
||
The compose command resolves `--configuration=${NX_SERVE_CONFIGURATION:-development}` at parse time, so the SPAs pick up the `https` config defined in `apps/portal-shell/project.json` and `apps/portal-admin/project.json`. The BFF stays HTTP behind the proxy — only the public origin is HTTPS.
|
||
|
||
5. `./infra/local/dev.sh up apps` → browser opens `https://apf-portal.dev-jg.local:4200/`. No cert warning (mkcert's CA is trusted by the workstation after step 1).
|
||
|
||
Native `nx serve` (WSL / localhost) is **unaffected** — it keeps using the `development` configuration by default, no SSL required, and the `localhost` URIs registered in Entra still work.
|
||
|
||
When real DNS + corp-CA-signed certs arrive, the hostname can be reused as-is (Entra registrations are literal strings — they don't care who signs the cert). Drop the cert files back into `.secrets/` and remove the mkcert step.
|
||
|
||
### Team mkcert CA on `vm-gitlab` — sharing the trust root
|
||
|
||
The previous section is the **solo flow** (one dev mints their own CA, certs only trusted by their own workstation). It does not let a teammate browse another dev's VM without a certificate warning — every dev has their own private CA, none of which the others trust.
|
||
|
||
For a multi-dev team the canonical pattern is one shared CA held on `vm-gitlab`. The CA private key (`rootCA-key.pem`) stays on `vm-gitlab` — never copied to any workstation; only the public `rootCA.pem` is distributed to each developer's Windows trust store, and the R&D Lead mints per-VM certs on `vm-gitlab` when a new VM (or new developer) joins. Browsing any dev VM from any workstation then "just works" — green padlock, no warning.
|
||
|
||
This subsection assumes the per-dev workstation procedure of "HTTPS dev-server setup" above is what every developer will do **once**, with the rootCA.pem they receive from this shared CA.
|
||
|
||
#### Initial setup on `vm-gitlab` (one-time, by the R&D Lead)
|
||
|
||
```bash
|
||
# 1. Install mkcert on vm-gitlab (no service to run — mkcert is one-shot).
|
||
sudo curl -fsSL https://dl.filippo.io/mkcert/latest?for=linux/amd64 \
|
||
-o /usr/local/bin/mkcert
|
||
sudo chmod +x /usr/local/bin/mkcert
|
||
|
||
# 2. Create the shared CAROOT, root-only.
|
||
sudo mkdir -p /srv/apf-portal/mkcert-ca
|
||
sudo chown root:root /srv/apf-portal/mkcert-ca
|
||
sudo chmod 700 /srv/apf-portal/mkcert-ca
|
||
|
||
# 3. Generate the CA into that CAROOT. (`-install` here just touches
|
||
# the local trust store of vm-gitlab — cosmetic for an infra VM,
|
||
# no harm.)
|
||
sudo CAROOT=/srv/apf-portal/mkcert-ca mkcert -install
|
||
|
||
# 4. Verify.
|
||
sudo ls -la /srv/apf-portal/mkcert-ca/
|
||
# → rootCA.pem (-rw-r--r--), rootCA-key.pem (-rw-------, root only)
|
||
```
|
||
|
||
After this, the CA exists and is owned by `root` on `vm-gitlab`. Developers never touch it directly.
|
||
|
||
#### Minting a cert for a dev VM (R&D Lead, on `vm-gitlab`)
|
||
|
||
Repeat once per VM hostname (`apf-portal.dev-jg.local`, `apf-portal.dev-vc.local`, `apf-portal.dev.local`, …). Replace `<host>` and the SSH/scp target accordingly:
|
||
|
||
```bash
|
||
sudo CAROOT=/srv/apf-portal/mkcert-ca mkcert \
|
||
-key-file /tmp/<host>-tls.key \
|
||
-cert-file /tmp/<host>-tls.pem \
|
||
apf-portal.<host>.local
|
||
|
||
# Sanity check.
|
||
sudo openssl x509 -in /tmp/<host>-tls.pem -noout -subject -issuer
|
||
# subject CN must be apf-portal.<host>.local; issuer the mkcert CA name.
|
||
|
||
# Ship to the target VM, renaming to the path the `https` Nx serve
|
||
# configuration expects (.secrets/dev-tls.{key,pem}).
|
||
sudo scp /tmp/<host>-tls.key <vm>:~/Works/apf_portal/.secrets/dev-tls.key
|
||
sudo scp /tmp/<host>-tls.pem <vm>:~/Works/apf_portal/.secrets/dev-tls.pem
|
||
|
||
# Wipe the staging copies.
|
||
sudo rm /tmp/<host>-tls.*
|
||
```
|
||
|
||
The certificate is good for ~2 years (mkcert default). When it nears expiry, regenerate with the same command and re-`scp` — the dev-server picks up the new files on next restart.
|
||
|
||
#### Onboarding a new developer
|
||
|
||
A new teammate needs **three things**: a copy of `rootCA.pem` (public, low-sensitivity), a per-VM cert minted by the R&D Lead, and the same hosts-file + `.env` configuration every dev follows.
|
||
|
||
**R&D Lead side** — on `vm-gitlab`:
|
||
|
||
```bash
|
||
# Hand off the public CA cert to the new dev via a secure channel
|
||
# (1Password shared vault, Bitwarden, direct scp). Never plain e-mail.
|
||
sudo cat /srv/apf-portal/mkcert-ca/rootCA.pem
|
||
```
|
||
|
||
Then mint that dev's per-VM cert (see "Minting a cert for a dev VM" above) and ship it to their VM's `~/Works/apf_portal/.secrets/`.
|
||
|
||
**New developer side** — on their Windows workstation:
|
||
|
||
```powershell
|
||
# 1. Install mkcert (only to get the `-install` command — no need to
|
||
# generate certs on the workstation).
|
||
choco install mkcert -y
|
||
|
||
# 2. Drop the rootCA.pem they received into the local CAROOT path.
|
||
$caroot = mkcert -CAROOT
|
||
Copy-Item "C:\path\to\rootCA.pem" "$caroot\rootCA.pem"
|
||
# NB: only rootCA.pem — they do NOT receive rootCA-key.pem.
|
||
|
||
# 3. Register the team CA in their Windows trust store.
|
||
mkcert -install
|
||
# Confirm the Windows security dialog. Their machine now trusts every
|
||
# cert minted by the team CA on vm-gitlab.
|
||
```
|
||
|
||
Then they:
|
||
|
||
- Edit `C:\Windows\System32\drivers\etc\hosts` (admin) and add the entries for every VM they want to reach (their own + the others as needed):
|
||
```
|
||
10.100.201.20 apf-portal.dev-vc.local
|
||
10.100.201.21 apf-portal.dev-jg.local
|
||
10.100.201.22 apf-portal.dev.local
|
||
```
|
||
- Edit `apps/portal-bff/.env` on their VM so the four `ENTRA_*_REDIRECT_URI` values point at `https://apf-portal.<their-host>:{4200,4300}/...` (the matching URIs are already registered Entra-side — no action there).
|
||
- Set `NX_SERVE_CONFIGURATION=https` in `infra/local/.env` on their VM.
|
||
- `./infra/local/dev.sh down && ./infra/local/dev.sh up apps`.
|
||
|
||
Total onboarding budget: ~5 min of R&D Lead time on `vm-gitlab` (mint + transfer) + ~10 min of work on the new dev's workstation + VM. No SSH access to `vm-gitlab` is granted to developers — only the R&D Lead operates the CA.
|
||
|
||
#### Operational notes
|
||
|
||
- **Departures.** mkcert has no CRL; revoking trust on a former dev's machine isn't actionable from the CA side. The risk surface is what that dev could have signed before leaving — and they only ever had the public `rootCA.pem`, never the private key, so they cannot have signed anything in your trust circle. No action required when a dev leaves.
|
||
- **CA rotation.** Rare (audit, suspected compromise, annual hygiene). Regenerate the CA on `vm-gitlab`, re-mint every VM's cert, redistribute the new `rootCA.pem` to each dev. Each dev re-imports + re-`mkcert -install`. No `.env` or Entra change.
|
||
- **Per-VM cert rotation.** Same pattern as initial mint — regenerate, scp, `dev.sh restart portal-shell portal-admin`. No client-side action.
|
||
- **Migration to a corp-signed CA.** When the infra team issues an internal-CA-signed cert (already trusted by every domain-joined workstation, no mkcert step), drop those files into `.secrets/dev-tls.{key,pem}` and remove the team mkcert CA from each dev's trust store. Entra registrations are unchanged — they reference hostname + port, not the issuer.
|
||
|
||
### Service endpoints (defaults)
|
||
|
||
| Service | Host port | Purpose |
|
||
| ---------------------------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||
| Postgres | 5432 | DB connection — `postgres://portal:<pwd>@localhost:5432/portal_dev` |
|
||
| Redis | 6379 | Sessions, OBO cache (per ADR-0010 / ADR-0014) |
|
||
| OTel Collector gRPC | 4317 | `OTEL_EXPORTER_OTLP_ENDPOINT` for the BFF and the SPA |
|
||
| OTel Collector HTTP | 4318 | OTLP/HTTP variant |
|
||
| pgweb (profile) | 8081 | http://localhost:8081 — Postgres GUI |
|
||
| Jaeger UI (profile) | 16686 | http://localhost:16686 — trace explorer |
|
||
| Caddy serve-static (profile) | 4200 | http://localhost:4200/ — production build with per-locale routing (`/fr/`, `/en/`) + smart `/` redirect, per ADR-0019. Run `pnpm exec nx build portal-shell --configuration=production` first or the proxy will 404 everything. |
|
||
|
||
All ports are overridable via `.env` if the host machine has conflicts.
|
||
|
||
### Operational tips
|
||
|
||
- **Persistence** — state lives in named Docker volumes (`apf-portal-postgres-data`, `apf-portal-redis-data`). Survives `docker compose down`. Use `docker compose -f dev.compose.yml down -v` to wipe (also wipes the audit-roles bootstrap, which re-runs on the next fresh boot).
|
||
- **Profile symmetry** — `dev.sh down` (and `status`, `logs`, …) always include every profile in scope, so profile-gated services are caught. If you bypass the script and call `docker compose down` directly, you must pass the same `--profile` flags as on `up`, otherwise pgweb and Jaeger keep running silently. Either pass them again, or `export COMPOSE_PROFILES=dbtools,observability` in your shell or `infra/local/.env`.
|
||
|
||
- **Bootstrap re-run** — the SQL in `local/init/postgres/` only runs on a **fresh** Postgres data volume. To replay after editing the file, `down -v` (loses all dev data) or run the SQL manually with `docker compose exec postgres psql -U portal -d portal_dev -f /docker-entrypoint-initdb.d/01-init.sql`.
|
||
- **Logs** — `docker compose -f dev.compose.yml logs -f <service>` to follow a single service. `otel-collector` is the loudest — its `debug` exporter prints every span / metric / log it receives.
|
||
- **Image upgrades** — same policy as the runner image (deliberate, not via `:latest`). Renovate's docker-compose manager will surface bumps automatically once the dashboard rule allows them.
|
||
|
||
### Production parity
|
||
|
||
This stack is **dev-only**. The corresponding production layout (HA Postgres, Redis Sentinel cluster, OTel Collector with a real backend, secret manager) lives in the future on-prem-infrastructure ADR — see `prod/` placeholder below.
|
||
|
||
---
|
||
|
||
## Entra group map — `test-tenant.entra.example.json`
|
||
|
||
Pure JSON object keyed on Entra security-group GUID (lower-case), valued by an `apf-role-<slug>` slug from the ADR-0025 functional-role catalogue. The BFF loads it at boot through `EntraGroupToRoleResolver` (from `shared-auth`) and uses it on every sign-in to translate the `groups` claim into the 24-entry catalogue's role slugs.
|
||
|
||
The 24 entries below cover the entire v1 catalogue — including `partenaire`, which ships empty in the test tenant by design but is kept in the schema so a typo or omission fails the parser at boot rather than silently dropping the role.
|
||
|
||
### Provisioning a real file
|
||
|
||
```bash
|
||
cp infra/test-tenant.entra.example.json infra/test-tenant.entra.json
|
||
|
||
# Then for each role replace the placeholder GUID with the real one
|
||
# from Entra:
|
||
# Microsoft Entra admin centre → Groups → <apf-role-*> → Object ID.
|
||
# Point the BFF at the file via apps/portal-bff/.env:
|
||
# ENTRA_GROUP_MAP_PATH=infra/test-tenant.entra.json
|
||
```
|
||
|
||
The real file (`infra/<env>-tenant.entra.json`) is git-ignored because the group GUIDs are tenant-private — leaking them does not authorize anything by itself, but it does reveal the tenant's internal authorization topology. Each environment (test / preprod / prod) carries its own file; the slugs are stable across environments, the GUIDs are not.
|
||
|
||
If `ENTRA_GROUP_MAP_PATH` is unset, the resolver runs with an empty map: every user signs in successfully but receives an empty `roles[]` (and consequently no `apf-role-*` UI). The BFF logs a WARN at boot so an operator can spot the missing config; this is a deliberate fail-soft posture so a fresh dev environment is not blocked by an Entra-side dependency.
|
||
|
||
Validation rules enforced at boot by `parseEntraGroupMap` (in `libs/shared/auth/`):
|
||
|
||
- keys must look like a GUID (`8-4-4-4-12` hex);
|
||
- values must be members of `FUNCTIONAL_ROLES`;
|
||
- the same GUID cannot map to two different slugs (case-insensitive).
|
||
|
||
A malformed file crashes the BFF at startup. The error message names the offending key / value.
|
||
|
||
---
|
||
|
||
## Future infra concerns — placeholders
|
||
|
||
These are listed here so a contributor knows where to expect related files; they don't exist yet.
|
||
|
||
| File | Purpose | Triggered by |
|
||
| --------------- | ----------------------------------------------------------------------------------------- | -------------------------------------------------- |
|
||
| `prod/*` | On-prem deployment manifests (k8s, Compose, or whatever the on-prem infra ADR settles on) | The on-prem infrastructure ADR (phase 3b) |
|
||
| `runbooks/*.md` | Operational runbooks (incident response, secret rotation, runner upgrade procedure, …) | First incident, or when ops cadence justifies them |
|