feat(infra): add local-dev Docker Compose stack (#57)
## Summary Bring up Postgres + Redis + OTel Collector in one command so contributors can run the BFF end-to-end without manually wiring each service. Replaces the throwaway `docker run postgres:17-alpine` one-liner that was in `docs/development.md` §3. ### What lands - **`infra/local/dev.compose.yml`** — three core services (`postgres:17.2-alpine`, `redis:7.4-alpine`, `otel/opentelemetry-collector-contrib:0.115.0`) plus two viewers gated behind Compose profiles: - `--profile dbtools` → `sosedoff/pgweb:0.16.2` (Postgres GUI on port 8081) - `--profile observability` → `jaegertracing/all-in-one:1.62` (Jaeger UI on 16686) - All ports overridable via `.env`. State in named volumes. Healthchecks on data services. - **`infra/local/.env.example`** — credentials + ports template. `POSTGRES_PASSWORD` and `REDIS_PASSWORD` are mandatory (compose refuses to boot without them); other keys default sensibly. - **`infra/local/init/postgres/01-init.sql`** — bootstrap SQL per **ADR-0013**: `audit_owner` / `audit_writer` / `audit_reader` / `audit_archiver` roles + `audit` schema. Default privileges encode the append-only contract (INSERT to writer, SELECT to reader, DELETE to archiver, no UPDATE/TRUNCATE to anyone). Applied on first Postgres boot only; documented re-run procedure. - **`infra/local/otel-collector.yaml`** — pipeline: OTLP gRPC/HTTP → batch → debug exporter (always) + forward to `jaeger:4317`. When the observability profile is off, the Jaeger export logs warn-level retries but doesn't block the debug pipeline. ### Surrounding doc updates - **`infra/README.md`** — new "Local-dev stack" section: service inventory, port table, first-time setup walkthrough, persistence/bootstrap-replay tips. The previous `local/` placeholder line is removed. - **`docs/development.md`** §3 — rewritten to walk through the compose-based setup; cross-links to `infra/README.md` for the full reference. Roadmap entry for "Local infra recipe" removed from §8 (now implemented); "Observability dev-loop" line adjusted to point at the new Jaeger profile. ### Out of scope - **Production parity** — HA Postgres, Redis Sentinel, real OTel backend (Tempo / Loki / etc.) — defer to the on-prem infrastructure ADR (phase 3b). The dev-only nature of this stack is called out explicitly in `infra/README.md`. - **Wiring the BFF** to actually use these endpoints (NestJS config, Prisma datasource URL, OTel SDK init) — that's the **B — Observability foundations** chantier, next up. ## Test plan - [ ] `cd infra/local && cp .env.example .env && docker compose -f dev.compose.yml up -d` → all three core services come up healthy; verify with `docker compose ps`. - [ ] `psql postgres://portal:<pwd>@localhost:5432/portal_dev -c "\dn"` shows the `audit` schema; `\dg` shows the four audit roles. - [ ] `redis-cli -a <pwd> PING` → `PONG`. - [ ] Send a fake OTLP trace via grpcurl → see it printed by `docker compose logs otel-collector`. - [ ] `--profile dbtools up -d` → http://localhost:8081 shows pgweb UI, can navigate to the audit schema. - [ ] `--profile observability up -d` → http://localhost:16686 shows Jaeger UI; collector logs no longer report Jaeger export retries. - [ ] `docker compose down -v` cleanly removes everything; next `up -d` re-runs the bootstrap SQL. --------- Co-authored-by: Julien Gautier <julien.gautier@apf.asso.fr> Reviewed-on: #57
This commit was merged in pull request #57.
This commit is contained in:
+72
-13
@@ -2,16 +2,16 @@
|
||||
|
||||
Infrastructure-as-code artefacts for the project. Separate from application code and from documentation: this folder contains the recipes and configs that the team and ops use to stand up running infrastructure (CI runners, future local-dev databases, future on-prem deploy assets).
|
||||
|
||||
| Subject | File / Folder | ADR / Reference |
|
||||
| -------------------------------------- | -------------------------------------------------- | ------------------------------------------------------------------- |
|
||||
| Self-hosted CI runners (Gitea Actions) | [`ci-runners.compose.yml`](ci-runners.compose.yml) | [ADR-0015 §"Runners"](../docs/decisions/0015-cicd-gitea-actions.md) |
|
||||
| Shared `act_runner` configuration | [`runner-config.yaml`](runner-config.yaml) | [ADR-0015 §"Runners"](../docs/decisions/0015-cicd-gitea-actions.md) |
|
||||
| Runtime state of the runners | `data/` (git-ignored after `.gitignore`) | — |
|
||||
| Env-vars template for the runners | `.env.example` (`.env` is git-ignored) | — |
|
||||
| Subject | File / Folder | ADR / Reference |
|
||||
| -------------------------------------- | -------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Self-hosted CI runners (Gitea Actions) | [`ci-runners.compose.yml`](ci-runners.compose.yml) | [ADR-0015 §"Runners"](../docs/decisions/0015-cicd-gitea-actions.md) |
|
||||
| Shared `act_runner` configuration | [`runner-config.yaml`](runner-config.yaml) | [ADR-0015 §"Runners"](../docs/decisions/0015-cicd-gitea-actions.md) |
|
||||
| Runtime state of the runners | `data/` (git-ignored after `.gitignore`) | — |
|
||||
| Env-vars template for the runners | `.env.example` (`.env` is git-ignored) | — |
|
||||
| Local-dev runtime stack | [`local/`](local/) | [ADR-0006](../docs/decisions/0006-persistence-postgresql-prisma.md), [ADR-0010](../docs/decisions/0010-session-management-redis.md), [ADR-0012](../docs/decisions/0012-observability-pino-opentelemetry.md), [ADR-0013](../docs/decisions/0013-audit-trail-separated-postgres-append-only.md) |
|
||||
|
||||
Future folders / files that will land here as the corresponding ADRs ship:
|
||||
|
||||
- **`local/`** — Docker Compose for the developer's machine (Postgres + Redis + OTel collector). Triggered by the first feature that needs Redis (sessions, ADR-0010).
|
||||
- **`prod/`** — On-prem deploy manifests (HA Postgres, Redis Sentinel, OTel collector + backend, secret manager). Triggered by the on-prem infrastructure ADR (phase 3b).
|
||||
|
||||
---
|
||||
@@ -121,13 +121,72 @@ Old, no-longer-referenced images can be reaped during the periodic `docker syste
|
||||
|
||||
---
|
||||
|
||||
## Local-dev stack — `local/`
|
||||
|
||||
A Docker Compose recipe spinning up the runtime services the BFF and ADRs assume — Postgres, Redis, OpenTelemetry Collector — plus optional viewers (pgweb, Jaeger UI) gated behind Compose profiles. Designed to start in a single command on a contributor's WSL2 / Linux / macOS host.
|
||||
|
||||
| File | Role |
|
||||
| -------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
|
||||
| [`local/dev.compose.yml`](local/dev.compose.yml) | Service definitions: postgres, redis, otel-collector, plus pgweb and jaeger behind profiles |
|
||||
| [`local/.env.example`](local/.env.example) | Credentials + ports template (copy to `.env`, which is git-ignored) |
|
||||
| [`local/init/postgres/01-init.sql`](local/init/postgres/01-init.sql) | Bootstrap SQL for ADR-0013: audit roles + schema, applied on first boot only |
|
||||
| [`local/otel-collector.yaml`](local/otel-collector.yaml) | Collector pipeline: OTLP receivers → batch → debug exporter (always) + forward to Jaeger when active |
|
||||
|
||||
### First-time setup
|
||||
|
||||
```bash
|
||||
cd infra/local
|
||||
|
||||
# 1. Configure local secrets (copy template, edit, do not commit).
|
||||
cp .env.example .env
|
||||
$EDITOR .env
|
||||
# Set strong dev values for POSTGRES_PASSWORD and REDIS_PASSWORD
|
||||
# (defaults in the template are placeholders that the compose
|
||||
# rejects with `must be set in infra/local/.env` if left as-is).
|
||||
|
||||
# 2. Bring up the core stack (postgres + redis + otel-collector).
|
||||
docker compose -f dev.compose.yml up -d
|
||||
|
||||
# 3. (Optional) Activate viewers when needed:
|
||||
docker compose -f dev.compose.yml --profile dbtools up -d # pgweb
|
||||
docker compose -f dev.compose.yml --profile observability up -d # Jaeger UI
|
||||
docker compose -f dev.compose.yml --profile dbtools --profile observability up -d # both
|
||||
|
||||
# 4. Verify health.
|
||||
docker compose -f dev.compose.yml ps
|
||||
```
|
||||
|
||||
### Service endpoints (defaults)
|
||||
|
||||
| Service | Host port | Purpose |
|
||||
| ------------------- | --------- | ------------------------------------------------------------------- |
|
||||
| Postgres | 5432 | DB connection — `postgres://portal:<pwd>@localhost:5432/portal_dev` |
|
||||
| Redis | 6379 | Sessions, OBO cache (per ADR-0010 / ADR-0014) |
|
||||
| OTel Collector gRPC | 4317 | `OTEL_EXPORTER_OTLP_ENDPOINT` for the BFF and the SPA |
|
||||
| OTel Collector HTTP | 4318 | OTLP/HTTP variant |
|
||||
| pgweb (profile) | 8081 | http://localhost:8081 — Postgres GUI |
|
||||
| Jaeger UI (profile) | 16686 | http://localhost:16686 — trace explorer |
|
||||
|
||||
All ports are overridable via `.env` if the host machine has conflicts.
|
||||
|
||||
### Operational tips
|
||||
|
||||
- **Persistence** — state lives in named Docker volumes (`apf-portal-postgres-data`, `apf-portal-redis-data`). Survives `docker compose down`. Use `docker compose -f dev.compose.yml down -v` to wipe (also wipes the audit-roles bootstrap, which re-runs on the next fresh boot).
|
||||
- **Bootstrap re-run** — the SQL in `local/init/postgres/` only runs on a **fresh** Postgres data volume. To replay after editing the file, `down -v` (loses all dev data) or run the SQL manually with `docker compose exec postgres psql -U portal -d portal_dev -f /docker-entrypoint-initdb.d/01-init.sql`.
|
||||
- **Logs** — `docker compose -f dev.compose.yml logs -f <service>` to follow a single service. `otel-collector` is the loudest — its `debug` exporter prints every span / metric / log it receives.
|
||||
- **Image upgrades** — same policy as the runner image (deliberate, not via `:latest`). Renovate's docker-compose manager will surface bumps automatically once the dashboard rule allows them.
|
||||
|
||||
### Production parity
|
||||
|
||||
This stack is **dev-only**. The corresponding production layout (HA Postgres, Redis Sentinel cluster, OTel Collector with a real backend, secret manager) lives in the future on-prem-infrastructure ADR — see `prod/` placeholder below.
|
||||
|
||||
---
|
||||
|
||||
## Future infra concerns — placeholders
|
||||
|
||||
These are listed here so a contributor knows where to expect related files; they don't exist yet.
|
||||
|
||||
| File | Purpose | Triggered by |
|
||||
| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------- |
|
||||
| `local/dev.compose.yml` | Postgres 17 + Redis (Sentinel-flavoured single-node) + OTel collector for local dev | First feature that needs Redis or end-to-end OTel traces |
|
||||
| `local/init/postgres/*.sql` | Bootstrap SQL to create the Postgres roles `audit_owner` / `audit_writer` / `audit_reader` / `audit_archiver` (per ADR-0013) and provision the dev DB | Same as above |
|
||||
| `prod/*` | On-prem deployment manifests (k8s, Compose, or whatever the on-prem infra ADR settles on) | The on-prem infrastructure ADR (phase 3b) |
|
||||
| `runbooks/*.md` | Operational runbooks (incident response, secret rotation, runner upgrade procedure, …) | First incident, or when ops cadence justifies them |
|
||||
| File | Purpose | Triggered by |
|
||||
| --------------- | ----------------------------------------------------------------------------------------- | -------------------------------------------------- |
|
||||
| `prod/*` | On-prem deployment manifests (k8s, Compose, or whatever the on-prem infra ADR settles on) | The on-prem infrastructure ADR (phase 3b) |
|
||||
| `runbooks/*.md` | Operational runbooks (incident response, secret rotation, runner upgrade procedure, …) | First incident, or when ops cadence justifies them |
|
||||
|
||||
Reference in New Issue
Block a user