02ac44e4982a0f4ec35a0b825f944f993f6f1b8a
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 julien
|
02ac44e498 |
feat(portal-bff): audit log foundation per ADR-0013 (#76)
## Summary Lays down the append-only audit log per ADR-0013: schema declaration, first migration with role grants, NestJS `AuditWriter` service. Typed event-family methods, the separate `AUDIT_DATABASE_URL` pool, the retention job, and the live-DB integration tests are explicitly listed as "wired as features land" in the ADR's confirmation block — they ship when the matching feature ADRs do. ## What lands **Prisma schema** ([`apps/portal-bff/prisma/schema.prisma`](apps/portal-bff/prisma/schema.prisma)): - `multiSchema` preview enabled; datasource declares `public` + `audit` schemas. - `AuditEvent` model: `id` (uuid), `createdAt`, `eventType` (free-form in v1), `audience` enum (`workforce | customer`), `actorIdHash`, `traceId`, `subject`, `outcome` enum (`success | failure | denied`), `payload` (jsonb). - Indexes on `createdAt`, `eventType`, `traceId` — covering the three obvious query shapes. **Migration** ([`prisma/migrations/*_init_audit_schema/migration.sql`](apps/portal-bff/prisma/migrations/20260510011453_init_audit_schema/migration.sql)): - Standard Prisma `CREATE TABLE` / enums output, then the **append-only contract** re-applied explicitly: - `ALTER TABLE/TYPE OWNER TO audit_owner`. - `GRANT INSERT` to `audit_writer`, `SELECT` to `audit_reader`, **`SELECT, DELETE`** to `audit_archiver` (SELECT is needed to evaluate the `created_at` predicate of "delete older than retention" — Postgres requires SELECT on every column referenced in DELETE's WHERE). - `GRANT USAGE` on the enum types to all three roles (without it `audit_writer.INSERT` fails with "permission denied for type"). - **No** GRANT for `UPDATE` / `TRUNCATE` to anyone — including `audit_owner` at runtime; only fresh schema migrations amend the table. **Service** ([`apps/portal-bff/src/audit/`](apps/portal-bff/src/audit/)): - `AuditWriter.recordEvent(input)` — single entry point. Wraps every INSERT in a transaction whose first statement is `SET LOCAL ROLE audit_writer`, so the role contract holds at runtime even from the otherwise-privileged BFF connection. - `traceId` auto-resolved from the active OTel span (so audit row joins with traces and Pino logs on the same `trace_id`). - `actorIdHash` auto-resolved from CLS (key `actorIdHash`) with explicit input-side override; `null` when neither is set (placeholder until ADR-0009 / ADR-0010 guards populate CLS). - Errors propagate (no catch-and-swallow), per ADR-0013's "blocking writes: no audit ⇒ no action". **Tests** — 8 unit tests on `AuditWriter` (mocked Prisma + CLS): role-locking ordering, input pass-through, `Prisma.JsonNull` for missing payload, CLS-vs-input precedence on `actorIdHash`, OTel trace capture, error propagation. ## End-to-end verification (manual, against local-dev Postgres) ``` INSERT under audit_writer: ok UPDATE under audit_writer: permission denied for table events DELETE under audit_writer: permission denied for table events DELETE under audit_archiver: ok, row removed (after the SELECT-grant fix) ``` ## ADR-0013 §Confirmation rewritten Two-block split: "wired in foundation PR" lists what landed here; "wired as features land" lists the typed event-family methods, AUDIT_DATABASE_URL connection split, startup self-test probe, retention purge job, salt-shared cross-correlation test, and live-DB role-contract integration tests — each anchored to the feature ADR that triggers it. ## Recovery for anyone with a pre-existing local-dev DB If your local-dev Postgres already had the audit migration applied **before** the SELECT-grant fix, the archiver's DELETE will fail. Two options: 1. Apply the missing grant directly: ```bash psql "$DATABASE_URL" -c "GRANT SELECT ON audit.events TO audit_archiver;" ``` 2. Or wipe the volume and re-migrate cleanly: ```bash ./infra/local/dev.sh down -v ./infra/local/dev.sh up pnpm --filter @apf-portal/source exec prisma migrate deploy # or `cd apps/portal-bff && pnpm exec prisma migrate deploy` ``` Fresh DBs land with the corrected migration directly. ## Out of scope (separate PRs) - Typed event-family methods (`signIn`, `signInFailed`, …) — added per matching feature ADR. - `AUDIT_DATABASE_URL` separate connection pool — defense-in-depth, when production needs it. - Startup self-test probe (deliberate failing UPDATE asserting rejection) — lands with the connection split. - Retention purge job (`audit_archiver` daily cron) — phase-3b infra. - Live-DB integration tests asserting the role contract — Testcontainers-style harness, separate PR. ## Test plan - [ ] CI green on this PR. - [ ] `prisma migrate deploy` succeeds on a fresh DB (the recovery instructions cover the SELECT-grant gap for already-migrated dev DBs). - [ ] `psql -c "\dp audit.events"` shows the expected privilege matrix: `audit_owner=arwdDxtm/audit_owner`, `audit_writer=a/audit_owner`, `audit_reader=r/audit_owner`, `audit_archiver=rd/audit_owner`. - [ ] BFF boots; calling `AuditWriter.recordEvent` from a controller (manual smoke once a real flow lands) writes to `audit.events` with the expected `trace_id` matching the request's Jaeger span. --------- Co-authored-by: Julien Gautier <julien.gautier@apf.asso.fr> Reviewed-on: #76 |
||
|
julien
|
b74d3f1b9b |
feat(portal-bff): observability foundations (Pino + CLS + OTel) (#70)
## Summary
Implements ADR-0012 phase 1, BFF side. The SPA wiring is a separate phase-2 PR.
The BFF now emits structured JSON logs to stdout, tagged with `trace_id` / `span_id` from the active OTel context, and exports OTLP traces over HTTP/Protobuf to the Collector that already runs in the local-dev compose. Anything Nest, Express, HTTP-out, Prisma (Postgres) or `ioredis` does is auto-spanned. A `GET /api/health` liveness endpoint is added to round things out.
## What lands
**Runtime libs added** (production deps):
- `nestjs-pino`, `pino`, `pino-http` — structured logging
- `nestjs-cls` — request-scoped context
- `@opentelemetry/api` / `sdk-node` / `resources` / `semantic-conventions`
- `@opentelemetry/exporter-trace-otlp-proto` (HTTP/Protobuf, port 4318)
- `@opentelemetry/instrumentation-{http,express,nestjs-core,pg,ioredis,pino}` — curated, **no** `auto-instrumentations-node` mass-import (anti-bricolage)
Dev: `pino-pretty` (gated by `NODE_ENV`).
**Code:**
- `apps/portal-bff/src/observability/tracing.ts` — OTel `NodeSDK` bootstrap. Documents the load-order constraint inline (must be the very first import of `main.ts`). Pure side-effect module.
- `apps/portal-bff/src/observability/observability.module.ts` — composes `ClsModule` (UUID per request stored as `request_id`) and `LoggerModule` (`pino-pretty` in dev, raw JSON in prod, `LOG_LEVEL` env-driven, `/health` excluded from auto-logging, `X-Request-Id` honoured if inbound).
- `apps/portal-bff/src/health/{health.controller,health.module,health.controller.spec}.ts` — `GET /api/health` returning `{status, uptimeSeconds, service, version}`. Cheap liveness only — `/readiness` lands when dependencies have a readiness story.
- `apps/portal-bff/src/config/check-database-url.{ts,spec.ts}` — fail-fast validator called from `main.ts` before NestFactory boots. Catches the same family of bug that bit pgweb in #63: a literal special character in `POSTGRES_PASSWORD` that needs URL-encoding in `DATABASE_URL`. Prisma requires a URL string (no discrete-flag escape hatch), so early validation + a clear error message is the v1 mitigation. Six unit tests cover happy path, missing URL, wrong scheme, encoded special chars, literal `@` in password, malformed URL.
**Wiring:**
- `main.ts` imports `./observability/tracing` as line 1, then uses `app.get(Logger)` from `nestjs-pino` with `bufferLogs: true` so early-bootstrap lines are not lost.
- `app.module.ts` imports `ObservabilityModule` first, then `PrismaModule`, then `HealthModule`.
- `apps/portal-bff/.env.example` promotes `LOG_LEVEL`, `OTEL_SERVICE_NAME`, `OTEL_SERVICE_VERSION`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_PROTOCOL`, `OTEL_TRACES_SAMPLER` from the "future" comment to active settings — defaults target the local-dev Collector.
- Both `apps/portal-bff/.env.example` and `infra/local/.env.example` now spell out the URL-encoding constraint on `POSTGRES_PASSWORD` with the char-by-char encoding table (`@` → `%40`, etc.).
**ADR-0012 §Confirmation** rewritten to distinguish what landed in this PR from what is wired as the corresponding feature ADRs ship (CLS keys for `session_id` / `user_id_hash` / `audience`, `LOG_USER_ID_SALT` enforcement, redact list, custom spans, SPA-side SDK, full integration tests, prod Collector config).
## Trace ↔ log correlation
Automatic via `@opentelemetry/instrumentation-pino` — every Pino record gets `trace_id` and `span_id` injected from the active OTel context. No CLS gymnastics needed for that concern.
## Verification
```bash
pnpm exec nx run-many -t lint test build # 8 projects green
pnpm audit --audit-level=moderate # 0 vulnerabilities
./infra/local/dev.sh up observability # start Collector + Jaeger
cp apps/portal-bff/.env.example apps/portal-bff/.env
pnpm nx serve portal-bff
curl http://localhost:3000/api/health
# → {"status":"ok","uptimeSeconds":N,"service":"portal-bff","version":"dev"}
```
Then hit `GET http://localhost:3000/api` once or twice and open http://localhost:16686 — the corresponding spans appear in Jaeger, and Pino logs on stdout carry the matching `trace_id`.
## Test plan
- [ ] `nx run-many -t lint test build` green on this PR's CI run.
- [ ] `pnpm audit` clean.
- [ ] BFF boots, `/api/health` returns the expected JSON.
- [ ] Pino logs in dev are colourised one-liners; in prod they would be raw JSON (toggled by `NODE_ENV=production`).
- [ ] With the local-dev stack's `--profile observability` active, traces are visible in Jaeger UI.
- [ ] Each Pino log line for a request carries the same `trace_id` as the trace span in Jaeger.
---------
Co-authored-by: Julien Gautier <julien.gautier@apf.asso.fr>
Reviewed-on: #70
|
||
 Julien Gautier
|
2b0e20bd85 |
chore: wire PostgreSQL + Prisma per ADR-0006
Add Prisma 7 + nestjs-prisma. The schema lives at
apps/portal-bff/prisma/schema.prisma with provider postgresql; the new
prisma-client generator (Prisma 7 default) outputs the typed client to
apps/portal-bff/generated/prisma/ which is gitignored.
apps/portal-bff/src/app/app.module.ts imports PrismaModule.forRoot
({ isGlobal: true }) so PrismaService is injectable across the BFF
without per-module imports.
apps/portal-bff/.env.example documents DATABASE_URL with a local-dev
default, plus a forward list of env vars introduced by upcoming phases
and ADRs (auth, sessions, MFA, observability, audit, downstream APIs)
- catalog reference, not implementation. The actual .env stays
gitignored at both repo root and app levels.
prisma.config.ts (Prisma 7's TypeScript config) is committed; it loads
DATABASE_URL via dotenv. Schema and migrations paths are pinned to
prisma/ relative to the bff app.
PostgreSQL provisioning, RLS policies for the dual-audience design,
the dedicated audit schema with role grants (audit_owner / audit_writer
/ audit_reader / audit_archiver per ADR-0013), and column-level
encryption for L3-scoped data are out of scope of this commit -
they belong with the future on-prem infrastructure ADR.
|
||
|
Julien Gautier
|
bea5e1954f |
chore: generate portal-shell and portal-bff apps per ADR-0004 / ADR-0005
Add the @nx/angular, @nx/nest, @nx/vite, @nx/eslint plugins, then generate the two apps. Adjust the empty-template tsconfig.base.json to be Angular-compatible (drop project references and customConditions that the empty-template defaults to but Angular doesn't support; keep the strict-TS extensions from ADR-0004). apps/portal-shell (Angular 21): - standalone APIs, routing, SCSS, esbuild - vitest-angular as unitTestRunner, playwright for e2e - strict mode - tags scope:portal-shell, type:app - app.config.ts wired with provideZonelessChangeDetection() per ADR-0004 (Angular 21 + Nx 22 generates without zone.js by default) apps/portal-bff (NestJS 11): - Express adapter (default per ADR-0005) - Jest as unitTestRunner - tags scope:portal-bff, type:app - main.ts wired with a global ValidationPipe configured whitelist + forbidNonWhitelisted + transform per ADR-0005 - Phase-2 security additions (helmet, CORS, sessions, CSRF, rate limit, auth guards, error filter) deferred to their respective ADRs - placeholder comment in main.ts Workspace dependencies: class-validator + class-transformer added (required by NestJS ValidationPipe at runtime). Nx-generated .gitignore additions (.angular, __screenshots__) merged into ours. .vscode/extensions.json and launch.json added by Nx are kept (do not override our existing settings.json). |