3a44a1deedbf6e3d529617d96862018d1c548a6f
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Julien Gautier
|
3a44a1deed |
feat(portal-bff): audit log foundation per ADR-0013
Lays down the append-only audit log: schema, migration with role grants, NestJS AuditWriter service. Typed event-family methods, separate AUDIT_DATABASE_URL pool, retention job, and live-DB integration tests are explicitly listed as "wired as features land" in ADR-0013 §Confirmation. Schema (apps/portal-bff/prisma/schema.prisma): - multiSchema preview enabled; datasource declares public + audit schemas. - AuditEvent model with: id (uuid), createdAt, eventType (free-form in v1, will formalise into a catalogue once we have N stable types), audience (workforce|customer enum), actorIdHash, traceId, subject, outcome (success|failure|denied enum), payload (jsonb). - Indexes on createdAt, eventType, traceId — covering the obvious query shapes (date-range scan, by event family, by trace for log-audit correlation). Migration (prisma/migrations/*_init_audit_schema/migration.sql): - CREATE TABLE / enums via Prisma's standard output. - Append-only contract re-applied explicitly: ALTER TABLE / TYPE OWNER TO audit_owner, then GRANT INSERT to audit_writer, SELECT to audit_reader, SELECT+DELETE to audit_archiver. SELECT is required for archiver because Postgres needs SELECT on every column referenced in DELETE's WHERE clause to evaluate "older than retention" — granted explicitly in this PR rather than silently failing later. - USAGE on the enum types granted to all three roles so audit_ writer's INSERT does not fail with "permission denied for type". - No GRANT for UPDATE / TRUNCATE to anyone, including audit_owner at runtime — only fresh schema migrations amend the table. Service (apps/portal-bff/src/audit/): - AuditWriter.recordEvent(input) — single entry point. Wraps every INSERT in a transaction whose first statement is `SET LOCAL ROLE audit_writer`, so the role contract holds at runtime even from the otherwise-privileged BFF connection. SET LOCAL is reset on COMMIT/ROLLBACK so the pool's next consumer sees the original role. - traceId auto-resolved from the active OTel span context. - actorIdHash auto-resolved from CLS (key 'actorIdHash') with explicit input-side override; null when neither is set (placeholder until ADR-0009 / ADR-0010 guards populate CLS). - Errors propagate (no catch-and-swallow), per ADR-0013's "blocking writes: no audit ⇒ no action". - AuditModule exposes the writer; wired into AppModule so any future feature module can inject it. Tests (apps/portal-bff/src/audit/audit.service.spec.ts, 8 cases): - Locks the transaction to audit_writer before INSERTing. - Passes input fields through. - Records Prisma.JsonNull when no payload is provided. - Reads actorIdHash from CLS when not passed. - Prefers explicit actorIdHash over CLS-resolved. - Stores actorIdHash = null when neither input nor CLS has one. - Captures the active OTel trace id. - Stores traceId = null when no span is active. - Propagates the underlying error (no swallow). End-to-end smoke against local-dev Postgres (manual, via psql): - INSERT under audit_writer: ok. - UPDATE under audit_writer: "permission denied for table events". - DELETE under audit_writer: "permission denied for table events". - DELETE under audit_archiver (after the SELECT grant fix): ok, row removed. ADR-0013 §Confirmation rewritten as "wired in foundation PR" / "wired as features land", with the latter listing the typed event-family methods, AUDIT_DATABASE_URL split, startup self-test probe, retention purge job, salt-shared cross-correlation test, and live-DB role-contract integration tests. |
||
 julien
|
8f2cd4e068 |
feat(portal-shell): wire spa-side opentelemetry tracing (#72)
## Summary Phase 2 of ADR-0012 — closes the loop SPA → BFF → DB. After this PR, a single user action (initial page load, click, form submit) produces one trace whose root span is owned by the SPA and whose child spans cover the BFF request, Postgres queries through Prisma, and (eventually) Redis / downstream-API hops. ## What lands **Browser-side OTel libs** (production deps): - `@opentelemetry/sdk-trace-web` — browser tracer + provider - `@opentelemetry/exporter-trace-otlp-http` — OTLP/HTTP+JSON exporter - `@opentelemetry/instrumentation` — auto-instrumentation runtime - `@opentelemetry/instrumentation-fetch` — `fetch` + W3C `traceparent` propagation - `@opentelemetry/instrumentation-document-load` — initial-paint timings - `@opentelemetry/instrumentation-user-interaction` — click / keypress / submit No `@opentelemetry/context-zone`: the workspace is zoneless per ADR-0004; the default `StackContextManager` covers the auto-instrumented paths. Custom spans across `await` will need explicit `context.with(...)` plumbing — fine, encountered as code lands. **Code**: - [`apps/portal-shell/src/observability/tracing.ts`](apps/portal-shell/src/observability/tracing.ts) — `WebTracerProvider` bootstrap. Documents the load-order constraint inline (same pattern as the BFF: must be the very first import of `main.ts`, otherwise auto-instrumentations miss everything imported above). - `apps/portal-shell/src/main.ts` now imports the tracing module as line 1. **CORS plumbing** for end-to-end trace propagation: - BFF (`apps/portal-bff/src/main.ts`) calls `enableCors` with a minimal dev allowlist (`http://localhost:4200`) and explicit permission for the W3C `traceparent` / `tracestate` headers. The full security-grade CORS (per-environment allowlists, helmet, cookie-session, CSRF) belongs to the future phase-2 security ADR — this PR adds the strict minimum for the SPA→BFF trace context to survive cross-origin pre-flight. - OTel Collector (`infra/local/otel-collector.yaml`) gains a `cors` block on its OTLP/HTTP receiver so the browser's own OTLP POST clears its pre-flight. **ADR-0012 §Confirmation** rewritten: a new "Wired in the SPA foundation PR (phase 2)" block enumerates what landed here; the carry-over "Wired as features land" list drops the SPA-side SDK item and adds a follow-up note about the security-grade CORS. ## Verification ```bash pnpm exec nx run-many -t lint test build # 8 projects green pnpm audit # 0 vulns ./infra/local/dev.sh up observability # bring up Collector + Jaeger ./infra/local/dev.sh # (separately, BFF stack — your choice) pnpm nx serve portal-bff # localhost:3000 pnpm nx serve portal-shell # localhost:4200 ``` Open http://localhost:4200 → a `document_load` trace appears in http://localhost:16686 with `service.name=portal-shell`. From DevTools, run `fetch('http://localhost:3000/api/health').then(r => r.json())` → a fetch span appears with a child BFF span on the same trace. ## Test plan - [ ] CI green on this PR. - [ ] After local up, `document_load` span visible in Jaeger UI for the SPA. - [ ] Cross-origin fetch from SPA carries `traceparent` (visible in Network tab) and produces a single end-to-end trace SPA → BFF in Jaeger. - [ ] DevTools console shows no CORS warnings about `traceparent`, `tracestate`, or the `localhost:4318/v1/traces` POST. --------- Co-authored-by: Julien Gautier <julien.gautier@apf.asso.fr> Reviewed-on: #72 |
||
|
julien
|
c3c15585ff |
style(portal-bff): apply prettier to check-database-url.ts (#71)
## Summary Cosmetic-only follow-up to #70. Collapses one over-cautious multi-line `throw new Error(...)` into a single line that fits within the project's 100-char `printWidth`. The reformat was produced by lint-staged **during** the amend that landed #70, but applied to the working tree only — the modification never made it back into the staged content of the amend. The merged commit therefore carries the un-prettified version. Spotted locally with `pnpm exec prettier --check apps/portal-bff/src/config/check-database-url.ts` failing. Left unchecked, the next PR's `check` job would break at `format:check` for unrelated reasons. ## Test plan - [ ] CI green on this PR (`format:check` clean). - [ ] No behavioural change — only whitespace. --------- Co-authored-by: Julien Gautier <julien.gautier@apf.asso.fr> Reviewed-on: #71 |
||
|
julien
|
b74d3f1b9b |
feat(portal-bff): observability foundations (Pino + CLS + OTel) (#70)
## Summary
Implements ADR-0012 phase 1, BFF side. The SPA wiring is a separate phase-2 PR.
The BFF now emits structured JSON logs to stdout, tagged with `trace_id` / `span_id` from the active OTel context, and exports OTLP traces over HTTP/Protobuf to the Collector that already runs in the local-dev compose. Anything Nest, Express, HTTP-out, Prisma (Postgres) or `ioredis` does is auto-spanned. A `GET /api/health` liveness endpoint is added to round things out.
## What lands
**Runtime libs added** (production deps):
- `nestjs-pino`, `pino`, `pino-http` — structured logging
- `nestjs-cls` — request-scoped context
- `@opentelemetry/api` / `sdk-node` / `resources` / `semantic-conventions`
- `@opentelemetry/exporter-trace-otlp-proto` (HTTP/Protobuf, port 4318)
- `@opentelemetry/instrumentation-{http,express,nestjs-core,pg,ioredis,pino}` — curated, **no** `auto-instrumentations-node` mass-import (anti-bricolage)
Dev: `pino-pretty` (gated by `NODE_ENV`).
**Code:**
- `apps/portal-bff/src/observability/tracing.ts` — OTel `NodeSDK` bootstrap. Documents the load-order constraint inline (must be the very first import of `main.ts`). Pure side-effect module.
- `apps/portal-bff/src/observability/observability.module.ts` — composes `ClsModule` (UUID per request stored as `request_id`) and `LoggerModule` (`pino-pretty` in dev, raw JSON in prod, `LOG_LEVEL` env-driven, `/health` excluded from auto-logging, `X-Request-Id` honoured if inbound).
- `apps/portal-bff/src/health/{health.controller,health.module,health.controller.spec}.ts` — `GET /api/health` returning `{status, uptimeSeconds, service, version}`. Cheap liveness only — `/readiness` lands when dependencies have a readiness story.
- `apps/portal-bff/src/config/check-database-url.{ts,spec.ts}` — fail-fast validator called from `main.ts` before NestFactory boots. Catches the same family of bug that bit pgweb in #63: a literal special character in `POSTGRES_PASSWORD` that needs URL-encoding in `DATABASE_URL`. Prisma requires a URL string (no discrete-flag escape hatch), so early validation + a clear error message is the v1 mitigation. Six unit tests cover happy path, missing URL, wrong scheme, encoded special chars, literal `@` in password, malformed URL.
**Wiring:**
- `main.ts` imports `./observability/tracing` as line 1, then uses `app.get(Logger)` from `nestjs-pino` with `bufferLogs: true` so early-bootstrap lines are not lost.
- `app.module.ts` imports `ObservabilityModule` first, then `PrismaModule`, then `HealthModule`.
- `apps/portal-bff/.env.example` promotes `LOG_LEVEL`, `OTEL_SERVICE_NAME`, `OTEL_SERVICE_VERSION`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_PROTOCOL`, `OTEL_TRACES_SAMPLER` from the "future" comment to active settings — defaults target the local-dev Collector.
- Both `apps/portal-bff/.env.example` and `infra/local/.env.example` now spell out the URL-encoding constraint on `POSTGRES_PASSWORD` with the char-by-char encoding table (`@` → `%40`, etc.).
**ADR-0012 §Confirmation** rewritten to distinguish what landed in this PR from what is wired as the corresponding feature ADRs ship (CLS keys for `session_id` / `user_id_hash` / `audience`, `LOG_USER_ID_SALT` enforcement, redact list, custom spans, SPA-side SDK, full integration tests, prod Collector config).
## Trace ↔ log correlation
Automatic via `@opentelemetry/instrumentation-pino` — every Pino record gets `trace_id` and `span_id` injected from the active OTel context. No CLS gymnastics needed for that concern.
## Verification
```bash
pnpm exec nx run-many -t lint test build # 8 projects green
pnpm audit --audit-level=moderate # 0 vulnerabilities
./infra/local/dev.sh up observability # start Collector + Jaeger
cp apps/portal-bff/.env.example apps/portal-bff/.env
pnpm nx serve portal-bff
curl http://localhost:3000/api/health
# → {"status":"ok","uptimeSeconds":N,"service":"portal-bff","version":"dev"}
```
Then hit `GET http://localhost:3000/api` once or twice and open http://localhost:16686 — the corresponding spans appear in Jaeger, and Pino logs on stdout carry the matching `trace_id`.
## Test plan
- [ ] `nx run-many -t lint test build` green on this PR's CI run.
- [ ] `pnpm audit` clean.
- [ ] BFF boots, `/api/health` returns the expected JSON.
- [ ] Pino logs in dev are colourised one-liners; in prod they would be raw JSON (toggled by `NODE_ENV=production`).
- [ ] With the local-dev stack's `--profile observability` active, traces are visible in Jaeger UI.
- [ ] Each Pino log line for a request carries the same `trace_id` as the trace span in Jaeger.
---------
Co-authored-by: Julien Gautier <julien.gautier@apf.asso.fr>
Reviewed-on: #70
|
||
|
Julien Gautier
|
2b0e20bd85 |
chore: wire PostgreSQL + Prisma per ADR-0006
Add Prisma 7 + nestjs-prisma. The schema lives at
apps/portal-bff/prisma/schema.prisma with provider postgresql; the new
prisma-client generator (Prisma 7 default) outputs the typed client to
apps/portal-bff/generated/prisma/ which is gitignored.
apps/portal-bff/src/app/app.module.ts imports PrismaModule.forRoot
({ isGlobal: true }) so PrismaService is injectable across the BFF
without per-module imports.
apps/portal-bff/.env.example documents DATABASE_URL with a local-dev
default, plus a forward list of env vars introduced by upcoming phases
and ADRs (auth, sessions, MFA, observability, audit, downstream APIs)
- catalog reference, not implementation. The actual .env stays
gitignored at both repo root and app levels.
prisma.config.ts (Prisma 7's TypeScript config) is committed; it loads
DATABASE_URL via dotenv. Schema and migrations paths are pinned to
prisma/ relative to the bff app.
PostgreSQL provisioning, RLS policies for the dual-audience design,
the dedicated audit schema with role grants (audit_owner / audit_writer
/ audit_reader / audit_archiver per ADR-0013), and column-level
encryption for L3-scoped data are out of scope of this commit -
they belong with the future on-prem infrastructure ADR.
|
||
|
Julien Gautier
|
0774014599 |
fix(portal-bff): use bracket notation for process.env access
The strict-TS option noPropertyAccessFromIndexSignature: true (set in
tsconfig.base.json per ADR-0004) forbids dot-notation access on index
signatures. process.env is typed as { [key: string]: string | undefined }
so process.env.PORT must be written process.env['PORT']. The Nx
generator wrote the dot form by default; fix to comply with the
project's strict-TS bar.
Touched: portal-bff main.ts and the three portal-bff-e2e support files
(global-setup, global-teardown, test-setup).
|
||
|
Julien Gautier
|
bea5e1954f |
chore: generate portal-shell and portal-bff apps per ADR-0004 / ADR-0005
Add the @nx/angular, @nx/nest, @nx/vite, @nx/eslint plugins, then generate the two apps. Adjust the empty-template tsconfig.base.json to be Angular-compatible (drop project references and customConditions that the empty-template defaults to but Angular doesn't support; keep the strict-TS extensions from ADR-0004). apps/portal-shell (Angular 21): - standalone APIs, routing, SCSS, esbuild - vitest-angular as unitTestRunner, playwright for e2e - strict mode - tags scope:portal-shell, type:app - app.config.ts wired with provideZonelessChangeDetection() per ADR-0004 (Angular 21 + Nx 22 generates without zone.js by default) apps/portal-bff (NestJS 11): - Express adapter (default per ADR-0005) - Jest as unitTestRunner - tags scope:portal-bff, type:app - main.ts wired with a global ValidationPipe configured whitelist + forbidNonWhitelisted + transform per ADR-0005 - Phase-2 security additions (helmet, CORS, sessions, CSRF, rate limit, auth guards, error filter) deferred to their respective ADRs - placeholder comment in main.ts Workspace dependencies: class-validator + class-transformer added (required by NestJS ValidationPipe at runtime). Nx-generated .gitignore additions (.angular, __screenshots__) merged into ours. .vscode/extensions.json and launch.json added by Nx are kept (do not override our existing settings.json). |