feat(portal-bff): observability foundations (Pino + CLS + OTel) (#70)
## Summary
Implements ADR-0012 phase 1, BFF side. The SPA wiring is a separate phase-2 PR.
The BFF now emits structured JSON logs to stdout, tagged with `trace_id` / `span_id` from the active OTel context, and exports OTLP traces over HTTP/Protobuf to the Collector that already runs in the local-dev compose. Anything Nest, Express, HTTP-out, Prisma (Postgres) or `ioredis` does is auto-spanned. A `GET /api/health` liveness endpoint is added to round things out.
## What lands
**Runtime libs added** (production deps):
- `nestjs-pino`, `pino`, `pino-http` — structured logging
- `nestjs-cls` — request-scoped context
- `@opentelemetry/api` / `sdk-node` / `resources` / `semantic-conventions`
- `@opentelemetry/exporter-trace-otlp-proto` (HTTP/Protobuf, port 4318)
- `@opentelemetry/instrumentation-{http,express,nestjs-core,pg,ioredis,pino}` — curated, **no** `auto-instrumentations-node` mass-import (anti-bricolage)
Dev: `pino-pretty` (gated by `NODE_ENV`).
**Code:**
- `apps/portal-bff/src/observability/tracing.ts` — OTel `NodeSDK` bootstrap. Documents the load-order constraint inline (must be the very first import of `main.ts`). Pure side-effect module.
- `apps/portal-bff/src/observability/observability.module.ts` — composes `ClsModule` (UUID per request stored as `request_id`) and `LoggerModule` (`pino-pretty` in dev, raw JSON in prod, `LOG_LEVEL` env-driven, `/health` excluded from auto-logging, `X-Request-Id` honoured if inbound).
- `apps/portal-bff/src/health/{health.controller,health.module,health.controller.spec}.ts` — `GET /api/health` returning `{status, uptimeSeconds, service, version}`. Cheap liveness only — `/readiness` lands when dependencies have a readiness story.
- `apps/portal-bff/src/config/check-database-url.{ts,spec.ts}` — fail-fast validator called from `main.ts` before NestFactory boots. Catches the same family of bug that bit pgweb in #63: a literal special character in `POSTGRES_PASSWORD` that needs URL-encoding in `DATABASE_URL`. Prisma requires a URL string (no discrete-flag escape hatch), so early validation + a clear error message is the v1 mitigation. Six unit tests cover happy path, missing URL, wrong scheme, encoded special chars, literal `@` in password, malformed URL.
**Wiring:**
- `main.ts` imports `./observability/tracing` as line 1, then uses `app.get(Logger)` from `nestjs-pino` with `bufferLogs: true` so early-bootstrap lines are not lost.
- `app.module.ts` imports `ObservabilityModule` first, then `PrismaModule`, then `HealthModule`.
- `apps/portal-bff/.env.example` promotes `LOG_LEVEL`, `OTEL_SERVICE_NAME`, `OTEL_SERVICE_VERSION`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_PROTOCOL`, `OTEL_TRACES_SAMPLER` from the "future" comment to active settings — defaults target the local-dev Collector.
- Both `apps/portal-bff/.env.example` and `infra/local/.env.example` now spell out the URL-encoding constraint on `POSTGRES_PASSWORD` with the char-by-char encoding table (`@` → `%40`, etc.).
**ADR-0012 §Confirmation** rewritten to distinguish what landed in this PR from what is wired as the corresponding feature ADRs ship (CLS keys for `session_id` / `user_id_hash` / `audience`, `LOG_USER_ID_SALT` enforcement, redact list, custom spans, SPA-side SDK, full integration tests, prod Collector config).
## Trace ↔ log correlation
Automatic via `@opentelemetry/instrumentation-pino` — every Pino record gets `trace_id` and `span_id` injected from the active OTel context. No CLS gymnastics needed for that concern.
## Verification
```bash
pnpm exec nx run-many -t lint test build # 8 projects green
pnpm audit --audit-level=moderate # 0 vulnerabilities
./infra/local/dev.sh up observability # start Collector + Jaeger
cp apps/portal-bff/.env.example apps/portal-bff/.env
pnpm nx serve portal-bff
curl http://localhost:3000/api/health
# → {"status":"ok","uptimeSeconds":N,"service":"portal-bff","version":"dev"}
```
Then hit `GET http://localhost:3000/api` once or twice and open http://localhost:16686 — the corresponding spans appear in Jaeger, and Pino logs on stdout carry the matching `trace_id`.
## Test plan
- [ ] `nx run-many -t lint test build` green on this PR's CI run.
- [ ] `pnpm audit` clean.
- [ ] BFF boots, `/api/health` returns the expected JSON.
- [ ] Pino logs in dev are colourised one-liners; in prod they would be raw JSON (toggled by `NODE_ENV=production`).
- [ ] With the local-dev stack's `--profile observability` active, traces are visible in Jaeger UI.
- [ ] Each Pino log line for a request carries the same `trace_id` as the trace span in Jaeger.
---------
Co-authored-by: Julien Gautier <julien.gautier@apf.asso.fr>
Reviewed-on: #70
This commit was merged in pull request #70.
This commit is contained in:
@@ -188,18 +188,25 @@ The BFF refuses to start if `LOG_USER_ID_SALT` or `OTEL_SERVICE_NAME` is missing
|
||||
|
||||
### Confirmation
|
||||
|
||||
- `apps/portal-bff/src/observability/observability.module.ts` registers `nestjs-pino`, `nestjs-cls`, the OTel SDK initialiser, and the redact-list test fixture.
|
||||
- `apps/portal-bff/src/main.ts` calls `tracingSetup()` _before_ `NestFactory.create(AppModule)` — OTel must be initialised before any auto-instrumented module is loaded.
|
||||
- The Pino transport in dev uses `pino-pretty` (human-readable); prod emits raw JSON.
|
||||
- `apps/portal-shell/src/observability/tracing.ts` initialises `@opentelemetry/sdk-trace-web` with the OTLP/HTTP exporter pointing to the BFF's `/v1/traces` ingress (or directly to the collector if exposed).
|
||||
- Every controller receives `req` and `res` and the http auto-instrumentation produces a span; non-trivial service methods open custom spans via `tracer.startActiveSpan('domain.<verb>', ...)` and propagate the CLS context within them.
|
||||
- Integration tests:
|
||||
- a single user-action request emits one trace with correct parent-child structure (SPA → BFF → DB);
|
||||
- every log line for that request carries the same `trace_id`;
|
||||
- the redact list strips its targets from a captured stream;
|
||||
- a missing `LOG_USER_ID_SALT` prevents BFF startup.
|
||||
- The CLS-bound `user_id_hash` is identical for two consecutive requests of the same user, and different across environments (different salt).
|
||||
- The OTel Collector configuration ships in the deployment manifest (covered by the future infrastructure ADR), not in source — the application remains backend-agnostic.
|
||||
**Wired in the BFF foundation PR (phase 1):**
|
||||
|
||||
- `apps/portal-bff/src/observability/tracing.ts` initialises the OTel `NodeSDK` with the OTLP HTTP/Protobuf exporter (target: `OTEL_EXPORTER_OTLP_ENDPOINT`, default `http://localhost:4318/v1/traces`).
|
||||
- `apps/portal-bff/src/main.ts` `import`s `./observability/tracing` as its very first line — anything above it bypasses auto-instrumentation.
|
||||
- `apps/portal-bff/src/observability/observability.module.ts` registers `nestjs-cls` (mounted as global middleware, populates `request_id` as a per-request UUID) and `nestjs-pino` (`LoggerModule`, `pino-pretty` in dev / raw JSON in prod, `LOG_LEVEL` env-driven, `/health` excluded from auto-logging).
|
||||
- Auto-instrumentations active in v1: HTTP, Express, NestJS, PostgreSQL (`pg` driver under Prisma), IORedis (pre-wired for ADR-0010 / ADR-0014), and Pino. The Pino instrumentation auto-decorates every log record with `trace_id` and `span_id` from the active OTel context, so log ↔ trace correlation is automatic.
|
||||
- `apps/portal-bff/src/health/health.controller.ts` exposes `GET /api/health` (cheap liveness — process metadata only, no DB / Redis / downstream check).
|
||||
- `apps/portal-bff/.env.example` declares `LOG_LEVEL`, `OTEL_SERVICE_NAME`, `OTEL_SERVICE_VERSION`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_PROTOCOL`, `OTEL_TRACES_SAMPLER` with sensible dev defaults.
|
||||
|
||||
**Wired as the corresponding features land:**
|
||||
|
||||
- CLS keys `session_id`, `user_id_hash`, `audience` — populated by guards/interceptors when ADR-0009 / ADR-0010 land.
|
||||
- `LOG_USER_ID_SALT` enforced (BFF refuses to start without it) — same trigger.
|
||||
- Pino `redact` list for PII paths — wired alongside the first DTO that carries a redactable field.
|
||||
- Custom spans `tracer.startActiveSpan('domain.<verb>', …)` — added per service method that warrants one (creating a row, calling a downstream, etc.).
|
||||
- SPA-side `@opentelemetry/sdk-trace-web` initialiser, OTLP/HTTP exporter targeting the Collector, `traceparent` header propagated to the BFF — phase-2 PR.
|
||||
- Integration tests: full SPA → BFF → DB trace with correct parent-child structure; every log line for one request carries the same `trace_id`; redact list strips its targets — wired with the auth + first real screen.
|
||||
- A separate `/readiness` endpoint that probes Postgres / Redis / OBO cache — wired with the first dependency that has a readiness story to tell.
|
||||
- OTel Collector deployment configuration in the runtime manifest — phase 3b on-prem ADR. The application stays backend-agnostic (any OTLP-capable Collector works).
|
||||
|
||||
## Pros and Cons of the Options
|
||||
|
||||
|
||||
Reference in New Issue
Block a user