feat(portal-bff): observability foundations (Pino + CLS + OTel) (#70)
## Summary
Implements ADR-0012 phase 1, BFF side. The SPA wiring is a separate phase-2 PR.
The BFF now emits structured JSON logs to stdout, tagged with `trace_id` / `span_id` from the active OTel context, and exports OTLP traces over HTTP/Protobuf to the Collector that already runs in the local-dev compose. Anything Nest, Express, HTTP-out, Prisma (Postgres) or `ioredis` does is auto-spanned. A `GET /api/health` liveness endpoint is added to round things out.
## What lands
**Runtime libs added** (production deps):
- `nestjs-pino`, `pino`, `pino-http` — structured logging
- `nestjs-cls` — request-scoped context
- `@opentelemetry/api` / `sdk-node` / `resources` / `semantic-conventions`
- `@opentelemetry/exporter-trace-otlp-proto` (HTTP/Protobuf, port 4318)
- `@opentelemetry/instrumentation-{http,express,nestjs-core,pg,ioredis,pino}` — curated, **no** `auto-instrumentations-node` mass-import (anti-bricolage)
Dev: `pino-pretty` (gated by `NODE_ENV`).
**Code:**
- `apps/portal-bff/src/observability/tracing.ts` — OTel `NodeSDK` bootstrap. Documents the load-order constraint inline (must be the very first import of `main.ts`). Pure side-effect module.
- `apps/portal-bff/src/observability/observability.module.ts` — composes `ClsModule` (UUID per request stored as `request_id`) and `LoggerModule` (`pino-pretty` in dev, raw JSON in prod, `LOG_LEVEL` env-driven, `/health` excluded from auto-logging, `X-Request-Id` honoured if inbound).
- `apps/portal-bff/src/health/{health.controller,health.module,health.controller.spec}.ts` — `GET /api/health` returning `{status, uptimeSeconds, service, version}`. Cheap liveness only — `/readiness` lands when dependencies have a readiness story.
- `apps/portal-bff/src/config/check-database-url.{ts,spec.ts}` — fail-fast validator called from `main.ts` before NestFactory boots. Catches the same family of bug that bit pgweb in #63: a literal special character in `POSTGRES_PASSWORD` that needs URL-encoding in `DATABASE_URL`. Prisma requires a URL string (no discrete-flag escape hatch), so early validation + a clear error message is the v1 mitigation. Six unit tests cover happy path, missing URL, wrong scheme, encoded special chars, literal `@` in password, malformed URL.
**Wiring:**
- `main.ts` imports `./observability/tracing` as line 1, then uses `app.get(Logger)` from `nestjs-pino` with `bufferLogs: true` so early-bootstrap lines are not lost.
- `app.module.ts` imports `ObservabilityModule` first, then `PrismaModule`, then `HealthModule`.
- `apps/portal-bff/.env.example` promotes `LOG_LEVEL`, `OTEL_SERVICE_NAME`, `OTEL_SERVICE_VERSION`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_PROTOCOL`, `OTEL_TRACES_SAMPLER` from the "future" comment to active settings — defaults target the local-dev Collector.
- Both `apps/portal-bff/.env.example` and `infra/local/.env.example` now spell out the URL-encoding constraint on `POSTGRES_PASSWORD` with the char-by-char encoding table (`@` → `%40`, etc.).
**ADR-0012 §Confirmation** rewritten to distinguish what landed in this PR from what is wired as the corresponding feature ADRs ship (CLS keys for `session_id` / `user_id_hash` / `audience`, `LOG_USER_ID_SALT` enforcement, redact list, custom spans, SPA-side SDK, full integration tests, prod Collector config).
## Trace ↔ log correlation
Automatic via `@opentelemetry/instrumentation-pino` — every Pino record gets `trace_id` and `span_id` injected from the active OTel context. No CLS gymnastics needed for that concern.
## Verification
```bash
pnpm exec nx run-many -t lint test build # 8 projects green
pnpm audit --audit-level=moderate # 0 vulnerabilities
./infra/local/dev.sh up observability # start Collector + Jaeger
cp apps/portal-bff/.env.example apps/portal-bff/.env
pnpm nx serve portal-bff
curl http://localhost:3000/api/health
# → {"status":"ok","uptimeSeconds":N,"service":"portal-bff","version":"dev"}
```
Then hit `GET http://localhost:3000/api` once or twice and open http://localhost:16686 — the corresponding spans appear in Jaeger, and Pino logs on stdout carry the matching `trace_id`.
## Test plan
- [ ] `nx run-many -t lint test build` green on this PR's CI run.
- [ ] `pnpm audit` clean.
- [ ] BFF boots, `/api/health` returns the expected JSON.
- [ ] Pino logs in dev are colourised one-liners; in prod they would be raw JSON (toggled by `NODE_ENV=production`).
- [ ] With the local-dev stack's `--profile observability` active, traces are visible in Jaeger UI.
- [ ] Each Pino log line for a request carries the same `trace_id` as the trace span in Jaeger.
---------
Co-authored-by: Julien Gautier <julien.gautier@apf.asso.fr>
Reviewed-on: #70
This commit was merged in pull request #70.
This commit is contained in:
@@ -0,0 +1,83 @@
|
||||
/**
|
||||
* Observability wiring for the BFF (per ADR-0012).
|
||||
*
|
||||
* Composes:
|
||||
* - `nestjs-cls` for request-scoped context. v1 stores `request_id`
|
||||
* (UUID per inbound request); `session_id`, `user_id_hash`, and
|
||||
* `audience` keys are reserved for future ADR-0009 / ADR-0010
|
||||
* work and are populated by guards/interceptors as those modules
|
||||
* land.
|
||||
* - `nestjs-pino` for structured JSON logs. In dev (`NODE_ENV !==
|
||||
* 'production'`) we pipe through `pino-pretty` so the developer
|
||||
* sees colourised, human-readable lines on stdout. In prod the
|
||||
* raw JSON output is shipped via the container runtime's stdout
|
||||
* pipeline (per ADR-0012 §"stdout + OTLP shipping").
|
||||
*
|
||||
* Trace ↔ log correlation: handled outside this module by
|
||||
* `@opentelemetry/instrumentation-pino` (loaded in `tracing.ts`),
|
||||
* which auto-injects `trace_id` / `span_id` into every Pino log
|
||||
* record from the active OTel context.
|
||||
*/
|
||||
|
||||
import { Module } from '@nestjs/common';
|
||||
import { ClsModule } from 'nestjs-cls';
|
||||
import { LoggerModule } from 'nestjs-pino';
|
||||
import { randomUUID } from 'node:crypto';
|
||||
|
||||
const isProduction = process.env['NODE_ENV'] === 'production';
|
||||
const logLevel = process.env['LOG_LEVEL'] ?? (isProduction ? 'info' : 'debug');
|
||||
|
||||
@Module({
|
||||
imports: [
|
||||
ClsModule.forRoot({
|
||||
global: true,
|
||||
middleware: {
|
||||
mount: true,
|
||||
generateId: true,
|
||||
// UUID v4 per request — exposed to logs as `request_id` and
|
||||
// available everywhere via `cls.get('request_id')`. Stored
|
||||
// under the conventional `id` key by nestjs-cls; we
|
||||
// surface it through Pino's `customProps` below.
|
||||
idGenerator: () => randomUUID(),
|
||||
},
|
||||
}),
|
||||
LoggerModule.forRoot({
|
||||
pinoHttp: {
|
||||
level: logLevel,
|
||||
// Pretty-print only in dev. In prod the JSON stream is
|
||||
// consumed by the container runtime / Collector.
|
||||
...(isProduction
|
||||
? {}
|
||||
: {
|
||||
transport: {
|
||||
target: 'pino-pretty',
|
||||
options: {
|
||||
singleLine: true,
|
||||
translateTime: 'SYS:HH:MM:ss.l',
|
||||
},
|
||||
},
|
||||
}),
|
||||
// Hide some noise that has nothing to do with the work being
|
||||
// done. Health-check pings would drown out everything else.
|
||||
autoLogging: {
|
||||
ignore: (req) => req.url === '/api/health' || req.url === '/health',
|
||||
},
|
||||
// Re-key the request id so it surfaces under `request_id`
|
||||
// (snake_case, ADR-0012 convention) instead of pino's
|
||||
// default `req.id` / `reqId`.
|
||||
customProps: () => ({}),
|
||||
genReqId: (req) => {
|
||||
// Honour an inbound `X-Request-Id` header when present —
|
||||
// useful for end-to-end tracing across systems that propagate
|
||||
// it. Otherwise generate a fresh UUID v4. The same id is
|
||||
// used by the CLS middleware above (Nest-Pino integrates
|
||||
// with nestjs-cls automatically when both modules are
|
||||
// imported).
|
||||
const incoming = req.headers['x-request-id'];
|
||||
return typeof incoming === 'string' && incoming.length > 0 ? incoming : randomUUID();
|
||||
},
|
||||
},
|
||||
}),
|
||||
],
|
||||
})
|
||||
export class ObservabilityModule {}
|
||||
@@ -0,0 +1,80 @@
|
||||
/**
|
||||
* OpenTelemetry SDK bootstrap for the BFF (per ADR-0012).
|
||||
*
|
||||
* IMPORTANT — module load order
|
||||
* ─────────────────────────────
|
||||
* This file must be `import`ed (or `require`d) BEFORE any other module
|
||||
* that creates HTTP clients, DB clients, web frameworks, etc. The
|
||||
* auto-instrumentations work by patching modules at require time; they
|
||||
* miss anything already loaded into Node's module cache. In `main.ts`
|
||||
* we therefore have:
|
||||
*
|
||||
* import './observability/tracing'; // first line, no other imports above it
|
||||
* // ... rest of the bootstrap ...
|
||||
*
|
||||
* Anything else above that line will be silently un-instrumented.
|
||||
*
|
||||
* What this enables, out of the box
|
||||
* ─────────────────────────────────
|
||||
* - HTTP, Express, NestJS request spans (incoming and outgoing)
|
||||
* - PostgreSQL spans via Prisma (the underlying `pg` driver)
|
||||
* - Redis (`ioredis`) spans — wired in advance for ADR-0010 / ADR-0014
|
||||
* - Pino log records auto-decorated with the active `trace_id`/`span_id`
|
||||
* so log lines correlate with traces in the backend
|
||||
*
|
||||
* Sampling is 100% at the app per ADR-0012; tail sampling is delegated
|
||||
* to the OTel Collector. Transport is OTLP HTTP/Protobuf (`port 4318`)
|
||||
* to match the .env.example default and avoid the gRPC binary
|
||||
* dependency (`@grpc/grpc-js`).
|
||||
*/
|
||||
|
||||
import { NodeSDK } from '@opentelemetry/sdk-node';
|
||||
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';
|
||||
import { resourceFromAttributes } from '@opentelemetry/resources';
|
||||
import { ATTR_SERVICE_NAME, ATTR_SERVICE_VERSION } from '@opentelemetry/semantic-conventions';
|
||||
import { HttpInstrumentation } from '@opentelemetry/instrumentation-http';
|
||||
import { ExpressInstrumentation } from '@opentelemetry/instrumentation-express';
|
||||
import { NestInstrumentation } from '@opentelemetry/instrumentation-nestjs-core';
|
||||
import { PgInstrumentation } from '@opentelemetry/instrumentation-pg';
|
||||
import { IORedisInstrumentation } from '@opentelemetry/instrumentation-ioredis';
|
||||
import { PinoInstrumentation } from '@opentelemetry/instrumentation-pino';
|
||||
|
||||
const serviceName = process.env['OTEL_SERVICE_NAME'] ?? 'portal-bff';
|
||||
const serviceVersion = process.env['OTEL_SERVICE_VERSION'] ?? 'dev';
|
||||
const otlpEndpoint =
|
||||
process.env['OTEL_EXPORTER_OTLP_ENDPOINT'] ?? 'http://localhost:4318/v1/traces';
|
||||
|
||||
const sdk = new NodeSDK({
|
||||
resource: resourceFromAttributes({
|
||||
[ATTR_SERVICE_NAME]: serviceName,
|
||||
[ATTR_SERVICE_VERSION]: serviceVersion,
|
||||
}),
|
||||
traceExporter: new OTLPTraceExporter({ url: otlpEndpoint }),
|
||||
instrumentations: [
|
||||
new HttpInstrumentation(),
|
||||
new ExpressInstrumentation(),
|
||||
new NestInstrumentation(),
|
||||
new PgInstrumentation(),
|
||||
new IORedisInstrumentation(),
|
||||
new PinoInstrumentation(),
|
||||
],
|
||||
});
|
||||
|
||||
sdk.start();
|
||||
|
||||
// Graceful shutdown on container-stop signals so in-flight spans get a
|
||||
// chance to flush to the collector. The 2s timeout is generous in dev
|
||||
// and fast enough not to block container teardown in prod.
|
||||
const shutdown = (signal: string): void => {
|
||||
sdk
|
||||
.shutdown()
|
||||
.catch((err) => {
|
||||
// Best-effort cleanup — log to stderr because Pino may itself
|
||||
// be in the middle of shutting down.
|
||||
process.stderr.write(`OpenTelemetry shutdown error on ${signal}: ${String(err)}\n`);
|
||||
})
|
||||
.finally(() => process.exit(0));
|
||||
};
|
||||
|
||||
process.once('SIGTERM', () => shutdown('SIGTERM'));
|
||||
process.once('SIGINT', () => shutdown('SIGINT'));
|
||||
Reference in New Issue
Block a user