From 289a5bad094df3d6eb81e9c54ee0302cbfc29881 Mon Sep 17 00:00:00 2001 From: Julien Gautier Date: Fri, 15 May 2026 23:05:10 +0200 Subject: [PATCH] docs(development): refresh after phase-3a + add topology / trace diagrams MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The development guide drifted as portal-admin shipped, the docs site landed, and Prisma stayed pinned at 6.x. Surgical pass. Section 1 (Repo layout): * Add `apps/portal-admin/` and `apps/portal-admin-e2e/` — both exist on `main` since #134. * Drop the phantom `prisma.config.ts` (Prisma 6.x doesn't ship one) and fix the "Prisma 7" tag to "Prisma 6.x" with the ADR-0006 pin reference. * Add `docs/index.md`, `docs/architecture.md`, `docs/.vitepress/` + new workflows (`docs-site.yml`, `renovate.yml`) so the layout matches what `ls` actually shows. Section 3 (Initial setup) — new diagram: * Mermaid `flowchart` of the local-dev topology (host-side dev servers + Compose containers + viewer profiles + the optional docs site). Replaces the prose ports-table that was scattered across §3 and §5 for new contributors. Section 4 (Daily commands): * Add `portal-admin` to serve / test / generate examples. * Correct the single-test-file recipe — Nx's vitest executor rejects `--testFile`; use positional `path/to/file.spec.ts` for Vitest projects, `--testPathPattern=…` for Jest. * New "Documentation site" subsection — `docs:dev` / `docs:build` / `docs:preview`, link to ADR-0022. Section 5 (Observability): * Mermaid `sequenceDiagram` of how a click becomes a trace (browser span → traceparent → BFF span → Pino log line with matching trace_id → OTLP batch → Jaeger). Anchors the correlation rule that the prose explained but didn't visualise. Section 6 (Renovate): * Add "Transitive vulnerabilities — `pnpm.overrides`" subsection. Documents the pattern from #159 (vite + esbuild via VitePress) so the next contributor hitting a transitive vuln has the playbook on hand. Section 9 (Sections to come): * Split into "Code shipped — doc to write" and "Not yet". Many phase-2 items moved out of "to come" since the code has shipped (auth, sessions, MFA decorator, admin module, audit viewer, user directory, downstream strategies, OpenAPI tooling, capabilities endpoint). The roadmap now reflects what's actually still missing. --- docs/development.md | 175 ++++++++++++++++++++++++++++++++++++-------- 1 file changed, 145 insertions(+), 30 deletions(-) diff --git a/docs/development.md b/docs/development.md index 45135f3..93454c4 100644 --- a/docs/development.md +++ b/docs/development.md @@ -12,6 +12,8 @@ For decision rationale, see the [ADRs](decisions/). For onboarding the local env apf_portal/ ├── .gitea/workflows/ # CI pipelines (ADR-0015) │ ├── ci.yml # per-PR + push to main: check / scan / commits / perf / a11y +│ ├── docs-site.yml # build docs site on PR + push (ADR-0022) +│ ├── renovate.yml # daily Renovate run (cron 03:00 UTC) │ └── security-scheduled.yml # weekly full-tree scan + prod Lighthouse ├── .github/ # Nx AI-tooling skills, prompts, agents (Nx-managed) ├── .husky/ # local git hooks (ADR-0007) @@ -19,34 +21,42 @@ apf_portal/ │ └── commit-msg # → pnpm exec commitlint ├── apps/ │ ├── portal-shell/ # Angular 21 SPA (zoneless, standalone, Signals, Vitest, SCSS) -│ │ ├── public/ # static assets -│ │ ├── src/ # entry, app config, routes, styles +│ │ ├── public/ # static assets (incl. /logos) +│ │ ├── src/ # entry, app config, routes, styles, locales │ │ ├── .postcssrc.json # Tailwind PostCSS plugin (required by @angular/build esbuild — postcss.config.js is ignored) │ │ └── project.json # Nx project config (build, serve, test, lint targets) │ ├── portal-shell-e2e/ # Playwright e2e for portal-shell +│ ├── portal-admin/ # Angular 21 SPA (admin surface, distinct origin + session per ADR-0020) +│ │ ├── public/ # static assets (incl. /logos) +│ │ ├── src/ # entry, app config, routes, audit + users + profile pages +│ │ └── project.json +│ ├── portal-admin-e2e/ # Playwright e2e for portal-admin │ ├── portal-bff/ # NestJS 11 BFF (Express adapter, ValidationPipe, Jest) │ │ ├── src/ # main, app module, controllers, services -│ │ ├── prisma/schema.prisma # Prisma 7 schema (postgresql) -│ │ ├── prisma.config.ts # Prisma 7 TS config (loads DATABASE_URL from .env) +│ │ ├── prisma/schema.prisma # Prisma 6.x schema (PostgreSQL) — pin documented in ADR-0006 │ │ ├── .env.example # env-vars catalog (committed); .env stays gitignored │ │ └── project.json │ └── portal-bff-e2e/ # Jest e2e for portal-bff ├── libs/ │ ├── feature// # vertical feature libs (e.g. feature-auth) -│ └── shared// # cross-cutting libs (tokens, ui, util) +│ └── shared// # cross-cutting libs (state, tokens, ui, util) ├── docs/ -│ ├── README.md # doc index +│ ├── README.md # doc index (git/IDE side — excluded from the docs site) +│ ├── index.md # VitePress home (Hero layout, see ADR-0022) +│ ├── architecture.md # cross-cutting Mermaid diagrams (C4, module boundaries, CI) +│ ├── development.md # this file │ ├── decisions/ # ADRs (MADR 4.0.0) │ ├── setup/ # local-environment onboarding (Zsh, pnpm, Nx workspace) -│ └── development.md # this file +│ └── .vitepress/ # VitePress config + build cache (gitignored) ├── notes/ # personal scratchpad (gitignored) ├── CLAUDE.md # project rules + architecture summary ├── commitlint.config.cjs # Conventional Commits config ├── eslint.config.mjs # workspace ESLint with module boundaries ├── lighthouserc.js # Lighthouse CI thresholds (ADR-0017) ├── nx.json # Nx workspace config -├── package.json # workspace deps + scripts +├── package.json # workspace deps + scripts (incl. pnpm.overrides for transitive remediation) ├── pnpm-workspace.yaml # apps/* + libs/** +├── renovate.json # Renovate config (groupings, dashboard, vuln alerts) ├── tsconfig.base.json # shared TS strict config └── vitest.workspace.ts # Vitest workspace projects ``` @@ -123,6 +133,46 @@ cp apps/portal-bff/.env.example apps/portal-bff/.env `./infra/local/dev.sh help` lists the rest of the verbs (`down`, `status`, `logs`, `stop`, `restart`, `exec`). Full reference — service inventory, port table, persistence, bootstrap re-run procedure — lives in [`infra/README.md`](../infra/README.md) → "Local-dev stack". +### Topology at a glance + +How the pieces fit together once the stack is up — three Node dev servers on the host, the infrastructure containers behind Docker Compose, optional viewer profiles for inspection: + +```mermaid +flowchart LR + subgraph host["Dev host (WSL2 / Linux / macOS)"] + direction TB + shell["portal-shell
:4200"] + admin["portal-admin
:4300"] + bff["portal-bff
:3000"] + docs["docs site
:5173
(VitePress)"] + end + + subgraph compose["infra/local/ (Docker Compose)"] + direction TB + pg[("Postgres
:5432")] + redis[("Redis
:6379")] + otel["OTel Collector
:4318"] + subgraph obs["observability profile"] + jaeger["Jaeger UI
:16686"] + end + subgraph dbt["dbtools profile"] + pgweb["pgweb
:8081"] + end + end + + shell -. "/api/*" .-> bff + admin -. "/api/admin/*" .-> bff + bff --> pg + bff --> redis + bff -- "OTLP" --> otel + shell -- "OTLP/HTTP
spans" --> otel + admin -- "OTLP/HTTP
spans" --> otel + otel --> jaeger + pgweb --- pg +``` + +The docs site lives off to the side — it has no runtime dependency on the BFF or the infra stack and only reads files under `docs/`. Same goes for the SPAs from `portal-shell` to `portal-admin`: each runs its own dev server with its own port, talks to the same BFF, but holds a **distinct session cookie** (per [ADR-0020](decisions/0020-portal-admin-app.md) §"Sessions — distinct from `portal-shell`"). + --- ## 4. Daily commands @@ -131,26 +181,27 @@ cp apps/portal-bff/.env.example apps/portal-bff/.env ```bash pnpm nx serve portal-shell # http://localhost:4200 (Angular dev server) +pnpm nx serve portal-admin # http://localhost:4300 (Angular dev server) pnpm nx serve portal-bff # http://localhost:3000/api (NestJS) ``` -Both can run in parallel in two terminals; the SPA proxies API calls to the BFF in dev. +All three can run in parallel in separate terminals; the SPAs proxy API calls to the BFF in dev. Each SPA holds its own cookie / session — signing in to one does not authenticate the other (per [ADR-0020](decisions/0020-portal-admin-app.md)). ### Test ```bash pnpm nx test portal-shell # Vitest (single run; --configuration=watch for watch mode) +pnpm nx test portal-admin # Vitest pnpm nx test portal-bff # Jest pnpm nx run-many -t test # all projects pnpm nx affected -t test # only projects affected since main ``` -Run a single Vitest file: +Run a single test file: -```bash -pnpm nx test portal-shell --testFile=src/app/app.spec.ts -``` +- **Vitest projects** (`portal-shell`, `portal-admin`, `feature-auth`, `shared-*`): use a positional path filter — `pnpm nx test portal-shell --skip-nx-cache src/app/app.spec.ts`. `--testFile` is not accepted by the Nx vitest executor (`'testFile' is not found in schema`). +- **Jest projects** (`portal-bff`): `pnpm nx test portal-bff --testPathPattern=src/auth/auth.service`. ### Lint, type-check, format @@ -178,8 +229,9 @@ pnpm nx affected -t build ### Generate ```bash -# Component in portal-shell +# Component (specify the SPA — `portal-shell` or `portal-admin`) pnpm nx g @nx/angular:component --project=portal-shell --standalone +pnpm nx g @nx/angular:component --project=portal-admin --standalone # Service / controller / module in portal-bff pnpm nx g @nx/nest:service --project=portal-bff @@ -190,7 +242,7 @@ pnpm nx g @nx/js:library --name=shared- --directory=libs/shared/ \ --bundler=tsc --unitTestRunner=vitest \ --tags="scope:shared,type:shared" --no-interactive -# New Angular feature lib (front-only) +# New Angular feature lib (front-only — usable by either SPA) pnpm nx g @nx/angular:library --name=feature- --directory=libs/feature/ \ --standalone=true --unitTestRunner=vitest-analog \ --tags="scope:portal-shell,type:feature" --no-interactive @@ -198,6 +250,18 @@ pnpm nx g @nx/angular:library --name=feature- --directory=libs/feature/ Sweep generated files for `process.env.X` (dot notation) → `process.env['X']` (bracket notation), required by the strict-TS option `noPropertyAccessFromIndexSignature: true`. The Nx generators don't emit bracket form. +### Documentation site + +`docs/**/*.md` renders as a VitePress site per [ADR-0022](decisions/0022-docs-site-vitepress.md). No Nx project for it — pure pnpm scripts: + +```bash +pnpm docs:dev # http://localhost:5173 (HMR — edit a .md, see it live) +pnpm docs:build # static site → docs/.vitepress/dist/ +pnpm docs:preview # serve the built site locally to smoke-test +``` + +Adding a new ADR is a single-file change: drop `docs/decisions/NNNN-kebab-title.md` (MADR 4.0.0 frontmatter, see [`template.md`](decisions/template.md)) and the sidebar auto-lists it. Update [`decisions/README.md`](decisions/README.md) in the same PR so the curated tag-grouped index stays in sync. + ### Prisma ```bash @@ -233,6 +297,32 @@ pnpm ci:perf # production build + Lighthouse CI against the static-served The BFF and the SPA are both wired with OpenTelemetry tracing and structured Pino logging (per [ADR-0012](decisions/0012-observability-pino-opentelemetry.md)). This section is the practical guide to using them while debugging — finding a trace, reading the logs that go with it, untangling a slow request. +### How a click becomes a trace + +```mermaid +sequenceDiagram + actor U as User + participant SPA as portal-shell + participant BFF as portal-bff + participant Pino as BFF stdout (Pino) + participant OC as OTel Collector + participant J as Jaeger UI + + U->>SPA: click + Note over SPA: span `user_interaction`
(root, generates trace_id) + SPA->>BFF: fetch /api/foo
traceparent: 00---01 + Note over SPA: span `fetch` (child) + Note over BFF: span `HTTP GET /api/foo`
(continues same trace) + BFF->>Pino: log line
`{ ...payload, trace_id, span_id, request_id }` + BFF-->>SPA: 200 OK + SPA-)OC: OTLP/HTTP batch (every ~5 s) + BFF-)OC: OTLP batch + OC-)J: forward spans + Note over J: trace_id matches the one in Pino
full nested timeline visible +``` + +The single point of correlation between the two pillars is **`trace_id`**: Jaeger's URL bar carries it (`/trace/`), and `@opentelemetry/instrumentation-pino` injects the same value on every BFF log line emitted inside that request's scope. Grep one, navigate the other. + ### Bring up the observability stack Traces land in the Jaeger UI bundled in `infra/local/dev.compose.yml` under the `observability` profile. The OpenTelemetry Collector runs in the core profile; without Jaeger active, traces still go to the Collector but nothing visualises them — they appear as warnings in `dev.sh logs otel-collector` and are dropped after the buffer fills. @@ -346,6 +436,22 @@ Repo → Actions → "Renovate" workflow → Run workflow. Useful when you've ju - For a major bump that introduces breaking changes, **don't reflexively merge**: read the changelog, then either accept the work or close the PR with a "rejected" label. Renovate respects that label and won't keep re-opening the same major. - **Adding or removing** a dependency belongs in a feature PR, not in Renovate's scope. Renovate only updates _versions_ of existing deps. +### Transitive vulnerabilities — `pnpm.overrides` + +Renovate's vulnerability remediation handles **direct** `package.json` deps. For **transitive** vulnerabilities (a vulnerable package pulled in by a healthy direct dep), Renovate stays silent in the dashboard — the v40 image doesn't write `pnpm.overrides` automatically. The manual pattern lives in [`package.json`](../package.json) → `pnpm.overrides`, version-selector form: + +```jsonc +"esbuild@<0.25.0": ">=0.25.0", +"vite@<6.4.2": ">=6.4.2", +``` + +Two properties of this form worth knowing: + +1. **The override is gated on the vulnerable range** — it only kicks in when the resolved version is < the threshold, becoming a no-op the day the underlying dep's own bump ships a patched version. +2. **pnpm overrides bypass peer constraints** — if the parent dep declares `vite ^5.0.0` but no patched 5.x exists, pnpm will pick a patched 6.x/7.x. Verify the chantier still builds before merging the override. + +The first instance of this pattern is preserved in [#159](https://git.unespace.com/julien/apf_portal/pulls/159) (vite + esbuild via VitePress 1.6.4). Next time `pnpm audit` fails with a transitive vuln and Renovate is silent, this is the playbook. + --- ## 7. Conventional commit cycle @@ -441,21 +547,30 @@ The template guides without enforcing — sections can be left blank when irrele ## 9. Sections to come — roadmap by phase -This doc starts as a phase-1 + cross-cutting reference. As features for later phases land, the corresponding sections below are filled in directly. Each entry is mapped to the ADR / implementation work that unlocks it, so a contributor can see when each section becomes real and what triggers it. +As features for later phases land, the corresponding sections below are filled in directly. Entries are split between **code shipped — doc to write** (the work is on `main`, only the prose is missing) and **not yet** (neither code nor doc). When a section grows beyond a short subsection, it is extracted to its own file under `docs/development/`. Per the documentation convention (see [README.md](README.md)), we group into a folder once we have at least three related files; this doc is then re-organised into an index pointing at the extracted files. Until then, all sections live here. -| Future section | Phase | Triggered by | -| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Auth dev-loop** — Microsoft 365 Developer tenant configuration, MSAL Node connection, OIDC code-flow walkthrough, switching between dev and prod-like tenants. | 2 | Auth flow code lands ([ADR-0009](decisions/0009-auth-flow-oidc-pkce-msal-node.md)) once the dev tenant is provisioned by IT. | -| **Session inspection** — reading the Redis session store in dev, decrypting the AES-GCM `tokens` blob with the dev key, force-logout patterns. | 2 | Sessions module lands ([ADR-0010](decisions/0010-session-management-redis.md)). | -| **MFA step-up debugging** — triggering claims-challenge flows, verifying `mfaVerifiedAt` freshness, testing the SPA HTTP interceptor that handles 401 + claims challenge. | 2 | First `@RequireMfa()` route lands ([ADR-0011](decisions/0011-mfa-enforcement-entra-conditional-access.md)). | -| **Audit-log inspection workflow** — querying `audit.events` as `audit_reader`, joining with app logs by `trace_id`, validating the append-only role grants in dev. | 2 | Audit module lands ([ADR-0013](decisions/0013-audit-trail-separated-postgres-append-only.md)). | -| **Downstream API integration recipe** — adding a new `DownstreamApiConfig`, choosing the auth strategy (OBO vs service+assertion), wiring resilience policies, testing with a mocked downstream. | 2 | First downstream client lands ([ADR-0014](decisions/0014-downstream-api-access-obo-pattern.md)). | -| **Component patterns library** — the in-house, spartan-style components (Angular CDK + Tailwind) as they ship, with a11y notes per component (keyboard model, ARIA, screen-reader expectations). | 5b suite | First non-placeholder component in `libs/shared/ui/`. | -| **a11y testing workflow** — running axe-core via Playwright locally, screen-reader testing notes (NVDA / VoiceOver / TalkBack), the APF user-panel cadence and how to triage findings. | 3a | First Playwright e2e suite touching real screens ([ADR-0016](decisions/0016-accessibility-baseline-wcag-aa-targeted-aaa.md)). | -| **Performance debugging** — running Lighthouse CI locally with full config, reading the HTML reports, using `source-map-explorer` to investigate bundle bloat, interpreting BFF p95/p99 from OTel. | 3a | Lighthouse already wired in CI ([ADR-0017](decisions/0017-performance-budgets-lighthouse-ci.md)); section grows when first real route is added to the critical-routes list. | -| **Debugging tips** — Angular DevTools, NestJS inspector, Prisma query log, OTel trace navigation, common gotchas. | cross | Accumulates organically as the team encounters them. | -| **Release workflow** — tag-driven release, what `release.yml` does, version bumping, changelog generation from Conventional Commits. | 3b | On-prem infrastructure ADR + populated `release.yml`. | -| **GitLab migration runbook** — when the org migrates Gitea → GitLab, how the workflows are ported, which level-2 sections of [ADR-0015](decisions/0015-cicd-gitea-actions.md) get superseded. | future | GitLab migration ADR (6–18 months horizon). | -| **Architecture overview diagrams** — high-level component diagrams, data-flow diagrams, trust boundaries (for security review). | cross | First major architecture review or onboarding cohort ≥ 3 contributors. | +### Code shipped — doc to write + +| Future section | Code shipped in | What the section will cover | +| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| **Auth dev-loop** | [ADR-0009](decisions/0009-auth-flow-oidc-pkce-msal-node.md) implementation series (PRs #100s) | Microsoft 365 Developer tenant configuration, MSAL Node connection, OIDC code-flow walkthrough, switching between dev and prod-like tenants. | +| **Session inspection** | [ADR-0010](decisions/0010-session-management-redis.md) implementation | Reading the Redis session store in dev (`redis-cli`), decrypting the AES-GCM `tokens` blob with the dev `SESSION_ENCRYPTION_KEY`, force-logout patterns. | +| **MFA step-up debugging** | `@RequireMfa()` decorator + freshness guard (#128) | Triggering claims-challenge flows when the first consumer route lands, verifying `amrAt` freshness, testing the SPA interceptor on 401 + claims challenge. _(Decorator exists; no consumer route yet — section lands together with that.)_ | +| **Admin surface workflows** | [ADR-0020](decisions/0020-portal-admin-app.md) skeleton (#127) + audit viewer (#136) + user directory (#140–#142) | Walking through `/audit` and `/users` viewers, role assignment in Entra (`Portal.Admin`), cross-app navigation. | +| **Audit-log inspection workflow** | [ADR-0013](decisions/0013-audit-trail-separated-postgres-append-only.md) + admin viewer | Querying `audit.events` as `audit_reader`, joining with Pino logs by `trace_id` / `actor_id_hash`, validating the append-only role grants in dev. | +| **Downstream API integration recipe** | [ADR-0014](decisions/0014-downstream-api-access-obo-pattern.md) strategies (OBO #137, signed-assertion + JWKS #138/#139) | Adding a new `DownstreamApiConfig`, choosing the auth strategy, wiring resilience policies, testing with a mocked downstream. _(Strategy layer ready; no v1 consumer yet — section grows with the first.)_ | +| **OpenAPI / Scalar dev workflow** | [#143](https://git.unespace.com/julien/apf_portal/pulls/143) | Browsing `/api/docs`, annotating new controllers with `@ApiTags` / `@ApiOperation`, exporting the spec for Bruno / Postman, the CSRF caveat when hitting mutating routes from Scalar. | +| **Capabilities-driven SPA UX** | `/api/me/capabilities` + cross-app menu links (#151) | When to add a new key to `CapabilitiesView`, how the SPA's `CapabilitiesService` consumes it, the "binary boolean, never the raw role" rule. | + +### Not yet + +| Future section | Triggered by | +| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | +| **Component patterns library** — in-house spartan-style components (Angular CDK + Tailwind), a11y notes per component (keyboard model, ARIA, screen-reader expectations). | First non-placeholder component in `libs/shared/ui/` beyond `Icon` and `UserMenu`. | +| **a11y testing workflow** — running axe-core via Playwright locally, screen-reader testing notes (NVDA / VoiceOver / TalkBack), the APF user-panel cadence and how to triage findings. | First Playwright e2e suite touching real screens ([ADR-0016](decisions/0016-accessibility-baseline-wcag-aa-targeted-aaa.md)). | +| **Performance debugging** — running Lighthouse CI locally with full config, reading the HTML reports, using `source-map-explorer` to investigate bundle bloat, interpreting BFF p95/p99 from OTel. | First real route added to the critical-routes list ([ADR-0017](decisions/0017-performance-budgets-lighthouse-ci.md)). | +| **Debugging tips** — Angular DevTools, NestJS inspector, Prisma query log, OTel trace navigation, common gotchas. | Accumulates organically as the team encounters them. | +| **Release workflow** — tag-driven release, what `release.yml` does, version bumping, changelog generation from Conventional Commits. | On-prem infrastructure ADR + populated `release.yml`. | +| **GitLab migration runbook** — when the org migrates Gitea → GitLab, how the workflows are ported, which level-2 sections of [ADR-0015](decisions/0015-cicd-gitea-actions.md) get superseded. | GitLab migration ADR (6–18 months horizon). |