TESTING_STRATEGY — reporting-service

Sibling: APPLICATION_LOGIC · DATA_MODEL · SECURITY_MODEL · platform anchor: docs/standards/SERVICE_TEMPLATE

We follow the platform's trophy-shaped test mix and the three mandatory integration tests required by the service template.

1. Layers

Layer	Tooling	Coverage target	Notes
Unit (domain)	Vitest	≥ 90 % branches on `domain/`	Pure TS, no I/O
Unit (use case)	Vitest + in-memory ports	≥ 85 % lines on `application/`	Use the in-memory adapters from `test/support/inmemory/*`
Contract (events)	`@melmastoon/event-contracts` validator	100 % of published & consumed events	Runs against generated JSON Schemas
Contract (REST)	`pact` consumer side + provider verification	100 % of public endpoints	BFFs publish pacts; we verify
Integration (DB)	Testcontainers Postgres + Drizzle migrations	All repository methods	Real RLS, real transactions
Integration (Pub/Sub)	Pub/Sub emulator	Outbox publish, inbox dedupe	Includes ordering-key invariants
Integration (GCS)	`fake-gcs-server`	Artifact upload, signed URL issue
Integration (renderer)	Puppeteer + headless Chromium pinned via `ms-playwright`	PDF determinism (golden files)	Snapshot diff with tolerance
E2E happy path	Playwright + ephemeral env	Request → completed → delivered	Triggered nightly + on PR labelled `e2e`
Load	k6	50 RPS sustained, 200 RPS burst	Pre-prod weekly
Chaos	Gremlin / custom	Pub/Sub drop, GCS slow, BQ timeout	Quarterly game-day

CI fails if the targets above slip; vitest --coverage reports are uploaded as artifacts.

2. Mandatory integration tests

These three live in test/integration/ and must pass on every PR.

2.1 `tenant-isolation.spec.ts`

describe('tenant isolation', () => {
  it('list runs returns 0 rows for tenant B when authed as tenant A', async () => { … });
  it('GET /api/v1/reports/runs/{id from B} as A returns 404 not 403', async () => { … });
  it('regulatory submission cross-tenant attempt returns 404', async () => { … });
  it('GCS object key for tenant B cannot be signed by tenant A handler', async () => { … });
});

Two fixture tenants are seeded via Drizzle migrations + seed_test_tenants.sql. Each assertion is run against an end-to-end stack (Postgres testcontainer + emulators).

2.2 `outbox.spec.ts`

describe('transactional outbox', () => {
  it('persists event row in same tx as state change', async () => { … });
  it('does not publish if tx rolls back', async () => { … });
  it('publishes envelope with stable ordering_key per (tenantId, runId)', async () => { … });
  it('records publishedAt and stops re-publishing on success', async () => { … });
  it('retries with exponential backoff on transient publish error', async () => { … });
  it('emits exactly-once envelope id (envelope_id stable across retries)', async () => { … });
});

2.3 `inbox.spec.ts`

describe('inbox dedupe', () => {
  it('processes a Pub/Sub message exactly once even if delivered twice', async () => { … });
  it('rejects messages with invalid OIDC audience (401)', async () => { … });
  it('reverts handler effects on processing exception, leaving inbox row absent so retry will reprocess', async () => { … });
});

3. Determinism for renderer

PDF rendering must be byte-stable for the same input to make snapshot tests practical:

Fonts pinned in container image (/usr/share/fonts/melmastoon/*).
Chromium version pinned via container digest, not tag.
Puppeteer launched with --font-render-hinting=none, fixed device-scale, fixed viewport.
Generated SHA-256 over PDF bytes is asserted equal to the golden hash.
For locale variants we have one golden per locale-format pair (PDF + XLSX + CSV) under test/fixtures/golden/.

When intentional template/style changes happen, contributors run pnpm test:renderer -u to update goldens; PR template asks reviewers to inspect the diff PNGs in test/fixtures/golden/diff-out/.

4. Property-based and fuzz tests

Property tests (fast-check) for filter resolution: any combination of propertyId, dateRange, channel, status must round-trip through (spec → SQL → result count) without producing rows from a different tenant.
Fuzz tests on the API layer: random JSON bodies must produce 4xx with a valid Problem-Details document, never 5xx.

5. Saga participation tests

Although reporting-service does not orchestrate sagas, it produces and consumes events that take part in higher-level flows. Contract tests assert:

Publishing report.completed.v1 triggers exactly one notification.notification.requested.v1 per active subscription (asserted via Pub/Sub emulator and notification-service stub).
Receiving tenant.deleted.v1 causes purge within the SLA (15 min in tests; 24 h in prod).

6. Performance & load

k6 scenarios in test/load/:
- ad-hoc-runs.js — 50 RPS POST /runs, mixed templates.
- schedule-fan-out.js — 1000 schedules firing in a 60 s window per region.
- regulatory-flush.js — 200 pending submissions, simulate adapter latencies.
Targets:
- p95 end-to-end ad-hoc operational ≤ 8 s under 50 RPS.
- Worker pool autoscales between 2 and 20 pods.
- No GC pauses > 200 ms p99.
Load runs publish artifacts: summary.json, latency.csv, flamegraph.svg.

7. Chaos

Pub/Sub publish drop — outbox should retry; eventually publish; no duplicate state mutations.
GCS 500 burst — render retries with backoff; if exceeds budget, transitions run to failed with retriable=true and re-queues.
Renderer OOM — pod restarts; the in-flight run is retried via Pub/Sub redelivery; no double-delivery to subscribers.
Cloud Scheduler firing 2× in a minute — schedule fan-out is idempotent via (scheduleId, fireAt) key; second fire is ignored.

8. Linting & static checks

Check	Tool
TypeScript compile	`tsc --noEmit` (strict, exactOptionalPropertyTypes)
Lint	`eslint --max-warnings=0` (platform config)
Formatting	`prettier --check`
Naming	`bin/check-naming.ts` (asserts table names, event subjects, ULID prefixes)
Migration safety	`bin/check-migrations.ts` (asserts every new table has tenant_id + RLS)
OpenAPI	spectral lint against `bin/openapi.yaml`
Event schemas	`ajv compile` + `bin/check-event-schemas.ts`
Secrets	`gitleaks detect --no-git`
Deps	`pnpm audit --audit-level high` blocks merge

CI gate (.github/workflows/ci.yml) runs all of the above plus the unit + mandatory integration tests on every PR. Renderer/E2E suites run on label or nightly.

9. Test data & fixtures

test/fixtures/templates/ — sample templates per category, including AF police registration, KSA VAT, generic occupancy.
test/fixtures/analytics-rows/ — canned BigQuery result sets keyed by template + filter.
test/fixtures/jurisdictions/ — adapter stubs returning known receipts.
test/support/clock.ts — frozen time helper for deterministic ULIDs and snapshots.

Fixtures use only synthetic data; no real guest names, no real document numbers.

Cross-references: APPLICATION_LOGIC §6, SECURITY_MODEL §3, docs/standards/SERVICE_TEMPLATE.

1. Layers​

2. Mandatory integration tests​

2.1 tenant-isolation.spec.ts​

2.2 outbox.spec.ts​

2.3 inbox.spec.ts​

3. Determinism for renderer​

4. Property-based and fuzz tests​

5. Saga participation tests​

6. Performance & load​

7. Chaos​

8. Linting & static checks​

9. Test data & fixtures​