TESTING_STRATEGY — bff-backoffice-service
Sibling: APPLICATION_LOGIC · API_CONTRACTS · SECURITY_MODEL · FAILURE_MODES
1. Test pyramid
┌──────────────┐
│ E2E (4%) │ ← Playwright + real Electron build against stage
├──────────────┤
│ Contract (10%) │ ← Pact: BFF↔upstream + BFF↔desktop
├──────────────┤
│ Integration │ ← NestJS Test app + Postgres + Redis + Pub/Sub emulator + WireMock
│ (30%) │
├──────────────┤
│ Unit (56%) │ ← orchestrators, DPoP verifier, MFA store, idempotency, single-flight, audit envelope
└──────────────┘
Coverage gates (Vitest c8):
- Statements ≥ 90%
- Branches ≥ 85%
- Functions ≥ 90%
- Lines ≥ 90%
2. Tooling
| Layer | Tool |
|---|---|
| Unit | Vitest + @nestjs/testing |
| Integration | Vitest + Testcontainers (Postgres 16 + Redis 7 + Pub/Sub emulator) + WireMock |
| Contract | Pact JS |
| E2E | Playwright + Electron-renderer harness against stage |
| Load | k6 |
| Security | OWASP ZAP, pnpm audit, gitleaks, Trivy, Cosign |
| Mutation | Stryker (nightly on critical files) |
3. Unit tests
3.1 Critical files (100% line + branch)
| File | Notes |
|---|---|
src/security/dpop-verifier.ts | RFC 9449 conformance; replay; jkt match; htu/htm; iat skew |
src/security/mfa-attestation-store.ts | Single-use; TTL; scope match; replay |
src/application/use-cases/proxy-lock-action.use-case.ts | MFA gate; audit envelope before vendor call |
src/application/use-cases/proxy-domain-mutation.use-case.ts | Idem key; audit; activity append on both branches |
src/application/use-cases/decide-ai-suggestion.use-case.ts | Write-once; activity; outbox |
src/application/use-cases/negotiate-sync-handshake.use-case.ts | Token mint; cursor cache update |
src/application/use-cases/compose-dashboard.use-case.ts | Per-widget deadlines; partial composition; single-flight |
src/application/use-cases/refresh-session.use-case.ts | DPoP path through to iam-service |
src/security/audit-envelope-builder.ts | All required fields; no PII |
src/cache/single-flight.ts | Race + leader/follower correctness |
src/sse/sse-bus.ts | Multiplex; keep-alive; dropped-conn cleanup |
src/idempotency/idempotency-store.ts | Same key + same body returns same response; different body → 412 |
3.2 Property-based (fast-check)
- DPoP verifier: any tampered field rejected.
- MFA attestation: replay always blocked.
- Idempotency: N concurrent same-key calls → identical response.
- Tenant guard: random cross-tenant requests → CROSS_TENANT_REFERENCE.
- Single-flight: leader wins; followers receive same payload; no duplicates.
4. Integration tests
Run against ephemeral Postgres + Redis + Pub/Sub emulator + WireMock upstreams.
4.1 Mandatory (gating CI)
| Test | Asserts |
|---|---|
tenant-isolation.spec.ts | Cross-tenant blocked at controller, application, RLS |
outbox.spec.ts | Domain events written transactionally |
inbox.spec.ts | Cache-invalidation events deduplicated by message ID |
4.2 Functional
| Scenario | Asserts |
|---|---|
| Refresh with valid DPoP | 200 + new tokens |
| Refresh with invalid DPoP | 401 + DPOP_INVALID |
| Refresh with expired refresh token | 401 + SESSION_EXPIRED |
DPoP jti replay within 5 min | 401 + DPOP_INVALID |
| Dashboard composition cold | All widgets fresh; partial=false |
| Dashboard composition with one widget upstream timing out | partial=true; staleness set |
| Dashboard composition with circuit open on inventory | unavailable |
| Single-flight under N concurrent dashboard requests | One upstream fanout |
| Cache TTL expiry triggers re-composition | New composedAt |
| `GET /today | /arrivals |
GET /housekeeping/board | Verified against fixture |
GET /maintenance/board | Verified against fixture |
| AI suggestion list (cached + bypass) | Both paths |
| AI decision (idempotent) | Same idem key returns same row; different body → 412 |
| AI decision (already decided) | 409 + DECISION_ALREADY_RECORDED |
| Alert ack (idempotent + already-acked) | 409 + ALERT_ALREADY_ACKED |
| Preferences write with optimistic concurrency | 200 + version+1; 412 on stale If-Match |
Heartbeat updates device_sync_status | last_heartbeat_at advances; outbox-hints stored |
| Heartbeat rate-limit (>1/min) | 429 + Retry-After |
| Sync handshake success path | sst returned; cursor cached |
Sync handshake with appVersion below floor | 409 + SYNC.VERSION_BLOCKED |
POST /sync/cursor advances cursor | DB row updated; outbox event |
| Lock issue without MFA when policy doesn't require | success |
| Lock revoke without MFA token | 401 + MFA_REQUIRED |
| Lock revoke with valid MFA token | success; mfa consumed; row in lock_audit |
| Lock revoke replaying same MFA token | 401 + MFA_INVALID_OR_USED |
| Lock proxy when vendor returns 502 | 502 + LOCK.VENDOR_UNAVAILABLE; audit row outcome=failure |
| Mutation proxy: check-in success | reservation-service called; activity append; idem store written |
| Mutation proxy: idem replay returns same body | exact match |
| Mutation proxy: different body under same idem key | 412 + PRECONDITION_FAILED |
SSE: subscribe + receive ai.new | Event delivered within 200 ms |
| SSE: keep-alive comment every 25 s | Verified |
| SSE: per-device single connection | New conn closes prior |
Force-logout broadcast on iam.session.revoked.v1 | SSE event delivered to device; session blob dropped |
| Postgres down during mutation | 503 + CACHE_UNAVAILABLE; idem store unwritable; alert |
| Memorystore session-tier down | Mutating endpoints 503; reads bypass cache |
| Schema drift (WireMock returns malformed) | 502 + SCHEMA_DRIFT; alert |
| Cross-tenant attempt at every layer | Each blocked |
4.3 Idempotency tests
POST /reservations/{id}/check-inwith same idem key → identical response across N concurrent calls.POST /folios/{id}/chargeswith same key → one upstream call.POST /locks/{id}/issue-keywith same key → one vendor call.POST /ai/suggestions/{id}/decidewith same key → one decision row.
4.4 PII tests
telemetry-pii.spec.tsasserts no raw PII in any outbox row.audit-log-pii.spec.tsasserts the same for Cloud Logging payloads.activity-ledger-pii.spec.tsasserts activity rows reference resources by id, not PII.
5. Contract tests (Pact)
5.1 As consumer
This BFF is a Pact consumer of:
iam-service(refresh, validate, attest step-up)tenant-service(operator resolution, preferences mirror)reservation-service(arrivals/departures/in-house, mutation proxies)inventory-service(occupancy KPI)pricing-service(rate snapshot)housekeeping-service(board + summary + transitions)maintenance-service(board + summary + transitions)billing-service(folio + revenue KPI + charge proxy)lock-integration-service(issue + revoke)ai-orchestrator-service(fetch suggestions + report decision)notification-service(list alerts + acknowledge)sync-service(handshake)analytics-service(KPI snapshots)property-service(property meta)
5.2 As provider
Consumer pact from @ghasi/app-desktop-backoffice for every endpoint in API_CONTRACTS. Verified per PR.
6. E2E tests (stage)
Run nightly + on every release candidate. Real Electron build container with synthetic device key.
| Flow | Steps |
|---|---|
| Sign-in + refresh | DPoP-signed; tokens received; cnf.jkt verified |
| Dashboard happy path | All widgets fresh |
| Dashboard partial under chaos | One upstream timeout; partial=true |
| Workbench happy path | Today, arrivals, departures, in-house; all fixtures verified |
| Reservations check-in | Mutation proxy success; activity ledger row; outbox event |
| Folio charge | Mutation proxy; idempotency; activity row |
| Housekeeping transition | Same |
| Maintenance transition | Same |
| Lock issue | Vendor stub success; audit row; outbox event |
| Lock revoke | MFA step-up first; revoke success; audit; outbox |
| Lock revoke without MFA | 401; banner shown in renderer |
| AI suggestion decide | Decision recorded; orchestrator notified; outbox |
| Alert ack | Acked; outbox; not re-ackable |
| Sync handshake → direct pull → cursor advance | sst issued; pull success; cursor mirror updated |
| SSE channel happy path | Subscribe; receive ai.new + alerts.new + dashboard-refresh-hints + session.forceLogout |
| SSE→polling fallback | SSE blocked; renderer falls back to polling automatically |
| Force-logout | Triggered via iam.session.revoked.v1; SSE event delivered; renderer routes to sign-in |
| App-version-floor blocking | Old version → SYNC.VERSION_BLOCKED; upgrade prompt |
| Heartbeat | Posts every 60 s; status flips to offline after missed 5 min |
| Multi-device hint | Second device sees other device's pending-action count |
7. Load tests (k6)
7.1 Profiles
| Profile | RPS | Duration | Mix |
|---|---|---|---|
| Steady | 50 RPS | 15 min | 30% dashboard, 25% workbench, 15% mutations, 10% AI/alerts, 10% heartbeat, 10% SSE traffic |
| Peak | 0 → 300 RPS in 2 min, hold 10 min | 12 min | same |
| Burst | 0 → 200 RPS mutation-heavy | 5 min | 70% mutations |
| Long soak | 30 RPS | 8 h | mixed |
7.2 Targets
- Steady: p95 < 600 ms; error < 0.1%; instances ≤ 8.
- Peak: p95 < 1 s; cache hit > 80% on dashboard.
- Burst: mutation p95 < 1.5 s; idempotency dedup correctness 100%.
- Long soak: no memory growth > 10%; no connection leaks.
8. Chaos tests
Quarterly in stage:
| Experiment | Expected behaviour |
|---|---|
reservation-service 30 s outage | Workbench reads fail; dashboard partial; mutations 503 |
lock-integration-service outage | Lock proxy returns 502 + LOCK.VENDOR_UNAVAILABLE; audit row outcome=failure; alert raised |
ai-orchestrator-service outage | AI surfaces hidden; no banner |
iam-service outage | Refresh fails; sessions hold for token TTL; offline mode unaffected |
| Memorystore session-tier failover | < 30 s elevated latency; sessions preserved |
| Postgres failover | Mutating endpoints 503 for ≤ 60 s |
| Pub/Sub publish 100% failure | Outbox grows; recovers |
| SSE bus delay (artificial) | Force-logout takes longer; alert |
| DPoP key rotation drill (per device) | Single-device re-enroll path |
| HMAC sync-session signing-key rotation | Both keys honored during overlap |
9. Security tests
- DAST nightly: ZAP against stage.
- Per-PR static: ESLint +
eslint-plugin-security+@typescript-eslint/strict+pnpm audit. - Secrets scan:
gitleaks. - Image scan: Trivy.
- DPoP fuzzing: fast-check + tampered tokens; expected 100% reject.
- MFA fuzzing: replay + expired + wrong scope; expected 100% reject.
- CORS: only allow-listed origins succeed.
- Tenant isolation: cross-tenant blocked at every layer.
10. Test data
- 5 stage tenants with 3 properties each.
- 8 stage operators (front desk, supervisor, housekeeping, maintenance, GM, finance).
- 4 stage devices per tenant (win32, darwin, linux variants).
- Lock vendor stubs for ttlock, salto, assa-vostio, wiegand.
- Pre-seeded reservations, folios, alerts, AI suggestions.
11. CI pipeline
Per PR:
pnpm install --frozen-lockfilepnpm lint && pnpm typecheckpnpm test:unitpnpm test:integration- Pact consumer publish + verify upstream pacts
pnpm audit && gitleaks detect- Build + Trivy + Cosign sign
- Deploy to dev Cloud Run; smoke test
- (
e2elabel) — Playwright subset against dev
Nightly:
- Stryker mutation tests on critical files (≥ 80%)
- ZAP DAST against stage
- Long soak load test
- Chaos experiments (rotating)
12. Release readiness checklist (linked from SERVICE_READINESS)
- All P1 alerts have ack'd runbooks
- p95/p99 SLOs met in load test
- DPoP replay drill executed in stage in last 90 days
- MFA bypass attempts blocked in pen test
- Mutation score ≥ 80% on critical files
- Pact verifications green for all upstreams + desktop consumer pact
- DAST report has no high/critical findings
- Tenant isolation tests pass
- Lock-audit completeness tests pass
- Force-logout E2E latency under 5 s