file-storage-service — TESTING_STRATEGY

Companion: DOMAIN_MODEL §13 · SECURITY_MODEL §11 · SERVICE_TEMPLATE — Mandatory integration tests · DEFINITION_OF_DONE

The pyramid is enforced by CI: unit (≥ 80 %) + integration (≥ 70 % of use cases) + contract (every consumer + every event subscriber) + a small set of e2e / chaos scenarios. Coverage is measured by vitest --coverage with c8 (V8 provider). Lines and branches both must hit thresholds; the domain/ package thresholds are higher (lines ≥ 90 %, branches ≥ 85 %).

The three platform-mandatory integration tests (tenant-isolation.spec.ts, outbox.spec.ts, inbox.spec.ts) are non-negotiable readiness gates and run on every PR.

1. Test layout

test/
├── unit/                                  # 100 % isolated; no I/O
│   ├── domain/
│   │   ├── object-key.spec.ts
│   │   ├── content-type.spec.ts
│   │   ├── byte-size.spec.ts
│   │   ├── sha256.spec.ts
│   │   ├── retention-policy-name.spec.ts
│   │   ├── file-object/
│   │   │   ├── lifecycle.spec.ts          # FSM + invariants
│   │   │   ├── alias.spec.ts
│   │   │   ├── ai-provenance.spec.ts
│   │   │   └── domain-events.spec.ts
│   │   ├── upload-session.spec.ts
│   │   ├── variant.spec.ts
│   │   ├── scan-result.spec.ts
│   │   └── access-grant.spec.ts
│   └── application/
│       ├── initiate-upload.spec.ts
│       ├── confirm-upload.spec.ts          # incl. dedupe alias path
│       ├── abort-upload.spec.ts
│       ├── issue-download-url.spec.ts
│       ├── delete-file.spec.ts
│       ├── restore-file.spec.ts
│       ├── erase-by-guest.spec.ts
│       ├── erase-by-tenant.spec.ts
│       ├── apply-retention.spec.ts
│       ├── override-quarantine.spec.ts
│       └── internal-callbacks.spec.ts
├── integration/                            # real Postgres, real Redis, fake GCS, fake Pub/Sub
│   ├── tenant-isolation.spec.ts            # MANDATORY
│   ├── outbox.spec.ts                      # MANDATORY
│   ├── inbox.spec.ts                       # MANDATORY
│   ├── prefix-isolation.spec.ts
│   ├── quota-exhaustion.spec.ts
│   ├── upload-confirm-scan-optimize.spec.ts
│   ├── scan-bypass.spec.ts                 # EICAR + magic-byte + polyglot
│   ├── signed-url-revocation.spec.ts
│   ├── retention-sweep.spec.ts
│   ├── erasure.spec.ts                     # full GDPR cascade incl. CDN invalidate stub
│   ├── ocr-redact.saga.spec.ts
│   ├── alias-dedupe.spec.ts
│   └── concurrent-callbacks.spec.ts        # idempotent re-entry
├── contract/
│   ├── pact/
│   │   ├── property-service.consumer.pact.spec.ts
│   │   ├── notification-service.consumer.pact.spec.ts
│   │   ├── billing-service.consumer.pact.spec.ts
│   │   ├── reservation-service.consumer.pact.spec.ts
│   │   ├── theme-config-service.consumer.pact.spec.ts
│   │   └── bff-backoffice.consumer.pact.spec.ts
│   ├── events/
│   │   ├── file.upload.completed.v1.schema.spec.ts
│   │   ├── file.scan.passed.v1.schema.spec.ts
│   │   ├── file.scan.failed.v1.schema.spec.ts
│   │   ├── file.optimization.completed.v1.schema.spec.ts
│   │   ├── file.deleted.v1.schema.spec.ts
│   │   ├── file.access.denied.v1.schema.spec.ts
│   │   ├── file.retention.expired.v1.schema.spec.ts
│   │   ├── file.erasure.completed.v1.schema.spec.ts
│   │   └── file.bucket.quota_warning.v1.schema.spec.ts
│   └── consumed/
│       ├── tenant.deleted.v1.schema.spec.ts
│       ├── tenant.guest.erasure_requested.v1.schema.spec.ts
│       ├── tenant.plan_changed.v1.schema.spec.ts
│       └── property.photo.removed.v1.schema.spec.ts
└── e2e/
    ├── upload-thumbnail-roundtrip.e2e.spec.ts
    ├── desktop-offline-upload-replay.e2e.spec.ts   # Electron sync engine
    ├── erasure-end-to-end.e2e.spec.ts
    └── low-bandwidth-resumable.e2e.spec.ts          # 56 kbps + 30% packet loss simulator

2. Coverage targets

Layer	Lines	Branches
src/domain/**	≥ 90 %	≥ 85 %
src/application/**	≥ 85 %	≥ 75 %
src/infrastructure/**	≥ 75 %	≥ 65 %
src/presentation/**	≥ 80 %	≥ 70 %
Service-wide aggregate	≥ 80 %	≥ 70 %

Excluded from coverage (/* c8 ignore */ annotated): pure DI wiring (app.module.ts), generated OpenAPI / Pact stubs, bin/* scripts.

3. Mandatory integration tests

3.1 tenant-isolation.spec.ts

Provisions two tenants tnt_A, tnt_B. Asserts:

1. A FileObject created under tnt_A is invisible to a request carrying X-Tenant-Id: tnt_B (RLS).
2. A direct ID guess (GET /files/<med owned by A> from tnt_B) returns 404 RESOURCE_NOT_FOUND (not 403; never leak existence).
3. IssueDownloadUrlUseCase invoked from tnt_B for tnt_A's file ID raises ResourceNotFoundError and emits file.access.denied.v1 with reason='cross_tenant'.
4. The Postgres CHECK constraint on file_objects.object_key rejects a forged insert where object_key does not match tenants/<tenantId>/.
5. The DB trigger blocks a cross-tenant alias insert.

3.2 outbox.spec.ts

Asserts the transactional outbox correctness:

1. Domain events are written in the same transaction as the aggregate (rollback drops both).
2. The relay publishes at-least-once to a fake Pub/Sub.
3. A simulated crash mid-batch (kill the relay process between publish and mark_published_at) results in re-publish on restart, and consumer dedup absorbs the dup.
4. Publish ordering: per-aggregate FIFO observed.

3.3 inbox.spec.ts

Asserts consumed-event correctness:

1. Same messageId delivered twice runs the handler exactly once.
2. Handler crash between work and inbox commit causes re-delivery and re-application is safe (idempotent).
3. Cross-handler isolation: same eventId consumed by two handlers (scan-callback, audit-shadow) runs each exactly once.

4. Domain unit tests of note

Test	What it proves
object-key.spec.ts	toGcsKey() always starts with tenants/<tenantId>/; rejects .. and absolute paths
content-type.spec.ts	Allow-list per scope is enforced (table-driven across all 9 scopes); SVG with inline script rejected
file-object/lifecycle.spec.ts	FSM transitions are exactly the documented set; quarantined is sticky except via OverrideQuarantineUseCase
file-object/alias.spec.ts	Alias only allowed in initiated state; one-hop limit
byte-size.spec.ts	Per-scope cap is enforced; rejects negative / non-integer
retention-policy-name.spec.ts	Unknown names rejected; canonical set is the source of truth

5. Application unit tests of note

Test	What it proves
initiate-upload.spec.ts	Quota reservation is performed; idempotency-key returns the existing session; resumable threshold at 8 MiB
confirm-upload.spec.ts	Hash mismatch aborts session and soft-deletes object; dedupe finds canonical and aliases; non-dedupe path enqueues scan only (not optimize)
issue-download-url.spec.ts	Status gate works for every state; alias resolved to canonical; cross-tenant call rejected; cache reuse within bucket; URL bound to exact path
apply-retention.spec.ts	pii_id_scan triggers OCR saga at redactionAfterDays; quarantined files purged at 30 d; legal hold respected
erase-by-guest.spec.ts	Regulated-class within min retention is deferred; certificate produced even on partial; CDN invalidation invoked
internal-callbacks.spec.ts	Scan callback idempotent (same scanResultId twice); optimize callback batches into single optimization.completed.v1

6. Integration scenarios of note

6.1 prefix-isolation.spec.ts

Spins a real Postgres + a fake GCS adapter that records every call. Forces a code path that bypasses ObjectKey (regression-style: imports the GCS adapter directly with a hand-crafted string) and asserts it throws InvalidObjectKeyError. Also asserts the adapter rejects any key not matching ^tenants/tnt_[0-9A-HJKMNP-TV-Z]{26}/.

6.2 quota-exhaustion.spec.ts

Sets quotas.cap_bytes = 1_000_000 for tnt_A. Initiates uploads totaling > cap and asserts:

• 80 % crossing → melmastoon.file.bucket.quota_warning.v1 (thresholdPct=80) emitted exactly once.
• 95 % crossing → second warning emitted, dampened from re-firing for 1 h.
• 100 % attempt → 402 MELMASTOON.FILE.QUOTA_EXCEEDED.
• Aborted session releases the speculative reservation.

6.3 scan-bypass.spec.ts

Three sub-scenarios: (a) EICAR test signature → quarantined, (b) JPEG with application/pdf declared MIME → magic-byte mismatch → quarantined, (c) JPEG-PNG polyglot → quarantined. Each asserts the appropriate file.scan.failed.v1 with threats=[…] and an access.denied.v1 on subsequent download attempt.

6.4 signed-url-revocation.spec.ts

Issues a download URL, immediately revokes via DELETE /access-grants/{grt}, asserts:

• Redis ZSET contains the fingerprint with TTL = remaining grant TTL.
• A subsequent request with that URL through the private-bucket sidecar returns 403.
• The URL TTL ultimately expires both naturally and in the blacklist.

6.5 retention-sweep.spec.ts

Time-travels (uses an injected Clock):

• A pii_id_scan file ages past redactionAfterDays → OCR redact saga runs → new redacted file with aliasOf set → original purged.
• A default policy file aged past 90 d → hard-purged + file.retention.expired.v1.
• A tax_compliance file aged 100 d → still ready (within 7 y).
• A quarantined file aged 31 d → hard-purged.

6.6 erasure.spec.ts

Creates 14 files for gst_X across mixed scopes (12 erasable, 2 tax_compliance deferred). Runs EraseByGuestUseCase:

• 12 purged in GCS (fake adapter records calls).
• 2 deferred with releasedAt set.
• CDN invalidation invoked synchronously.
• Certificate row written + file.erasure.batch_completed.v1 emitted with the deferred IDs.

7. Contract tests

7.1 Pact (consumer → provider)

Each downstream service ships a consumer Pact that this service replays in the provider verifier:

Consumer	Routes covered
property-service	initiate (property_photo), confirm, get metadata, list variants, delete, download-url
notification-service	initiate (notification_attachment), confirm, download-url
billing-service	initiate (invoice_pdf, receipt_scan), confirm, download-url
reservation-service	initiate (guest_id_scan), confirm, download-url, soft-delete
theme-config-service	initiate (theme_asset, tenant_logo), confirm, download-url
bff-backoffice	quotas, access-log, erasure, override-quarantine

Provider states are seeded by pact-state-handlers.ts. CI publishes results to the Pact broker; deploy is blocked if any consumer's latest Pact is unverified against the candidate.

7.2 Event schemas

Every produced and consumed event has a JSON Schema in contracts/events/*.json. The schema spec validates:

• The example in EVENT_SCHEMAS parses.
• The schema is forward-compatible within .v1 (new versions are additive only).
• Round-trip serialization preserves all fields.

8. E2E scenarios

8.1 upload-thumbnail-roundtrip.e2e.spec.ts

Boots the full stack via docker compose. A test client uploads a 2 MB JPEG, polls until optimization.completed.v1, asserts that GET /files/{med} exposes thumb / hero / full / avif_hero variants and that each downloads successfully through a signed URL.

8.2 desktop-offline-upload-replay.e2e.spec.ts

Uses Playwright + Electron to:

1. Disconnect network.
2. Submit two photo uploads via the renderer UI; verify they queue in the local outbox.
3. Reconnect.
4. Assert the sync engine drains the queue, photos appear in property-service's gallery, local copies are removed.

8.3 erasure-end-to-end.e2e.spec.ts

Creates a guest in reservation-service, attaches an ID scan, triggers tenant.guest.erasure_requested.v1, polls until certificate is available, asserts:

• The file no longer exists in GCS (fake adapter call log).
• reservation-service's Guest.idScanRef is now null.
• The certificate is downloadable and verifies against the platform signer's public key.

8.4 low-bandwidth-resumable.e2e.spec.ts

Uses tc qdisc (Linux) to clamp uplink to 56 kbps with 30 % packet loss. Uploads a 12 MB PDF using the resumable protocol; asserts the upload completes within 10 minutes and survives a forced disconnect-reconnect cycle.

9. Performance / load tests

Run nightly on staging via k6:

• k6/upload-mix.js — 200 RPS mixed initiate+confirm+download with realistic scope distribution.
• k6/download-burst.js — 5000 RPS download-url issuance, 90 % cache reuse expected.
• k6/erasure-large.js — single tenant with 100 k files, single erasure run; assert ≤ 15 min.

Outputs feed Grafana panels under file-storage / load.

10. Chaos cases (low-bandwidth + offline)

Scenario	Tool	Expected behavior
GCS 503 burst	Toxiproxy	Initiate retries with backoff; no client-visible failure within burst window
Pub/Sub publish drops	Toxiproxy	Outbox lag rises, eventual catch-up; no events lost
Redis flush	direct FLUSHDB	Signed URL cache cold-misses but blacklist seeded from Postgres backstop
Optimizer worker poison	hand-crafted image bomb	Per-object retry cap 5 → DLQ; no service crash
Scan worker silent (no callback)	feature flag	Files stay scanning; sweeper marks expired after 60 min; alert fires
KMS unavailable on private bucket	IAM revoke	Reads against private bucket fail closed; metrics + alert
Network partition desktop ↔ cloud	Chaos Monkey for Electron	Desktop outbox queues; reconcile on reconnect

11. Static analysis & linting

• eslint with @ghasi/eslint-config (NestJS + TypeScript strict) + import-restriction rule pack (no NestJS / Drizzle / GCS in domain/).
• tsc --noEmit (strict, all options enabled).
• prettier enforced.
• npm audit --omit=dev blocking on any HIGH/CRITICAL.
• gitleaks blocks on any pattern resembling a service-account JSON key.

12. CI matrix

Job	Where	Blocks merge?
lint	GH Actions	yes
typecheck	GH Actions	yes
unit (vitest with coverage thresholds)	GH Actions	yes
integration (testcontainers postgres + redis + fake GCS + fake Pub/Sub)	GH Actions	yes
contract (pact provider verifier + json-schema validators)	GH Actions	yes
migration-check	GH Actions	yes
openapi-diff	GH Actions	yes (block on breaking)
e2e	nightly	no (alerts on failure)
k6	nightly	no (alerts on regression > 20 %)
pen-test (third party)	yearly	no (planned)