Skip to main content

Testing

:::info Source Sourced from services/progress-service/TESTING_STRATEGY.md in the documentation repo. :::

1. Coverage Targets​

LayerLineBranchMutation
Domain95%98%80%
Application90%90%70%
Integration80%——

2. Unit Tests​

  • Attempt state machine.
  • Completion detection logic (idempotent on duplicate passing statements).
  • GradingRule evaluation (pass/fail threshold).
  • xAPI statement validation (schema, required fields, verb registry).
  • Actor disambiguation (mbox vs account).
  • Duration computation (cross-timezone, DST).

3. Integration Tests (Testcontainers)​

  • Postgres + NATS.
  • Ingest 10k statements; verify dedup; verify attempt closure; verify completion event.
  • Replay statements from event log; verify projections match.
  • Partition detach + archive test.

4. Contract Tests​

Pact:

  • delivery → progress: .play_session.completed.v1 shape.
  • assessment → progress: .attempt_result.scored.v1 shape.
  • progress → certification: progress.completion.recorded.v1.
  • progress → assignment: same.
  • progress → analytics: same.

OpenAPI diff for /api/v1/*.

5. xAPI Conformance​

  • ADL xAPI 1.0.3 test suite green on every release.
  • cmi5 test suite green.
  • xAPI LRS Conformance Badge maintained.

6. E2E Tests​

  • J-05: complete a course → completion event → certificate issued → transcript shows certificate ref.
  • J-11: retake a failed attempt → new attempt → pass → new completion idempotent.
  • J-12: airplane-mode — queue 50 statements offline → reconnect → all ingested.

7. Load Tests​

  • 10k statements/sec sustained ingestion (k6).
  • 1M statement query (xAPI since window) p95 < 2s.
  • Transcript render for learner with 10k statements p95 < 5s.

8. Chaos Tests​

  • Kill Postgres primary mid-ingest → verify no statement loss (outbox retained on client).
  • Partition Postgres → NATS for 2 min → verify outbox drains on reconnect.
  • Corrupt a statement row (manually) → verify Merkle-anchored audit detects.
  • Replay 1M statements on fresh DB → verify projections reproduce byte-exact.

9. Offline Tests​

  • Client queues 1000 statements; reconnects; all ingest.
  • Duplicate statements on re-push → deduped on statementId.
  • Partial batch failure → successful statements persisted, failed retried.

10. AI Safety Tests​

  • No AI owned here, but data-contract tests ensure PII redaction for AI consumer layer.

11. Replay Tests​

  • Event log → rebuild attempts + completion_records tables.
  • Compare row-by-row with previous state; bit-exact for completion records.
  • Statements are raw events; log is the statements.

12. Security Tests​

  • Cross-tenant statement ingestion attempt → rejected.
  • Spoofed actor (different userId than JWT sub) → rejected.
  • Signed statement with invalid signature → rejected.
  • GDPR erasure replay: user deleted, later query returns empty.

13. CI Gates​

  • Unit + integration pass.
  • OpenAPI + xAPI conformance green.
  • Pact contracts verified.
  • Two-tenant isolation test green.
  • Mutation score ≥ 80%.
  • Migration fwd+back test green.