Skip to main content

Testing

:::info Source Sourced from services/analytics-service/TESTING_STRATEGY.md in the documentation repo. :::

1. Coverage Targets​

Domain 95% / 80% mutation. Integration 80%.

2. Unit Tests​

  • Metric query template rendering.
  • SQL parser + tenant filter injection.
  • Dashboard widget rendering logic.
  • Cohort criteria evaluation against fixture events.

3. Integration Tests (Testcontainers)​

  • Postgres + NATS + ClickHouse + Kafka Connect.
  • Full pipeline: emit event → NATS → Kafka → ClickHouse → materialized view updated → dashboard reflects.
  • Export job: query → CSV → S3 → signed URL.

4. Contract Tests​

  • Consumes event schemas from all services — validated against schema registry.
  • OpenAPI diff for API endpoints.

5. E2E​

  • J-15: admin creates dashboard → sees learner progress.
  • J-16: compliance officer exports audit data.

6. Load Tests​

  • Ingest 100k events/sec sustained.
  • 100 concurrent dashboard renders.
  • 10 concurrent exports of 1M rows.

7. Chaos​

  • Kafka Connect down → events buffer in NATS; drain on recovery.
  • ClickHouse node failure → queries fail over to replica.
  • Dashboard cache eviction storm → DB handled; rate limits apply.

8. Security Tests​

  • Cross-tenant query attempt via raw SQL → blocked.
  • PII leak via export without redaction → denied.
  • Compliance export without role → 403.
  • AI-generated SQL with destructive op → rejected.

9. AI Tests​

  • NL query accuracy eval on question corpus.
  • At-risk predictor bias eval (demographic parity).
  • Anomaly detection false-positive rate.

10. GDPR Tests​

  • Erasure replays across hot + cold; verifies zero rows remain.

11. CI Gates​

Unit + integration + two-tenant iso + schema registry conformance + mutation ≥ 80%.