Testing

:::info Source Sourced from services/tenant-service/TESTING_STRATEGY.md in the documentation repo. :::

Blueprint doc 11 of 17. Companion: 16 Testing Strategy | SECURITY_MODEL

---

1. Test Pyramid

                          ┌─────────────────┐
                          │     E2E (~20)   │   Playwright, full stack, ephemeral tenants
                          └─────────────────┘
                       ┌──────────────────────┐
                       │   Contract (~60)     │   Pact (API + events), schema CI
                       └──────────────────────┘
                    ┌──────────────────────────┐
                    │  Integration (~250)      │   Testcontainers (PG, Redis, NATS)
                    └──────────────────────────┘
                 ┌──────────────────────────────┐
                 │     Unit (~1,200)            │   Vitest, pure domain + application
                 └──────────────────────────────┘

Coverage target: ≥ 85% lines / ≥ 90% branches on domain + application layers. Infrastructure adapters: ≥ 70% lines.

---

2. Unit Tests (Domain Layer)

Pure TypeScript, zero framework, Vitest runner. Tests read like the ubiquitous language.

2.1 Aggregate Tests

describe('Tenant', () => {
  test('provisions with trial status when no plan', () => {
    const t = Tenant.provision({ name: 'Acme', slug: 'acme', type: 'org', homeRegion: 'eu', plan: null, ownerEmail: 'x@acme.com' });
    expect(t.status).toBe('trial');
  });

  test('rejects status transition active → trial', () => {
    const t = makeActiveTenant();
    expect(() => t.trialize()).toThrowAggregateError('INVALID_STATUS_TRANSITION');
  });

  test('homeRegion is immutable except via residency saga', () => {
    const t = makeActiveTenant({ homeRegion: 'eu' });
    expect(() => t.updateHomeRegion('us')).toThrowAggregateError('HOME_REGION_IMMUTABLE');
    expect(() => t.migrateResidency('us', { sagaId: 'sga_01HX' })).not.toThrow();
  });
});

describe('OrgUnit', () => {
  test('computes ltree path from parent chain', () => {
    const root = OrgUnit.create({ tenantId, parent: null, name: { en: 'Acme' } });
    const child = OrgUnit.create({ tenantId, parent: root, name: { en: 'Engineering' } });
    expect(child.ltreePath).toBe('acme.engineering');
  });

  test('rejects move that creates cycle', () => {
    const a = makeOrgUnit('a');
    const b = OrgUnit.create({ parent: a, name: { en: 'b' } });
    const c = OrgUnit.create({ parent: b, name: { en: 'c' } });
    expect(() => a.moveUnder(c)).toThrowAggregateError('CYCLE_DETECTED');
  });

  test('rejects depth beyond tenant maxOrgDepth', () => {
    let node = makeOrgUnit('root', { maxDepth: 3 });
    for (let i = 0; i < 2; i++) node = OrgUnit.create({ parent: node, name: { en: `l${i}` } });
    expect(() => OrgUnit.create({ parent: node, name: { en: 'too_deep' } })).toThrowAggregateError('MAX_DEPTH_EXCEEDED');
  });
});

describe('Membership', () => {
  test('activating an invited membership sets joinedAt', () => {
    const m = Membership.invite({ tenantId, userId, roleIds: ['learner'] });
    m.activate({ now: '2026-04-15T10:00:00Z' });
    expect(m.status).toBe('active');
    expect(m.joinedAt).toBe('2026-04-15T10:00:00Z');
  });

  test('cannot suspend invited membership without activation', () => {
    const m = Membership.invite({ tenantId, userId, roleIds: ['learner'] });
    expect(() => m.suspend('reason')).toThrowAggregateError('INVALID_STATUS_TRANSITION');
  });
});

describe('Role invariants', () => {
  test('system role rejects modification', () => {
    const r = Role.system({ name: 'learner', permissions: [] });
    expect(() => r.updatePermissions([...])).toThrowAggregateError('SYSTEM_ROLE_IMMUTABLE');
  });

  test('permission with unknown resource rejected', () => {
    expect(() => Role.create({ name: 'x', permissions: [{ resource: 'nonexistent', action: 'read' }] }))
      .toThrowAggregateError('UNKNOWN_RESOURCE');
  });
});

2.2 Policy Engine Tests

describe('PolicyEngine', () => {
  test.each([
    [{ op: 'eq', field: 'ctx.tenant_id', value: 'tnt_A' }, { ctx: { tenant_id: 'tnt_A' } }, true],
    [{ op: 'eq', field: 'ctx.tenant_id', value: 'tnt_A' }, { ctx: { tenant_id: 'tnt_B' } }, false],
    [{ op: 'in', field: 'resource.org_unit_id', values: ['a', 'b'] }, { resource: { org_unit_id: 'a' } }, true],
    [{ op: 'and', conditions: [{ op: 'eq', ... }, { op: 'eq', ... }] }, {...}, true],
    [{ op: 'not', condition: { op: 'eq', ... } }, {...}, false],
  ])('%o against %o → %s', (predicate, context, expected) => {
    expect(policyEngine.evaluate(predicate, context)).toBe(expected);
  });

  test('max depth 10 enforced on nested predicates', () => {
    const deep = makeNestedPredicate(11);
    expect(() => policyEngine.validate(deep)).toThrow('MAX_DEPTH_EXCEEDED');
  });
});

2.3 ABAC DSL Fuzzing

import fc from 'fast-check';
describe('ABACQuery fuzzing', () => {
  test('any valid AST serializes and parses idempotently', () => {
    fc.assert(fc.property(validABACQueryArb(), (query) => {
      const serialized = JSON.stringify(query);
      const parsed = ABACQuerySchema.parse(JSON.parse(serialized));
      expect(parsed).toEqual(query);
    }));
  });

  test('any random JSON rejected unless it matches DSL', () => {
    fc.assert(fc.property(fc.jsonObject(), (json) => {
      const result = ABACQuerySchema.safeParse(json);
      if (result.success) {
        // if parse succeeded, must match DSL shape
        expect(isValidABACQuery(result.data)).toBe(true);
      }
    }));
  });
});

---

3. Integration Tests

Testcontainers with Postgres 16, Redis 7, NATS JetStream. Each test gets an isolated schema.

3.1 RLS + Isolation

describe('Postgres RLS', () => {
  test('SELECT without WHERE only returns own tenant rows', async () => {
    await withTenantContext(tenantA, async (conn) => {
      await conn.query(`INSERT INTO tenant.memberships (tenant_id, user_id, email, status) VALUES ($1, $2, $3, 'active')`, [tenantA.id, 'user1', 'u1@a.com']);
    });
    await withTenantContext(tenantB, async (conn) => {
      await conn.query(`INSERT INTO tenant.memberships (tenant_id, user_id, email, status) VALUES ($1, $2, $3, 'active')`, [tenantB.id, 'user2', 'u2@b.com']);
    });
    const rowsA = await withTenantContext(tenantA, (c) => c.query('SELECT * FROM tenant.memberships'));
    expect(rowsA.rows).toHaveLength(1);
    expect(rowsA.rows[0].tenant_id).toBe(tenantA.id);
  });

  test('attempt to set other tenant fails RLS policy', async () => {
    await withTenantContext(tenantA, async (conn) => {
      await expect(conn.query(
        `INSERT INTO tenant.memberships (tenant_id, user_id, email, status) VALUES ($1, $2, $3, 'active')`,
        [tenantB.id, 'userX', 'x@b.com']
      )).rejects.toThrow(/row-level security/);
    });
  });
});

3.2 Outbox + NATS Publish

describe('Outbox publish flow', () => {
  test('tenant provision writes outbox row and relay publishes to NATS', async () => {
    const { tenantId } = await provisionTenant({ name: 'Test', slug: 'test', ...});
    const outboxRow = await db.oneOrNone(`SELECT * FROM tenant.outbox WHERE topic LIKE 'tenant.org.provisioned%' AND tenant_id=$1`, [tenantId]);
    expect(outboxRow).toBeTruthy();
    // wait for relay
    await waitFor(async () => {
      const messages = await natsTestSub.receivedBefore(2_000);
      expect(messages.some(m => m.subject.startsWith('tenant.org.provisioned'))).toBe(true);
    });
  });

  test('outbox idempotent on re-attempt', async () => {
    // simulate crash mid-publish; second attempt should not duplicate
  });
});

3.3 LTree Operations

describe('OrgUnit ltree integrity', () => {
  test('moving subtree recomputes all descendant paths', async () => {
    const a = await createOU('acme');
    const b = await createOU('engineering', a.id);
    const c = await createOU('platform', b.id);
    const d = await createOU('sales');
    await moveOU(b.id, d.id);  // move engineering under sales
    const updated = await loadOU(c.id);
    expect(updated.ltreePath).toBe('sales.engineering.platform');
  });
});

3.4 Event Consumption (Inbox)

describe('identity.user.registered.v1 handler', () => {
  test('activates matching pending invitations', async () => {
    await inviteMember(tenantA, 'alice@acme.com');
    await publishEvent('identity.user.registered.v1', { userId: 'usr_01HX', email: 'alice@acme.com', ... });
    await waitFor(async () => {
      const m = await loadMembershipByEmail(tenantA, 'alice@acme.com');
      expect(m.status).toBe('active');
    });
  });

  test('duplicate event is deduped via inbox', async () => {
    const evt = { eventId: 'evt_01HX', ... };
    await publishEvent('identity.user.registered.v1', evt);
    await publishEvent('identity.user.registered.v1', evt);  // duplicate
    const rows = await db.query(`SELECT * FROM tenant.inbox WHERE event_id=$1`, ['evt_01HX']);
    expect(rows.length).toBe(1);
  });
});

---

4. Contract Tests (Pact)

4.1 API Contracts (Consumer-Driven)

Consumers: web client, mobile client, admin portal. Each publishes Pact contract; CI verifies tenant-service against all.

Example contract (web client ↔ tenant-service for member list):

interaction: list memberships
given: tenant tnt_A exists with 3 active members
request:
  method: GET
  path: /api/v1/tenants/tnt_A/memberships
  headers:
    Authorization: Bearer eyJ...
    X-Tenant-Id: tnt_A
  query:
    status: active
response:
  status: 200
  body:
    data: [{ id: eachLike('mbr_01'), userId: eachLike('usr_01'), status: 'active', roleIds: eachLike('rol_01') }]
    meta: { page: { size: 50, totalApproximate: 3 } }

4.2 Event Contracts

Consumers of tenant events publish schema conformance tests:

Consumer	Events verified
identity-service	tenant.org.user_invited.v1, tenant.org.membership_suspended.v1
notification-service	tenant.org.user_invited.v1, tenant.org.membership_activated.v1
enrollment-service	tenant.org.membership_activated.v1, tenant.org.membership_suspended.v1
assignment-service	tenant.dynamic_group.evaluated.v1, tenant.org_unit.moved.v1
analytics-service	All tenant events

4.3 JSON Schema CI Gate

Every event schema has a registered JSON Schema. CI runs:

1. Schema syntax validation.
2. Breaking-change detection against prior version (additive-only).
3. Payload fixture validation (round-trip with generator).

---

5. E2E Tests (Playwright)

Full-stack scenarios against staging environment with ephemeral tenants.

Scenario	Story
Provision org + invite + accept + switch tenant	US-1, US-9
Define org units, assign to members	US-6
Define custom role + assign + authz check	US-7
Dynamic group eval + assignment target	US-8
SSO admin configuration + SP-initiated login	US-10
CSV bulk user import	US-11
Tenant suspension flow (admin-triggered)	admin ops
Feature flag override + UI gating	admin ops

---

6. Security Tests

6.1 Two-Tenant Simulator (mandatory CI gate)

See SECURITY_MODEL.md §4.1. Runs on every PR.

6.2 SAST

• CodeQL on every push
• Semgrep with custom rules (tenant_id missing in WHERE clause, dangerous RLS bypass)
• npm audit / Snyk on lockfile

6.3 DAST

• OWASP ZAP weekly in staging
• Burp Suite quarterly penetration (manual + automated)

6.4 AuthZ Matrix Test

// For every (role, resource, action) combination, verify expected allow/deny
describe('AuthZ matrix', () => {
  test.each(roleResourceActionMatrix)(
    '%s can %s %s: %s',
    (roleName, action, resource, expectedAllowed) => {
      const decision = authzCheck({ roleName, resource, action });
      expect(decision.allowed).toBe(expectedAllowed);
    }
  );
});

Matrix size: 11 roles × ~50 resource-action pairs = 550 cases. Auto-generated from permission registry.

6.5 Injection Tests

• SQL injection attempts on every query parameter
• ABAC DSL injection (attempts to submit eval()-like predicates)
• SAML XML XXE (external entity injection)
• JWT tampering (kid manipulation, alg: none)

---

7. Performance Tests (k6)

7.1 Target SLIs

Endpoint	Target
POST /authz/check (cached)	10k rps; p95 ≤ 5ms; p99 ≤ 15ms
POST /authz/check (uncached)	2k rps; p95 ≤ 20ms; p99 ≤ 50ms
GET /tenants/{id}	5k rps; p95 ≤ 30ms
POST /tenants/{id}/memberships (invite)	500 rps; p95 ≤ 200ms
POST /tenants/{id}/dynamic-groups/{gid}/evaluate	10 rps; p95 ≤ 5s (group size 10k)

7.2 Saturation Tests

• Ramp: 1k → 20k rps over 10 min
• Identify breakpoint; document recovery characteristic
• Verify graceful degradation (rate limiting kicks in, no cascading failures)

7.3 Dynamic Group Stress

• Tenant with 100k memberships, 50 dynamic groups, re-eval all
• Verify within SLO; verify pagination works

---

8. Chaos Tests

Scenario	Expected behavior
Postgres primary fails	Read replicas serve cached authz; writes fail gracefully with 503 + Retry-After
Redis cluster drop	Cache miss storm mitigated by request coalescing; DB-backed fallback
NATS JetStream outage	Outbox fills; service degrades writes with 503 after threshold; resume on recovery
Full AZ outage (1 of 3)	Traffic reroutes; no data loss; brief elevated latency
Clock skew on node	Token expiry logic tolerates ±60s; alert if > 5min
KMS unavailable	Envelope decryption of SSO secrets fails; SSO config reads return cached; writes fail with 503

Chaos drill frequency: monthly in staging, quarterly in pre-prod.

---

9. Replay Tests

Per platform requirement (16 §14):

• Rebuild projections from seq=0 on a synthetic tenant; verify final state matches authoritative DB state.
• Dynamic group evaluations are idempotent (same query hash + same membership state = same result).
• Outbox → NATS → inbox flow loss-free under random crash injection.

---

10. CI Quality Gates

Gate	Blocking?
Unit tests pass	Yes
Integration tests pass	Yes
Two-tenant isolation suite pass	Yes
Coverage ≥ 85% domain+app	Yes
Contract tests (Pact) pass vs registered consumers	Yes
Event schema backward compatibility	Yes
OpenAPI diff check (no breaking changes without v-bump)	Yes
CodeQL no HIGH/CRITICAL	Yes
Semgrep no HIGH/CRITICAL	Yes
SBOM generated + uploaded	Yes
k6 smoke test (staging)	Yes on main merges
Prompt regression (AI features)	Yes if AI prompts touched

---

11. Test Environments

Env	Purpose	Data
dev	Developer local	Seeded synthetic
ci	PR verification	Ephemeral per run
staging	Pre-prod E2E	Synthetic anonymized clone
pre-prod	Canary + perf	Shadow-traffic mirror
prod	Live	Real

---

12. Flake Policy

• Flaky tests quarantined immediately on second failure.
• Quarantine timebox: 5 business days to fix or delete.
• Quarantined suite runs separately; must return to main gate or be deleted at timebox end.
• Root-cause required for every quarantine (usually: timing, shared state, unbounded resource).