Testing

:::info Source Sourced from services/identity-service/TESTING_STRATEGY.md in the documentation repo. :::

Companion: 16 Testing Strategy & QA · DOMAIN_MODEL · SECURITY_MODEL

Identity-service test strategy targets 80%+ line coverage, 100% coverage of security-critical paths, and includes dedicated contract, integration, security, and E2E suites.

1. Test Pyramid

                    ┌──────────────┐
                    │  E2E Tests   │  ~20 tests
                    │  (Playwright)│
                    └──────────────┘
                ┌──────────────────────┐
                │  Security Tests      │  ~40 tests
                │  (fuzzing, simulation)│
                └──────────────────────┘
            ┌──────────────────────────────┐
            │  Integration Tests            │  ~120 tests
            │  (Testcontainers + real infra)│
            └──────────────────────────────┘
        ┌──────────────────────────────────────┐
        │  Contract Tests (Pact)                │  ~15 pairs
        └──────────────────────────────────────┘
    ┌────────────────────────────────────────────────┐
    │  Unit Tests                                     │  ~400 tests
    │  (Domain invariants, use cases, pure logic)    │
    └────────────────────────────────────────────────┘

2. Unit Tests

2.1 Scope

Pure domain logic, zero infrastructure:

• Value object construction and validation
• Aggregate invariants and state transitions
• Domain services (PasswordPolicyService, LockoutPolicyService, AdaptiveMFAService)
• Use case orchestration logic (with mocked ports)

2.2 Framework

• Runner: vitest (fast, TypeScript-native)
• Assertion: built-in + @testing-library/jest-dom equivalents
• Mocks: vitest mock functions; manual test doubles for ports

2.3 Coverage Targets

Module	Target
Domain aggregates	100%
Domain services	100%
Value objects	100%
Application handlers	90%
Infrastructure adapters	80%
Overall	80%+

2.4 Key Unit Test Suites

2.4.1 User Aggregate

describe('User aggregate', () => {
  test('enforces email uniqueness invariant', () => { /* U-INV-1 */ });
  test('requires at least one credential or external identity', () => { /* U-INV-2 */ });
  test('rejects login when status is locked', () => { /* U-INV-3 */ });
  test('rejects login when status is disabled', () => { /* U-INV-4 */ });
  test('email verification is one-way (cannot un-verify)', () => { /* U-INV-6 */ });
  test('maximum 5 MFA factors enforced', () => { /* U-INV-7 */ });
  // state transitions
  test('pending_verification → active on verify_email', () => {});
  test('active → locked on failed attempts threshold', () => {});
  test('locked → active on lockedUntil expiry', () => {});
});

2.4.2 Credential Entity

describe('Credential', () => {
  test('only one password credential per user', () => { /* C-INV-1 */ });
  test('failedAttempts increments on failure, resets on success', () => { /* C-INV-2 */ });
  test('progressive lockout: 5 failures → 15 min, 10 → 30 min, 15 → 60 min, 20 → 120 min', () => { /* C-INV-3 */ });
  test('rotatedAt updates on password change', () => { /* C-INV-6 */ });
});

2.4.3 Session Aggregate

describe('Session', () => {
  test('revoked sessions cannot be refreshed', () => { /* S-INV-2 */ });
  test('refresh token rotation: new hash replaces old, old goes to history', () => {});
  test('rotation reuse detection: reusing old hash revokes entire family', () => { /* S-INV-4 */ });
  test('max 10 active sessions per user; oldest revoked on overflow', () => { /* S-INV-5 */ });
});

2.4.4 PasswordPolicyService

describe('PasswordPolicyService', () => {
  test('rejects password < 12 chars', () => {});
  test('rejects password missing character classes', () => {});
  test('rejects password equal to email or variations', () => {});
  test('rejects password in rotation history (last 5)', () => {});
  test('rejects password found in breach list', async () => {
    const mockBreachChecker = { check: vi.fn().mockResolvedValue(true) };
    // ...
  });
  test('accepts valid strong password', () => {});
});

2.4.5 AdaptiveMFAService

describe('AdaptiveMFAService', () => {
  test('requires MFA for unknown device', () => {});
  test('requires MFA for impossible travel', () => {});
  test('does not lower MFA threshold based on AI output', () => {
    // fallback safety: AI says low risk, rules say high risk → MFA required
  });
  test('falls back to rules when ai-gateway unavailable', () => {});
  test('respects tenant-level MFA policy override', () => {});
});

2.4.6 Token Signing

describe('TokenSigner', () => {
  test('signs JWT with EdDSA Ed25519', () => {});
  test('includes kid in header', () => {});
  test('includes all required claims', () => {});
  test('rejects signing when KMS unavailable', () => {});
});

3. Integration Tests

3.1 Scope

Full service running against real infrastructure via Testcontainers:

• Postgres (real schema, migrations applied)
• Redis
• NATS JetStream
• Mock KMS (testcontainers kms-local image)

3.2 Framework

• Runner: vitest with --pool=forks for isolation
• Infrastructure: testcontainers-node
• HTTP: supertest against NestJS app

3.3 Key Integration Test Suites

3.3.1 End-to-end auth flow

describe('Auth integration', () => {
  beforeEach(async () => {
    await resetDatabase();
    await resetRedis();
  });

  test('register → verify email → login → refresh → logout', async () => {
    // register
    const { userId } = await post('/api/v1/auth/register', { email: 'u@test.io', password: 'StrongP@ss2026' });
    // verify
    const verifyToken = await getVerifyTokenFromOutbox(userId);
    await post('/api/v1/auth/email/verify', { token: verifyToken });
    // login
    const { accessToken, refreshToken } = await post('/api/v1/auth/login', { email: 'u@test.io', password: 'StrongP@ss2026' });
    expect(accessToken).toBeDefined();
    // refresh
    const rotated = await post('/api/v1/auth/refresh', { refreshToken });
    expect(rotated.refreshToken).not.toBe(refreshToken);
    // logout
    await post('/api/v1/auth/logout', {}, { Authorization: `Bearer ${rotated.accessToken}` });
    // old refresh token no longer works
    const retry = await post('/api/v1/auth/refresh', { refreshToken: rotated.refreshToken });
    expect(retry.status).toBe(401);
  });
});

3.3.2 Lockout enforcement

test('5 failed logins lock account for 15 minutes', async () => {
  for (let i = 0; i < 5; i++) {
    await post('/api/v1/auth/login', { email, password: 'wrong' });
  }
  const response = await post('/api/v1/auth/login', { email, password });
  expect(response.status).toBe(423);
  expect(response.body.error.code).toBe('auth.account_locked');
});

3.3.3 Refresh token rotation reuse

test('refresh token reuse revokes entire session family', async () => {
  const { refreshToken: rt1 } = await login();
  const { refreshToken: rt2 } = await post('/api/v1/auth/refresh', { refreshToken: rt1 });
  // reuse old token
  const reuse = await post('/api/v1/auth/refresh', { refreshToken: rt1 });
  expect(reuse.status).toBe(401);
  // new token should also be revoked
  const later = await post('/api/v1/auth/refresh', { refreshToken: rt2 });
  expect(later.status).toBe(401);
});

3.3.4 Device binding flow

test('register device → trust → bind offline → certificate valid', async () => {
  const { deviceId } = await post('/api/v1/users/me/devices', { fingerprint, publicKey });
  await trustDevice(deviceId);
  const { certificate } = await post(`/api/v1/users/me/devices/${deviceId}/bind-offline`, {});
  expect(certificate).toContain('BEGIN CERTIFICATE');
  // verify cert signature against identity CA
  const valid = await verifyCert(certificate);
  expect(valid).toBe(true);
});

3.3.5 Outbox → NATS

test('user registration emits identity.user.registered.v1', async () => {
  const natsSpy = await subscribeToNATS('identity.user.registered.v1');
  await post('/api/v1/auth/register', { email, password });
  const event = await natsSpy.waitForMessage(5000);
  expect(event.payload.primaryEmail).toBe(email);
  expect(event.eventVersion).toBe(1);
  // schema validation
  expect(() => validateSchema('identity/user/registered/v1', event.payload)).not.toThrow();
});

3.3.6 Two-tenant isolation

describe('Two-tenant isolation', () => {
  test('user A in tenant 1 cannot access user B in tenant 2', async () => {
    const tokenA = await loginAs('userA', 'tenant1');
    const response = await get('/api/v1/users/userB', {}, { Authorization: `Bearer ${tokenA}`, 'X-Tenant-Id': 'tenant2' });
    expect(response.status).toBe(403);
  });

  test('API key scoped to tenant 1 rejected for tenant 2 resources', async () => {
    const apiKey = await createAPIKey('tenant1');
    const response = await get('/api/v1/resource', {}, { 'X-API-Key': apiKey, 'X-Tenant-Id': 'tenant2' });
    expect(response.status).toBe(403);
  });

  test('session issued for tenant 1 has tid=tenant1 only', async () => {
    const token = await loginAs('user', 'tenant1');
    const decoded = decodeJWT(token);
    expect(decoded.tid).toBe('tenant1');
    expect(decoded.tids).not.toContain('tenant2');
  });
});

3.4 Migrations Testing

describe('Database migrations', () => {
  test('all migrations apply cleanly on empty database', async () => {
    await runMigrations('up', 'latest');
  });

  test('all migrations are reversible', async () => {
    await runMigrations('up', 'latest');
    await runMigrations('down', 0);
    await runMigrations('up', 'latest');
  });

  test('migrations are idempotent (re-applying is safe)', async () => {
    await runMigrations('up', 'latest');
    await runMigrations('up', 'latest');
  });
});

4. Contract Tests (Pact)

Identity-service is a provider for several consumers. Pact tests verify that event schemas and JWT claim shapes remain stable.

4.1 Pact Interactions

Consumer	Contract
tenant-service	Consumes identity.user.registered.v1 → must have userId, primaryEmail, homeTenantId?
notification-service	Consumes identity.password.reset_requested.v1 → must have userId, resetTokenHash
content-service	Consumes identity.device.bound_for_offline.v1 → must have userId, deviceId, publicKeyFingerprint
sync-service	Consumes identity.device.bound_for_offline.v1, identity.session.revoked.v1
All services	JWT claim schema (iss, aud, sub, tid, tids, roles, scope, amr, did, jti, v)

4.2 Pact Broker Integration

• Broker hosted at pact.ghasi.io.
• CI uploads identity-service provider pacts on merge to main.
• Consumer CIs fetch latest pacts and verify their expectations hold.
• Breaking change to identity blocks consumer CI; surface to identity team PR.

4.3 Example Pact Test

describe('Pact: identity-service provider', () => {
  const provider = new Pact.Verifier({
    providerBaseUrl: 'http://localhost:3001',
    pactBrokerUrl: 'https://pact.ghasi.io',
    providerName: 'identity-service',
    consumerVersionSelectors: [{ latest: true }],
  });

  test('verifies pacts with tenant-service', async () => {
    await provider.verifyProvider();
  });
});

5. E2E Tests

5.1 Scope

Full browser-based flows against a deployed identity-service (staging).

5.2 Framework

• Playwright for web flows
• Playwright for .NET / Java N/A — not cross-language here
• Appium for mobile flows (if needed)

5.3 Key E2E Scenarios

#	Scenario
E2E-1	User registers, receives verification email, verifies, logs in
E2E-2	User enables TOTP MFA, logs out, logs in with MFA challenge
E2E-3	User requests password reset, receives email, resets, logs in with new password
E2E-4	User logs in via Google SSO (JIT provision), profile includes external identity
E2E-5	User logs in via SAML (corporate IdP, mocked), session includes SAML AMR
E2E-6	User registers a device, gets offline binding certificate, certificate validates
E2E-7	User creates an API key, uses it for a catalog-service call
E2E-8	User logs in, is locked after 5 failed attempts, automatic unlock after 15 min
E2E-9	User with active session in tenant 1 switches to tenant 2 (session re-mint)
E2E-10	Admin locks a user; user cannot log in; admin unlocks; user can log in

6. Security Tests

6.1 Credential Stuffing Simulation

Load-test harness simulates a credential stuffing attack:

• 10,000 email/password pairs from public breach lists
• 1,000 requests/second sustained
• Assertions:
  • < 1% of requests slip past rate limiter
  • Account lockouts triggered appropriately
  • No cross-account impact
  • Metrics identity_login_failures_total reflect actual volume
  • Adaptive MFA triggers elevate to MFA on suspicious patterns

6.2 Timing Attack

test('login response time is constant regardless of email existence', async () => {
  const t1 = await measure(() => post('/auth/login', { email: 'exists@ex.io', password: 'wrong' }));
  const t2 = await measure(() => post('/auth/login', { email: 'nonexistent@ex.io', password: 'wrong' }));
  // within 10% of each other
  expect(Math.abs(t1 - t2) / Math.max(t1, t2)).toBeLessThan(0.1);
});

6.3 Token Tampering

test('tampered JWT is rejected', async () => {
  const token = await login();
  const parts = token.split('.');
  parts[1] = base64url(JSON.stringify({ ...decodePayload(parts[1]), roles: ['platform_admin'] }));
  const tampered = parts.join('.');
  const response = await get('/api/v1/users/me', {}, { Authorization: `Bearer ${tampered}` });
  expect(response.status).toBe(401);
});

6.4 SAML Assertion Replay

test('replayed SAML assertion is rejected', async () => {
  const assertion = await captureSAMLAssertion();
  await post('/auth/sso/saml/callback', { SAMLResponse: assertion });
  const replay = await post('/auth/sso/saml/callback', { SAMLResponse: assertion });
  expect(replay.status).toBe(400);
  expect(replay.body.error.code).toBe('auth.sso_assertion_invalid');
});

6.5 SQL Injection

• Fuzzing harness sends common SQLi payloads to every string input.
• Assert all rejected at validation layer; none reach SQL.

6.6 Dependency Vulnerability Scan

• npm audit on every CI run; fail on high/critical.
• Snyk scheduled scans weekly.
• SBOM generated; submitted to platform SCA.

6.7 OWASP ASVS Checks

Automated ASVS L2 compliance checklist:

• V2 Authentication
• V3 Session Management
• V6 Cryptography
• V7 Error handling and logging
• V8 Data Protection
• V13 API and Web Service

7. Load & Performance Tests

k6 scripts under tests/load/:

Script	Target	Assertion
login-steady.js	1,000 logins/sec sustained for 10 min	p95 < 100ms, error rate < 0.1%
refresh-burst.js	10,000 refreshes in 60s	p95 < 30ms
jwks-cache.js	10,000 JWKS requests/sec	100% cache hit rate
register-ramp.js	Ramp from 0 to 100 registrations/sec	DB pool not saturated
mfa-challenge.js	500 MFA challenges/sec	p95 < 150ms

8. Chaos Tests

Run quarterly:

Chaos Scenario	Expected Behavior
Kill Postgres primary	Failover < 30s; no data loss
Kill NATS JetStream leader	Replication picks up; outbox retries
Kill Redis	Rate limiting degrades gracefully (fail-open with stricter static limits)
Kill KMS	Existing tokens verify; no new tokens signed; alert fires
Network partition to ai-gateway	Adaptive MFA falls back to rules
Disk full on Postgres	Circuit-breaker kicks in; writes rejected with 503
Slow query (10x baseline)	Timeout + fallback

9. Coverage Enforcement

CI pipeline:

1. Run unit + integration tests.
2. Generate coverage report (c8 or istanbul).
3. Fail if:
   • Overall coverage < 80%
   • Domain layer coverage < 100%
   • Security-critical modules (auth, credential, session) < 95%

Coverage report published to Codecov; trend-tracked.

10. Test Data Management

• Factories: @ghasi/test-factories provides builders for User, Session, Device, etc.
• Seed data: Deterministic seed for staging environment; synthetic data for load tests.
• GDPR-safe: No real user data in test environments; all fixtures synthetic.

11. Mutation Testing

Run monthly with stryker:

• Target: domain layer has mutation score ≥ 80%.
• Surfaces weak assertions that pass despite bugs.

12. Accessibility Testing

Identity-service endpoints are consumed by the frontend; frontend team owns a11y tests. Identity-service ensures:

• Error messages are clear, non-jargon
• WWW-Authenticate headers correct
• No reliance on timing or visual feedback for auth flows

13. Chaos + Security = Red Team

Quarterly internal red team exercise:

• Attack surface probe
• Credential stuffing simulation
• Token theft simulation
• SSO provider substitution
• Side-channel timing attacks

Results feed into the risk register.