Migration Backward Compatibility

:::info Source Sourced from docs/roadmap/migration-backward-compatibility.md in the documentation repo. :::

Execution-layer companion to ROADMAP.md and architectural-freeze-points.md.

This document defines what must remain backward-compatible across slices, how existing tenants migrate, how offline bundles evolve, how AI prompt versions evolve, how event schemas version, and how export-format versions evolve.

1. Overarching Rules

1. No breaking changes to customer-facing surfaces without a migration window. A "migration window" is the overlap period where both old + new work simultaneously.
2. Additive is safe by default. Adding an optional field, a new event subject, a new error code, or a new API endpoint is always permitted without a migration window.
3. Removing, renaming, or narrowing is breaking. These require a new vN, a migration window documented here, and customer communication.
4. Offline bundles live forever on devices. Any schema change that would break a mounted bundle must be forward-compatible in the client or accompanied by a mandatory re-download.
5. AI prompt versions are append-only per tenant. Deprecated prompts remain callable for their deprecation window (≥ 1 milestone). Tenants that pin a version keep it until they explicitly upgrade.
6. Event versions follow dual-publish windows. Producers publish both vN and vN+1 for ≥ 1 release cycle; consumers declare which they consume; CI blocks removal of old producer until zero consumers register for it.

2. Per-Slice Backward-Compatibility Requirements

2.1 S0 → S1

Artifact	Compat requirement
EventEnvelope	Frozen at M0 end; S1 only adds new subjects, not new envelope fields.
Sync pull/push API	v1 locked; S1 adds new entity registrations but pull/push semantics unchanged.
AIClient port	Frozen; S1 adds SSE streaming (additive method).
Prompt registry	S1 adds tutor prompts; no existing prompt is changed.
Tenant + Identity API	/tenants, /auth/* backward-compat; device-binding endpoints are new (additive).
IndexedDB schema (client)	First-time schema install; no migration needed (clean install for pilots).

Tenant migration: N/A (M0 is internal only; M1 is first customer touch).

2.2 S1 → S2

Artifact	Compat requirement
PlayPackage manifest	S2 adds navigation.tree mode (additive); existing linear mode unchanged. Player tolerates unknown navigation modes by falling back to linear.
Block schema	New block kinds (text, image, video, audio, quiz ref, embed, interaction) added; none of S1's runtime blocks change shape.
Catalog API	New GET /courses + POST /courses endpoints; S1 enrollment + delivery still work against the same CourseVersion IDs.
Publish events	New saga events (authoring.course_draft.published.v1, content.play_package.built.v1, catalog.course_version.published.v1) arrive; S1 consumers (delivery, progress, search) tolerate new events they don't consume (ignored).
Offline bundles	S1 bundles remain mountable; S2 bundles have richer block content but same envelope + license + encryption scheme.

Tenant migration:

• Pilot tenants transition from operator-assisted content to self-serve authoring.
• Existing enrollments + progress are untouched; new courses created via authoring coexist.
• Feature flag authoring.self_serve toggled per tenant at CSM's discretion.

2.3 S2 → S3 (both within M2)

Artifact	Compat requirement
Marketplace API	New endpoints; no S2 API changes.
Billing events	New events; consumed only by marketplace + notification.
SCORM 1.2 export	New capability; existing HTML5 export unchanged.
Catalog	Marketplace listings surface alongside non-marketplace courses; visibility filter determines.
License → Enrollment	New enrollment source (purchase); existing sources (manual) unchanged.
Webhook API	New; additive.
Offline bundles	License envelope gains optional purchaseRef field (additive). Player tolerates unknown envelope fields.

Tenant migration:

• Existing tenants see marketplace as a new tab; no action required.
• Provider opt-in is self-serve; requires KYC.
• Pricing plans attached to existing courses via UI; courses without plans are not listed.

2.4 S3 → S4

Artifact	Compat requirement
Identity: SAML	New SSO provider configs; existing OIDC + email+password unchanged. SSO is additive per tenant.
Tenant: ABAC + DynamicGroup	New role capabilities; existing simple roles (org_admin, learner) continue to work as-is (equivalent to "full-access within scope" semantics, backward-compat).
Assignment + ComplianceWindow	New aggregates; no existing enrollments changed. Existing manual enrollments coexist; assignments create enrollments alongside them.
SCORM 2004 + xAPI	New export formats; SCORM 1.2 + HTML5 unchanged.
Content: SCORM import	New capability; existing content untouched.
Assessment: branching scenarios	New block type; existing quiz blocks unchanged.
Certification: templates + recert	New; existing basic certificates continue to verify. Templates default to system template for tenants that don't customize.
AI prompts	New prompts for AI Co-Author full, AI admin insight, AI grading. Existing tutor + summary + co-author MVP prompts pinned; deprecated only if accuracy eval shows improvement.
Offline bundles	Branching scenario data bundled; existing bundles without branching still work (player handles missing scenario gracefully).

Tenant migration:

• SAML activation per tenant; no forced migration off email+password.
• Roles: existing org_admin inherits all ABAC grants automatically; custom roles are opt-in.
• Assignments: opt-in feature; no forced assignment unless admin creates one.
• Certificate templates: system default auto-applied; custom override per tenant.
• AI prompts: new versions auto-selected for new calls; existing cached outputs unchanged.

2.5 S4 → S5

Artifact	Compat requirement
Authoring: Yjs	Live collab is additive; existing single-author editing continues as before (solo = single-participant collab session).
Offline authoring	New client capability; existing online-only editing unchanged. Feature flag authoring.offline per tenant.
Block registry	New block kinds (timeline, gallery, flashcards, etc.) additive; existing kinds unchanged.
Media: AI image + TTS	New capabilities; existing upload+transcode+caption unchanged.
Search: hybrid	Semantic results mixed with lexical; existing queries degrade gracefully if vector index not yet built for a tenant (lexical fallback).
LTI 1.3	New capability.
Localization	New locale variants on existing blocks; missing locale falls back to defaultLocale (existing behavior).
Offline bundles	New bundles may contain richer block types; older bundles mountable. Player shows "block not supported" for unknown kinds (forward-compat).

Tenant migration:

• Live collab: auto-available; no migration.
• Offline authoring: feature-flagged; rolled out per tenant.
• LTI 1.3: opt-in per tenant; requires LTI key config.
• Localization: existing single-locale courses are unaffected; adding a locale is additive.

2.6 S5 → S6

Artifact	Compat requirement
Data residency	Migration saga only runs when a tenant explicitly requests it; existing tenants stay in their home region.
HIPAA tier	Opt-in; existing tenants unaffected; new tenants can choose at creation.
White-label	Opt-in add-on; existing brand unaffected.
AI admin insight v2	Existing v1 dashboards replaced with v2 (same URL, richer content); no customer action needed.
Recommendations	New panels; home page adds recommendation rail; existing layouts unchanged for tenants that opt out.
Mobile native features	Update via app store; existing PWA behavior unchanged; new native features additive.
Developer SDK v1	New; existing API keys and webhooks continue to work via the same REST surface. SDK is a typed wrapper.
Marketplace scale	Multi-currency, coupon engine, gift licenses — additive; existing single-currency flows unchanged.

Tenant migration:

• Residency change: requires admin initiation + confirmation; saga orchestrated with freeze window (~minutes for median tenant).
• HIPAA: new tier at creation or upgrade; requires BAA + restricted provider routing.
• Mobile: app-store update standard; no data migration on device.

3. Offline Bundle Forward Compatibility

Offline bundles may remain mounted on devices for days or weeks. Every bundle carries:

• manifest.version (currently "1.0").
• licenseEnvelope.envVersion (currently 1).

The Player maintains a compatibility table:

Bundle manifest version	Player supports from	Behavior if unsupported
1.0	M1 forever	—
1.1 (if ever added)	M3+	Player at M1/M2 shows "update your app"

Rules:

• New optional fields in manifest or envelope: Player ignores unknowns. Additive-safe.
• New required fields: only in a new manifest version. Player at old version shows friendly update prompt.
• Block kind additions: Player renders "block not supported in this version" stub for unknown kinds. Content still navigable.
• Encryption algorithm change: versioned in encryption.alg; client supports current + N-1.
• Signature algorithm change: versioned in envelope sigAlg; client validates current + N-1.

Forced re-download trigger conditions:

• Bundle explicitly revoked by server.
• License envelope expired.
• Manifest version bump that client doesn't support (very rare).
• Media assets changed (server re-builds + issues new bundle ID).

4. AI Prompt Version Evolution

Prompts are semver-pinned per tenant in the prompt registry.

Rule	Detail
New prompt version	Append to registry; no existing version changed.
Promotion	Requires eval suite to pass.
Per-tenant pinning	Tenant auto-picks latest-active version unless admin pins a specific version.
Deprecation	A prompt version is deprecated when a newer version passes eval at higher score; deprecated prompts remain callable for ≥ 1 milestone (3 months).
Removal	Only after all tenants have moved off (tracked by usage metric); requires ADR.
Cross-version safety	Every version is independently safety-evaluated; safety verdicts are not inherited across versions.
Local-model prompts	Local prompt templates mirror cloud counterparts; version-tracked separately with localPromptVersion.

Migration path for tenants when a prompt is deprecated:

1. Admin sees "New version available" in AI settings with comparison eval.
2. Admin can test new version on a draft before switching.
3. If admin doesn't act within deprecation window, system auto-migrates to latest active version at next period reset.
4. Opt-out: admin may pin forever (prompt remains callable even after deprecation, until hard removal).

5. Event Schema Versioning

Full detail in 04 Event-Driven Architecture §10.

Summary for migration purposes:

Change	Action	Consumer impact
Add optional field	Publish within same vN; bump minor in schema registry	None — consumers ignore unknowns
Add required field	New vN+1; dual-publish vN + vN+1 for ≥ 1 release cycle	Consumers upgrade at their pace
Remove field	New vN+1; dual-publish; CI blocks vN removal until zero consumers	Consumers migrate off vN
Rename field	Treat as remove + add; new vN+1	Same as above
New event subject	Publish; no consumer impact	Consumers opt-in
Deprecate event	Continue publishing with deprecated: true metadata; alert consumers; remove after window	Consumers migrate

CI enforcement: Pact broker tracks which consumer consumes which subject × version; producer cannot stop publishing until all consumers move.

6. API Versioning

Full detail in 05 API Design §2.

Change	Action	Consumer impact
Add endpoint	Deploy; no version bump	None
Add optional field to response	Deploy; bump minor header	None — clients ignore unknowns
Add required field to request	New /api/v2/…; keep /api/v1/… for ≥ 1 release cycle	Clients migrate at their pace
Remove field from response	New /api/v2/…; dual-serve	Clients migrate
Change field type	New /api/v2/…	—
Remove endpoint	Deprecation header + Sunset; dual-serve; remove after window	Clients migrate

SDK impact: Developer SDK wraps the REST API; each SDK major version targets one API major version. SDK minor versions track additive API changes.

7. SCORM / xAPI / cmi5 Version Rules

These are industry standards; our export formats conform to specific standard versions.

Standard	Version shipped	Evolution
SCORM 1.2	1.2 (ADL conformant)	Frozen; standard does not evolve.
SCORM 2004	4th Edition	Frozen; if ADL revises, our export maintains backward-compat with 4th Ed.
xAPI	1.0.3	Additive (new verbs/extensions); breaking changes rare and ADL-driven.
cmi5	1.0	Standard-pinned.

Rule: any internal change that causes an export to fail ADL conformance tests is a Sev1 regression and blocks release.

8. IndexedDB / SQLite Schema Migration (Client)

Client-side databases evolve with the app. Migration strategy:

• Dexie (IndexedDB): schema versioned via Dexie .version(n) chaining; each new n includes an upgrade function.
• SQLite (mobile): migration files applied on app open; version tracked in _meta table.
• Rules:
  • New tables: additive.
  • New columns: added with defaults; migration runs on app start.
  • Column removals: never within one major app version; soft-removed (column stays, code ignores).
  • Table renames: never; add new + migrate data + deprecate old.
  • Data format changes inside a jsonb / data BLOB column: versioned by v key inside the JSON; reader supports current + N-1.

Rollback safety: downgrades are not supported on client (app updates are one-way). If a schema migration causes data loss, the client drops to "clean-pull" mode: wipes local store, re-pulls from server via sync. Outbox items that haven't been pushed are preserved in a separate recovery table and offered for manual re-push.

9. Tenant Migration Runbook Template

When any structural change affects existing tenants:

1. Pre-flight: identify affected tenants via analytics query.
2. Communication: in-app banner + email 14 days before change.
3. Opt-in window: tenants activate on their schedule within the overlap window.
4. Auto-migration: at window end, remaining tenants auto-migrated during a low-traffic slot.
5. Verification: per-tenant smoke test post-migration (automated).
6. Rollback: documented per change; typically "revert feature flag + restore from snapshot."
7. Audit: migration log retained per tenant; available in compliance export.

10. Summary Compatibility Matrix

Artifact	Evolves how	Migration burden on customer
REST APIs	Additive within major; breaking requires new major + overlap	SDK + webhook consumers update at their pace
Events	Additive within vN; breaking → vN+1 dual-publish	Internal; no customer migration
Sync protocol	/sync/v1/ long-lived	Client app update
Offline bundles	Manifest + envelope versioned; additive-safe	Re-download only when forced by revocation or major version
AI prompts	Semver; per-tenant pinning; deprecation + auto-migrate	Admin action or passive auto-upgrade
SCORM/xAPI	ADL-standard-pinned	None
Client DB schema	Dexie/SQLite migration on app start	Automatic; invisible to user
Pricing/entitlements	Tier additions additive; feature-flag-gated	Self-serve upgrade or auto-entitlement per plan
Security (auth, MFA)	Additive; enforcement per tenant policy	Admin toggles enforcement

11. Governance

• Every breaking change requires ADR + architecture-sync sign-off.
• Every breaking change has a customer-facing "migration guide" attached to the release notes.
• Every migration is tested in staging with a clone of production tenant data before rollout.
• Every migration has a rollback plan documented and tested.
• The migration-backward-compatibility doc is maintained as a living document; each slice appends its migration notes before release.