Skip to main content

DEPLOYMENT_TOPOLOGY — reporting-service

Sibling: OBSERVABILITY · SECURITY_MODEL · LOCAL_DEV_SETUP · platform anchor: docs/02 §7 Deployment


1. Runtime

  • Container image: Node 20 LTS Alpine + headless Chromium (pinned digest), ~480 MB compressed.
  • Compute: Cloud Run (request-driven API) + Cloud Run Jobs for scheduled batch and worker pool.
  • Orchestration: Cloud Deploy → Cloud Run revisions; Skaffold for dev.
  • Two services per region:
    • reporting-api — public/internal HTTP surface, min 2, max 50 instances, concurrency 80, 1 vCPU / 1 GiB.
    • reporting-worker — Pub/Sub-driven render pool, min 2, max 50 instances, concurrency 1, 2 vCPU / 4 GiB (renderer needs more memory and a fresh Chromium per task).
  • Cron triggers: Cloud Scheduler → HTTPS POST /internal/scheduler/fire on reporting-api.
  • CDN: Cloud CDN fronts only artifact downloads, with signed URLs (no caching of authenticated responses).

2. Topology diagram

[BFF]──HTTPS──┐

reporting-api (Cloud Run, regional, min 2)

├── Cloud SQL (private IP, regional, HA)
├── Memorystore Redis (regional)
├── Pub/Sub (publish: reporting.* topics; subscribe: tenant.deleted, schedule.fired internal)
├── Secret Manager (regulatory adapter creds, KMS DEK refs)
├── Cloud KMS (CMEK + field-level DEK)
└── ai-orchestrator-service (mTLS internal)

[Cloud Scheduler]──HTTPS(OIDC)──▶ reporting-api /internal/scheduler/fire

└─▶ Pub/Sub: melmastoon.reporting.run.dispatch (internal)


reporting-worker (Cloud Run, push-subscribed, min 2)

├── analytics-service Query API (read)
├── BigQuery (analytics_curated.*) read-only via SA
├── GCS (artifacts bucket + regulatory bucket)
├── notification-service (publish report.delivered chain)
├── ai-orchestrator-service (callouts)
└── regulatory adapters (egress allow-list)

3. Regions & residency

Per platform standard (tenant.dataResidency), each residency has an isolated stack:

ResidencyRegionCloud SQLGCS buckets
AFme-central1melmastoon-afmelmastoon-reports-af, melmastoon-reports-regulatory-af
INasia-south1melmastoon-inmelmastoon-reports-in, melmastoon-reports-regulatory-in
KSAme-central2melmastoon-ksamelmastoon-reports-ksa, melmastoon-reports-regulatory-ksa
EUeurope-west4melmastoon-eumelmastoon-reports-eu, melmastoon-reports-regulatory-eu

Cross-region replication is forbidden for regulatory_10y_objectlock artifacts.


4. Identity, networking, secrets

  • Service accounts:
    • reporting-api@<proj>.iam.gserviceaccount.com — roles: roles/cloudsql.client, roles/pubsub.publisher (own topics), roles/secretmanager.secretAccessor (scoped resources), roles/cloudkms.cryptoKeyEncrypterDecrypter on melmastoon-reporting.
    • reporting-worker@<proj>.iam.gserviceaccount.com — adds roles/storage.objectCreator on artifacts buckets, roles/storage.objectViewer on regulatory bucket (no delete), roles/bigquery.dataViewer on analytics_curated.*.
  • Workload Identity binds Kubernetes ↔ GCP SA where applicable; Cloud Run uses GSA directly.
  • Networking: Direct VPC egress via Serverless VPC Access connector for Cloud SQL private IP and outbound regulatory adapter allow-list.
  • Egress allow-list: Cloud NAT with static IPs for regulatory submissions; allow-list maintained per jurisdiction.
  • Ingress: internal-only for /internal/* paths via Cloud Run "internal-and-cloud-load-balancing" mode + IAM-bound BFF service accounts.

5. Configuration

12-factor; values from env vars sourced from Secret Manager and gcloud run services update. Key vars (non-secret):

PORT=8080
NODE_ENV=production
LOG_LEVEL=info
GCP_PROJECT=…
GCP_REGION=…
PG_HOST=… (private IP)
PG_DB=melmastoon
PG_SCHEMA=reporting
PUBSUB_TOPIC_RUN_DISPATCH=projects/$PROJECT/topics/melmastoon.reporting.run.dispatch
GCS_REPORTS_BUCKET=melmastoon-reports-$REGION
GCS_REGULATORY_BUCKET=melmastoon-reports-regulatory-$REGION
KMS_KEY_DEK=projects/$PROJECT/locations/$REGION/keyRings/melmastoon-reporting/cryptoKeys/subscription-pii
RENDERER_MAX_CONCURRENCY=4
RENDERER_PAGE_TIMEOUT_MS=30000
AI_ORCHESTRATOR_BASE=https://ai-orchestrator.…internal
ANALYTICS_BASE=https://analytics.…internal

Secrets referenced by env name only; the runtime resolves them through the secretmanager.googleapis.com client at startup and on a 5-min refresh.


6. Deploy pipeline

GitHub Actions → Cloud Build → Artifact Registry → Cloud Deploy. Stages:

  1. build — multi-arch image, push to Artifact Registry with immutable tag = git SHA.
  2. migrate — runs pnpm db:migrate against the target Cloud SQL using a one-shot Cloud Run Job; fails the rollout if a migration error occurs. Migrations are forward-only and additive; destructive changes use a 2-phase plan (MIGRATION_PLAN).
  3. canary — 10 % traffic for 15 min; SLO burn alarm wired in.
  4. promote — 100 % traffic; previous revision retained for 24 h for instant rollback.
  5. smoke — synthetic canary report run; fail rollback if status ≠ completed.

7. Capacity & cost envelope

ResourceSteady statePeak
reporting-api2-4 pods30 pods
reporting-worker4-8 pods50 pods
Cloud SQL4 vCPU / 16 GiB regional HAautoscale storage
Memorystore2 GiB tier5 GiB
GCS egress< 100 GB/day500 GB on month-end peak
BigQuery slotsshared on analytics-service reservation; max 200 slots/query for reporting reads

Budget alerts at 80 % of the monthly forecast send a notification to the platform-finance channel.

Cross-references: DATA_MODEL §7 backups, OBSERVABILITY, docs/02 §7.