LOCAL_DEV_SETUP — analytics-service

Sibling: APPLICATION_LOGIC · DATA_MODEL · TESTING_STRATEGY

Bring analytics-service up locally with all dependencies stubbed or emulated. Targets developer time-to-first-widget < 15 min.

1. Prerequisites

Tool	Version
Node.js	20 LTS (`nvm use 20`)
pnpm	≥ 9.x
Docker / Docker Desktop	latest
Make (`make`)	any
`gcloud` CLI	latest (used only for `pubsub` emulator + sandbox BQ)
`bq` CLI	latest (optional, for sandbox queries)
Java 17	required by Pub/Sub & BigQuery emulators

2. Clone & install

git clone git@github.com:ghasi/melmastoon.git
cd melmastoon
pnpm install
cd services/analytics-service
cp .env.example .env.local   # fill blanks (see §4)

3. Local stack

docker-compose.dev.yml boots:

Service	Port	Purpose
`postgres` (Postgres 16)	5432	metadata + RLS
`pubsub-emulator` (gcloud)	8085	Pub/Sub topics & subscriptions
`bigquery-emulator` (`ghcr.io/goccy/bigquery-emulator`)	9050	curated + raw datasets
`redis`	6379	cache + idempotency store
`otel-collector`	4317 / 4318	traces/metrics/logs export
`signoz` (optional, `--profile observability`)	3301	local SigNoz UI

make dev:up        # boots stack
make dev:seed      # creates topics, subscriptions, BQ datasets, Postgres schema, seed data
make dev:logs      # tails compose logs
make dev:down      # stop & remove

4. Environment

.env.local (extract):

NODE_ENV=development
SERVICE_NAME=analytics-service
DATABASE_URL=postgres://analytics:analytics@localhost:5432/melmastoon
PUBSUB_EMULATOR_HOST=localhost:8085
PUBSUB_PROJECT=local
BIGQUERY_EMULATOR_HOST=http://localhost:9050
BIGQUERY_PROJECT=local
BIGQUERY_LOCATION=US
BIGQUERY_CURATED_DATASET=analytics_curated
BIGQUERY_RAW_DATASET=events_raw
DEFAULT_QUERY_BYTE_CAP=104857600   # 100 MiB locally
DEFAULT_TENANT_DAILY_BUDGET=1073741824
AI_ORCHESTRATOR_BASE_URL=http://localhost:7301   # local stub
AI_ORCHESTRATOR_AUDIENCE=http://localhost:7301
LOOKER_EMBED_KMS_KEY=local:dev-signing-key       # symmetric stub
JWT_DEV_SHARED_SECRET=dev-only-not-for-prod

Stubs:

ai-orchestrator-service → tools/stubs/ai-orchestrator/ (in-process Express stub returning canned metric_explainer and forecast.produced.v1 payloads).
iam-service → dev JWT minter at tools/stubs/iam/mint.ts.
tenant-service → seeded directly into Postgres + BigQuery dataset.

5. Run the service

pnpm dev                    # nest watch mode, port 7311
pnpm dev:sink               # Pub/Sub sink subscriber
pnpm dev:etl-once -- --job=metric.occupancy.daily   # run an ETL once
pnpm dev:looker             # mint a local embed token for tnt_dev_001

Quick smoke:

TOKEN=$(pnpm --silent jwt:mint -- --tenantId=tnt_dev_001 --userId=usr_dev_admin --roles=tenant.admin)

curl -s http://localhost:7311/api/v1/analytics/metrics \
  -H "Authorization: Bearer $TOKEN" -H "X-Tenant-Id: tnt_dev_001" | jq

curl -s -X POST http://localhost:7311/api/v1/analytics/queries:run \
  -H "Authorization: Bearer $TOKEN" -H "X-Tenant-Id: tnt_dev_001" \
  -H "Content-Type: application/json" \
  -d '{
    "sourceTables":["fact_reservation_v1"],
    "sqlTemplate":"SELECT business_date, COUNT(*) AS bookings FROM `local.analytics_curated.fact_reservation_v1` WHERE tenant_id = @tenant_id GROUP BY business_date ORDER BY business_date DESC LIMIT 7",
    "params":{}
  }' | jq

6. Triggering ETL & DQ locally

pnpm dev:scheduler:tick -- --jobId=etl.occupancy.daily
pnpm dev:dq:run -- --checkId=dqc_freshness_fact_reservation
pnpm dev:forecast:write -- --fixture=test/fixtures/forecast/baseline.json

Inspect:

bq --api=http://localhost:9050 --project_id=local query \
  "SELECT * FROM analytics_curated.fact_reservation_v1 LIMIT 10"

bq --api=http://localhost:9050 --project_id=local query \
  "SELECT * FROM dq_results.dq_runs ORDER BY checked_at DESC LIMIT 10"

7. Useful scripts

Script	Purpose
`pnpm sql:lint`	Custom lint for tenant_id presence in templates
`pnpm openapi:gen`	Regenerate OpenAPI spec from controllers
`pnpm contracts:check`	Validate published-event fixtures
`pnpm migrate:dev`	Apply Postgres migrations
`pnpm seed:demo`	Seed dashboards, widgets, sample curated data

8. Debugging tips

LOG_LEVEL=debug shows every BigQuery call template + bytes_processed (no full SQL leaks because we log templates only).
Use SigNoz local profile to see end-to-end traces from controller → BigQuery emulator → response.
Pub/Sub emulator does not enforce OIDC — strict mode disabled in dev; do not rely on local behavior for auth.
BigQuery emulator does not enforce CMEK or authorized views; tenant isolation tests run against the sandbox BQ project, not the emulator.

9. Common pitfalls

Forgetting X-Tenant-Id header → 400 MELMASTOON.IAM.TENANT_HEADER_MISSING.
Dataset region mismatch (emulator uses US); use BIGQUERY_LOCATION=US locally.
Long-running ETL fixture exceeds 100 MiB cap → bump DEFAULT_QUERY_BYTE_CAP for the run only.
Dashboards seeded for tnt_dev_001; minting a token with another tenant returns empty list (correct behavior).

10. Resetting

make dev:reset    # wipes Postgres data, recreates BigQuery datasets, drains Pub/Sub topics

Cross-references: TESTING_STRATEGY, API_CONTRACTS, DATA_MODEL.

1. Prerequisites​

2. Clone & install​

3. Local stack​

4. Environment​

5. Run the service​

6. Triggering ETL & DQ locally​

7. Useful scripts​

8. Debugging tips​

9. Common pitfalls​

10. Resetting​