Fraud Intelligence Service — AI Integration

Version: 1.0 Status: Draft Owner: Trust and Safety Last Updated: 2026-04-21 Companion: DOMAIN_MODEL · APPLICATION_LOGIC · SECURITY_MODEL · docs/architecture/ADR-0004 §3

---

1. Purpose

The fraud-intel-service is the only ML-heavy service in the platform. It runs four model families end-to-end on-cluster (training and inference), each tuned to a specific fraud category. Models are versioned, signed, hot-reloaded, and explainable (SHAP/feature-attribution). No cloud LLM is ever invoked on raw subscriber data; text-classification components run on local Triton-served transformers.

The design constraints are non-negotiable:

1. Sub-100 ms hot-path inference for the synchronous Score gRPC.
2. Regulator-grade explainability — every detection event carries aiProvenance with model ID, version, training-set hash, and SHAP top-3 contributing features.
3. PII never leaves the cluster. Text-classification models are local. SMS body content is anonymised before inference even on local models.
4. Per-tenant fairness audit. Detection precision must not vary by > 0.10 across tenant cohorts (small-volume vs large-volume; bank/gov vs marketing).
5. Continuous improvement via HITL. Trust & Safety analyst decisions on FraudCase are the labelled training corpus for the next nightly retrain.

---

2. Provider Strategy — Local-Only

Provider	Use	Status
Triton Inference Server (NVIDIA)	Primary serving for XGBoost (FIL backend), Isolation Forest (FIL), GraphSAGE (PyTorch), TransformerText (TensorRT-LLM backend)	Required in prod, staging, and CI
vLLM (legacy from compliance-engine prototype)	Optional secondary text classifier	Disabled in fraud-intel-service
Cloud LLM (Anthropic/OpenAI)	n/a	Disallowed. Fraud signal data + MSISDNs may not be sent off-cluster
Mock provider	Dev/CI	Deterministic pre-canned predictions

Provider selection is fixed at deploy time via INFERENCE_PROVIDER=triton|mock. There is no runtime failover to cloud — a Triton outage triggers the rule-based pattern matcher fallback (see §10).

---

3. Model Architecture

3.1 AIT Detection — XGBoost (gradient-boosted trees)

Aspect	Value
Why XGBoost	Tabular features (counts, ratios, entropies); strong baseline; SHAP-explainable; Triton FIL backend has sub-millisecond inference
Inputs	12 features per (tenant_id, dst_mno, sender_id, 5min) from fraud_features.ait_window_features
Output	Calibrated probability `P(AIT
Calibration	Platt scaling on a held-out 10% set; Brier score ≤ 0.10
Hyperparameters	max_depth=6, n_estimators=400, learning_rate=0.05, subsample=0.85, colsample_bytree=0.7, tree_method=hist
Training set	Last 90 d of fraud.signals ∪ confirmed fraud.cases.decisions; positive class oversampled 4× via SMOTE-NC
Test set	Frozen quarterly canon: fraud-test-corpus-2026q2
Acceptance metrics	AUC ≥ 0.92, FPR ≤ 0.5% at 0.85 threshold, recall ≥ 0.85, per-tenant fairness Δ ≤ 0.10
Explainability	TreeSHAP top-3 features per prediction, persisted in fraud_features.ait_predictions.shap_top3
Inference latency	P99 ≤ 5 ms per row on Triton FIL CPU backend

3.2 AIT Cohort Anomaly — GraphSAGE (graph neural network)

Aspect	Value
Why GraphSAGE	Cross-tenant rings form bipartite (tenant ⇄ recipient-MSISDN-hash) graphs; GraphSAGE inductively scores new nodes
Inputs	Bipartite graph snapshot per 1-h window; node features: tenant (messages_24h, sender_id_count, dlr_success_rate), MSISDN (distinct_tenants_targeting, distinct_sender_ids_received, otp_share)
Output	Per-cohort cohort_anomaly_score ∈ [0,1]
Architecture	2 SAGE convolution layers, hidden_dim=64, mean aggregator, dropout=0.2; output MLP head 64→32→1 sigmoid
Framework	PyTorch 2.x; ONNX export for Triton PyTorch backend
Training	Weekly retrain on rolling 30-d graph; positive labels from case_decisions.decision = CONFIRM_FRAUD AND category = AIT_RING
Acceptance metrics	AUC ≥ 0.88, recall on adversarial test corpus ≥ 0.80
Inference latency	P95 ≤ 50 ms per cohort batch on a Triton GPU pod (T4); batch up to 512 cohorts

3.3 SIM-Box Detection — Isolation Forest (outlier detection)

Aspect	Value
Why iForest	SIM-box patterns are anomalies in low-dimensional MSISDN-block feature space; iForest is unsupervised and resistant to label scarcity
Inputs	5 features per /28 block: msisdn_range_density, body_template_hash_concentration, hlr_mismatch_rate, imsi_unique_count, mno_bind_concentration
Output	Anomaly score [0,1] (calibrated)
Hyperparameters	n_estimators=200, max_samples='auto', contamination=0.005, random_state=42
Ensemble	Combined with rule-based predicate (density>0.6 AND template>0.4 AND hlr_mismatch>0.3); both must agree to emit
Training	Quarterly retrain on rolling 90-d feature snapshots; outliers post-confirmed via case decisions become positive labels for a follow-up XGBoost classifier (planned 2027-Q1)
Acceptance metrics	Precision ≥ 0.92, recall ≥ 0.80 on labelled subset
Inference latency	P99 ≤ 3 ms per row on Triton FIL CPU

3.4 OTP-Harvest / Grey-Route — XGBoost (per-category models)

Same XGBoost architecture as AIT, with category-specific feature vectors and acceptance thresholds:

Category	Features	Acceptance
OTP_HARVEST	otp_count, revocation_count, revocation_rate, cohort_size, tenant_age_days, sender_class	AUC ≥ 0.90, precision ≥ 0.90
GREY_ROUTE	mt_to_non_peered_ratio, total_mt, hlr_mismatch_rate, peer_age_days, dlr_anomaly	AUC ≥ 0.90, precision ≥ 0.92

3.5 OTP-Pattern Text Classifier — Local Transformer (optional v2.1+)

For multilingual OTP-keyword detection in OTP-harvest tagging (UC-12), v1 uses regex; v2.1 introduces a small distilled transformer:

Aspect	Value
Model	xlm-roberta-base distilled to 6 layers (~70 M params)
Languages	English, Pashto, Dari, Arabic, Urdu (script-shared with Pashto/Dari)
Serving	Triton tensorrt_llm backend on T4 GPU; INT8 quantisation
Inference latency	P99 ≤ 30 ms per body
PII handling	Body anonymised pre-inference (digits → #, names → [NAME], URLs → [URL]) per §6
Acceptance	F1 ≥ 0.95 on labelled multilingual OTP-keyword corpus

---

4. Triton Serving Topology

┌──────────────────────┐          ┌────────────────────────┐
│ fraud-intel-worker   │ ───────► │   triton-fraud-cpu     │
│  (Python pipelines)  │  gRPC    │   (XGBoost FIL,        │
│                      │  ModelInfer  iForest FIL)         │
│                      │          │   3-6 replicas, CPU    │
│                      │          └────────────────────────┘
│                      │
│                      │          ┌────────────────────────┐
│                      │ ───────► │   triton-fraud-gpu     │
│                      │  gRPC    │   (GraphSAGE PyTorch,  │
│                      │          │    XLM-R Transformer)  │
│                      │          │   2-3 replicas, T4 GPU │
│                      │          └────────────────────────┘
└──────────────────────┘
        │
        ▼
   model registry (Postgres) + artifacts (MinIO s3://fraud-models/)

Model repository layout (mounted as PVC on Triton):

/models/
├── ait_xgboost/
│   ├── 1/
│   │   └── xgboost.model           # FIL-loadable
│   ├── 2/
│   │   └── xgboost.model
│   └── config.pbtxt                # Triton model config
├── cohort_graphsage/
│   ├── 1/model.onnx
│   └── config.pbtxt
├── simbox_iforest/
│   ├── 1/model.bin
│   └── config.pbtxt
└── otp_keyword_xlmr/
    ├── 1/model.plan                # TensorRT engine
    └── config.pbtxt

config.pbtxt excerpt (XGBoost FIL):

name: "ait_xgboost"
backend: "fil"
max_batch_size: 8192
input [{ name: "input__0" data_type: TYPE_FP32 dims: [12] }]
output [{ name: "output__0" data_type: TYPE_FP32 dims: [1] }]
parameters [
  { key: "model_type" value: { string_value: "xgboost" } },
  { key: "predict_proba" value: { string_value: "true" } }
]
instance_group [{ count: 4 kind: KIND_CPU }]

Hot-reload. Triton's model repository polling (--model-control-mode=poll, --repository-poll-secs=30) detects new versions in MinIO via PVC sync. Model promotion (UC-12) updates the Postgres registry; the worker pulls the new artifact, places it under /models/<model>/<new_version>/, and Triton picks it up within 30 s. No worker restart.

---

5. Feature Store

The fraud_features.* ClickHouse schema (see DATA_MODEL §2) is the single source of truth for both training and inference. Both paths use the same Python feature-engineering module (fraud-features package) so there is no train-serve skew.

# fraud_features/transformers.py — used by both training and inference
class AitFeatureTransformer:
    FEATURE_NAMES = [
        'submit_count', 'dlr_delivered_count', 'dlr_failed_count',
        'dlr_success_rate', 'unique_dst_msisdns', 'mean_segments_per_msg',
        'entropy_of_dst_prefix', 'unique_sender_ids', 'repeated_body_ratio',
        'peer_asn_diversity', 'cohort_anomaly_score', 'tenant_age_days',
    ]

    def fit_transform(self, df: pd.DataFrame) -> np.ndarray: ...
    def transform(self, df: pd.DataFrame) -> np.ndarray: ...

    @property
    def feature_set_hash(self) -> str:
        return sha256(','.join(sorted(self.FEATURE_NAMES)).encode()).hexdigest()

feature_set_hash is persisted on every ModelVersion and validated at inference: a mismatch refuses the load and emits fraud.alert.feature_skew.v1.

---

6. PII Anonymisation Pre-Inference

Even on local Triton-served transformers, body and MSISDN anonymisation runs as defence-in-depth.

Pattern	Replacement
E.164 phone numbers (any country)	[PHONE]
5+ contiguous digits (OTPs, account numbers)	[NUMERIC]
Monetary amounts (USD 100, $50, 100 AFN, 100 ؋)	[AMOUNT]
URLs	[URL] (presence preserved for phishing detection)
Common Pashto/Dari/English first names (curated 10K list)	[NAME]
MSISDNs in feature events	sha256(msisdn + nationalSalt)

Anonymisation is mandatory (ANONYMIZE_BEFORE_INFERENCE=true is enforced; setting to false refuses pod start in non-dev environments).

---

7. Sample Feature Vector (AIT)

{
  "modelId": "ml_ait_xgboost",
  "modelVersion": "2.1.4",
  "featureSetHash": "8f2c1c07a3...",
  "windowStart": "2026-04-21T10:00:00Z",
  "windowEnd":   "2026-04-21T10:05:00Z",
  "subjectScope": "TENANT",
  "subjectId":    "tnt_acme_marketing",
  "features": {
    "submit_count":           42100,
    "dlr_delivered_count":     7600,
    "dlr_failed_count":       34500,
    "dlr_success_rate":          0.18,
    "unique_dst_msisdns":     39822,
    "mean_segments_per_msg":     1.0,
    "entropy_of_dst_prefix":     1.92,
    "unique_sender_ids":          1,
    "repeated_body_ratio":       0.97,
    "peer_asn_diversity":         3,
    "cohort_anomaly_score":      0.81,
    "tenant_age_days":            7
  },
  "prediction": {
    "score": 0.94,
    "shap_top3": [
      { "feature": "dlr_success_rate",      "value": 0.18, "contribution": -0.42 },
      { "feature": "cohort_anomaly_score",  "value": 0.81, "contribution": +0.31 },
      { "feature": "tenant_age_days",       "value": 7,    "contribution": +0.24 }
    ],
    "runtimeMs": 3.7
  }
}

The shap_top3 is the regulator-defensible explanation: a tenant 7 days old, 18% DLR success, in a high-anomaly cohort is the smoking gun.

---

8. Training Pipeline (Airflow DAG)

# dags/fraud_train_ait.py — runs nightly 02:00 Asia/Kabul

with DAG('fraud_train_ait', schedule='0 22 * * *', tz='UTC') as dag:
    snapshot       = ClickHouseSnapshotOperator(task_id='snapshot',     window='90d')
    label_join     = JoinHitlLabelsOperator    (task_id='join_labels',  window='90d')
    feature_engr   = FeatureTransformOperator  (task_id='features',     transformer='AitFeatureTransformer')
    train          = XgboostTrainOperator      (task_id='train',        params=AIT_PARAMS)
    eval_holdout   = EvaluateOperator          (task_id='eval_holdout', set='holdout_20')
    eval_canon     = EvaluateOperator          (task_id='eval_canon',   set='fraud-test-corpus-2026q2')
    eval_adv       = EvaluateOperator          (task_id='eval_adv',     set='adversarial-corpus-v3')
    fairness       = FairnessAuditOperator     (task_id='fairness',     cohorts=['bank','gov','sme','marketing'])
    model_card     = ModelCardOperator         (task_id='model_card')
    register       = ModelRegisterOperator     (task_id='register',     api_url=FRAUD_INTEL_API_URL)

    snapshot >> label_join >> feature_engr >> train >> [eval_holdout, eval_canon, eval_adv, fairness] >> model_card >> register

Pre-conditions for register: holdout AUC ≥ 0.92, canon AUC ≥ 0.90, adversarial recall ≥ 0.80, max per-cohort fairness Δ ≤ 0.10. Any fail → DAG marks REJECTED and alerts.

---

9. Model Card (YAML)

# Generated and stored at s3://fraud-models/<modelId>/<version>/model_card.yaml
model_details:
  name: ait_xgboost
  version: 2.1.4
  pipeline: XGBOOST
  category: AIT
  license: Internal-Ghasi
  created: 2026-04-15
  authors: [trust-and-safety@ghasi.af]

intended_use:
  primary_use: Detect Artificially Inflated Traffic (AIT) campaigns on outbound SMS aggregator traffic
  out_of_scope:
    - Single-message spam classification (use compliance-engine AI rules)
    - Inbound MO SIM-box detection (use simbox_iforest)
    - Subscriber-level fraud (subscriber-protection scope)

training_data:
  source: ClickHouse fraud_features.events 2026-01-15 to 2026-04-15
  size: 412_703_891 rows
  positive_class_count: 11_204
  positive_class_source: confirmed fraud.cases.case_decisions.decision='CONFIRM_FRAUD'
  oversampling: SMOTE-NC 4x positive
  training_set_hash: a3f2c19d...

evaluation:
  holdout:
    auc: 0.937
    precision: 0.945
    recall:    0.881
    f1:        0.911
    fpr_at_threshold_0.85: 0.0034
    brier:     0.082
  fairness:
    cohort_deltas:
      bank:      { auc_delta: -0.012 }
      gov:       { auc_delta: -0.008 }
      sme:       { auc_delta: +0.004 }
      marketing: { auc_delta: +0.016 }
  adversarial:
    test_corpus: adversarial-corpus-v3
    recall: 0.84

ethical_considerations:
  - Model relies on aggregate features only; no SMS content
  - MSISDN values pseudonymised before training
  - Bank/Gov sender-IDs are allowlisted to prevent service-class bias

limitations:
  - Cold-start tenants (< 7d activity) score as PROBATION; not modelled here
  - Model assumes 5-min window; bursts shorter than this evade detection (mitigated by streaming detector)
  - Coordinated cross-cohort attacks may evade individual scoring (mitigated by graph cohort job)

---

10. Drift Detection & Fallback

10.1 Drift detection

Daily job computes:

• Feature drift: Population Stability Index (PSI) per feature against the training reference. PSI > 0.25 on any single feature → FraudFeatureDriftHigh MEDIUM alert. PSI > 0.50 → HIGH.
• Prediction drift: Wasserstein distance between today's score distribution and rolling 7-d baseline. > 0.15 → MEDIUM alert.
• Calibration drift: Brier score on the last 7 d of confirmed/dismissed cases. Brier > 0.15 → HIGH alert + automatic shadow-mode re-evaluation of latest training run.

10.2 Rule-based fallback

When Triton is unavailable or drift is critical, pipelines fall back to deterministic rule-based pattern matchers (FraudPattern rows):

SELECT * FROM fraud.patterns
WHERE category = 'AIT'
  AND is_active = TRUE
ORDER BY priority;

Each pattern is a JSONB predicate evaluated in-process. Confidence is a static value per pattern (typically 0.85 for high-precision patterns). The fallback is intentionally lower recall, higher precision — it catches obvious cases while ML is unavailable.

---

11. HITL Feedback Loop

fraud.case.opened.v1 ──► T&S analyst reviews in admin-dashboard
                          │
                          ▼
                 POST /v1/fraud/cases/{caseId}/decide
                 { decision, reason, executeAction? }
                          │
                          ▼
                 fraud.case_decisions row + fraud.case.decided.v1
                          │
                          ▼
       ┌──── nightly Airflow DAG reads last 30 d decisions ────┐
       │                                                        │
       ▼                                                        ▼
   training set update                              fairness audit (cohort delta check)
       │
       ▼
   shadow run ≥ 24h
       │
       ▼
   POST /v1/admin/fraud/models/{id}/promote
   (atomic swap; previous → RETIRED; new → ACTIVE)
       │
       ▼
   fraud.model.promoted.v1 → workers hot-reload on next batch boundary

A decision = REFINE_FEATURES includes featureCorrections JSON; this is treated as a schema-change request rather than a label, queued for the data-science team's review before incorporation.

---

12. AiProvenance Touch Points

Every fraud.detected.* event and every fraud.detections row carries a complete aiProvenance block:

{
  "modelId": "ml_ait_xgboost",
  "modelVersion": "2.1.4",
  "pipeline": "XGBOOST",
  "trainingSetHash": "a3f2c19d...",
  "featureSetHash":  "8f2c1c07...",
  "shapTop3": [
    { "feature": "dlr_success_rate",     "value": 0.18, "contribution": -0.42 },
    { "feature": "cohort_anomaly_score", "value": 0.81, "contribution": +0.31 },
    { "feature": "tenant_age_days",      "value": 7,    "contribution": +0.24 }
  ],
  "runtimeMs": 3.7
}

Where the value comes from:

Field	Source	Why
modelId, modelVersion	fraud.models + fraud.model_versions	Identifies the exact deployed artifact
pipeline	fraud.models.pipeline	XGBoost / GraphSAGE / iForest / Transformer
trainingSetHash	fraud.model_versions.training_set_hash	Reproducibility — exact training rows
featureSetHash	fraud.model_versions.feature_set_hash	Detect train-serve skew
shapTop3	TreeSHAP (XGBoost), DeepLIFT (GraphSAGE), feature-importance (iForest)	Regulator-defensible "why"
runtimeMs	Triton response timing	Per-call latency budget

Regulator dispute resolution flow: an aggrieved tenant disputes an AIT detection → audit retrieves aiProvenance → data-science reproduces the prediction by loading model_versions.artifact_uri (verified by SHA-256), feeding the persisted feature vector → identical score → defensible.

---

13. Cost Model

Component	Throughput	Cost (illustrative monthly)
Triton CPU pool (XGBoost + iForest) — 4 pods × 8 vCPU	50K predictions/s aggregate	~$1,200
Triton GPU pool (GraphSAGE + Transformer) — 2 pods × T4	500 cohorts/s, 200 bodies/s	~$1,400
Airflow training cluster — 8 vCPU + 32 Gi spot	1 nightly run × 6 h	~$200
MinIO model artifacts (cross-region)	~5 Gi total	~$50
ClickHouse cluster (3×r6i.4xlarge)	10 M events/h	~$2,800
Total ML serving + training infrastructure		~$5,650 / month

Notable: zero per-message LLM API cost — everything is CapEx-amortised infrastructure.

---

14. Future Roadmap

Enhancement	Rationale	Timeline
Fine-tuned SMS-domain transformer for OTP-text classification	Reduce dependency on regex for novel OTP-keyword variants	2026-Q4
Active learning: surface highest-uncertainty cases first to analysts	Maximises labelling efficiency	2027-Q1
Federated learning across MNO peers (if regulator permits)	Cross-MNO AIT detection without sharing raw signals	2027-Q3
Joint model: sender-reputation + content + behaviour	Reduce false positives on legitimate high-volume senders	2027-Q4
GPU autoscaling via NVIDIA Time-Slicing	Lower idle GPU cost at off-peak	2026-Q3