Platform topology
Get oriented before opening any single surface. Who calls in, what depends on what, on what wire, with what auth.
System context
The services
| Service | Stack | Role |
|---|---|---|
| Control plane | NestJS / TypeScript | API, governance, policy, identity, RBAC, ledger, jobs |
| Billing service | NestJS / TypeScript | Stripe webhooks, entitlements, feature flags |
| Inference router | FastAPI / Python (CPU) | Runpod spawn-and-forward |
| Inference service | FastAPI / Python (GPU) | Guardian + Actor, multi-LoRA, deterministic kernel |
| Training orchestration | FastAPI / Python (CPU) | Job dispatch |
| Training GPU | FastAPI / Python (GPU) | Spot-aware LoRA fine-tuning |
The control plane is the always-on CPU surface (auth, RBAC, ledger, Guardian lifecycle). The inference service is the on-demand / HA GPU surface (Guardian inference, LoRA hot-swap, batching). Training is the as-needed GPU surface (LoRA fine-tuning, TDG, densification).
External dependencies
- Relational DB — Postgres / MSSQL / SQLite (per tier).
- LLM providers — OpenAI / Anthropic / Azure / custom, reached via proxied calls.
- Runpod — dynamic GPU cloud for inference pods.
- Azure Blob — model and adapter storage.
- SIEM — Splunk / Datadog / CloudWatch for audit and governance log export.
Three planes, one kernel
The control plane orchestrates; the governance (inference) plane enforces; the training plane produces. All three share the deterministic SGLang kernel, the same identity model, and the same Glass Box Ledger.
→ Request lifecycle — the canonical path through the control plane. → Architecture — the Guardian itself. → Self-hosting — deploying these services in your own VPC.