Inference Service API

Service: services/inference-service (Python / FastAPI, GPU). Reached by the control-plane HttpInferenceAdapter with an X-Internal-Token — not a public, tenant-authenticated surface; tenant auth happens at the control-plane. See Authentication for the public contract. Determinism: every /v1/govern response carries a deterministic_receipt block.

The GPU-side endpoints not already covered by the control-plane Chat & Proxy API — determinism fingerprint, persona drive (evals/red-team), dynamic LoRA management, and embeddings. See the Deterministic Inference product page.

Endpoint summary

Method	Path	Purpose
GET	/v1/determinism-fingerprint	Declared + observed kernel config + effective fingerprint.
POST	/v1/persona/drive	Non-deterministic persona/red-team trajectory driver.
GET	/v1/loras	List loaded adapters.
POST	/v1/loras/load	Load an adapter by name/version (or path).
POST	/v1/loras/unload	Unload (refuses pinned governance adapters).
GET	/v1/loras/{lora_name}/exists	Fast availability check.
GET	/v1/loras/{lora_name}/metadata	Provenance / version metadata.
POST	/v1/embed	Embeddings (501 fail-open when not configured).
GET	/health	Readiness.

GET /v1/determinism-fingerprint

Returns the inference-side batch-invariant config so the control-plane DLIR verifier can pin every receipt to a specific kernel configuration:

{
  "declared": { "batch_invariance_level": "kernel_level" },
  "declared_fingerprint": "<sha256>",
  "observed": { "CUBLAS_WORKSPACE_CONFIG": ":4096:8", "torch.cuda.device_name": "…" },
  "effective_fingerprint": "<sha256 of declared + observed>",
  "service_version": "1.42.0"
}

The effective_fingerprint is what build_deterministic_receipt stamps into every /v1/govern receipt; the verifier's kernel_still_matches check compares against it.

POST /v1/persona/drive

Runs a non-deterministic multi-turn loop (Evals Phase 3): a persona model drives the customer's Agent-Under-Test like a varied human, and the resulting trajectory is returned for the control-plane to score with the deterministic evaluator SLM. An adversarial persona makes this an automated red-team run.

Request (PersonaDriveRequest): persona (model + temperature), scenario, agent_under_test (an OpenAI-compatible endpoint), max_turns, optional provider. Response: { scenario_id, messages, turn_count }. The persona is intentionally sampled at temperature > 0, deterministic = false — the judging is what is deterministic, not the driving.

LoRA management

POST /v1/loras/load and /v1/loras/unload hot-swap task adapters. Governance adapters (gov_*) are pinned and refuse unload — a production Guardian must never lose its weights mid-request. GET /v1/loras/{name}/metadata returns provenance (training run id, dataset hash, base model) so a receipt can be tied to a specific adapter version.

POST /v1/embed

Embeddings for the Vector Integrity manifold and RAG ingestion. Returns 501 (fail-open) when embeddings are not configured for this deployment — callers must handle the no-op.

Errors

HTTP	error.code	When
502	bad_gateway	Persona drive failed upstream.
503	service_unavailable	Engine not ready / GPU not initialized.
409	conflict	Attempt to unload a pinned governance adapter.

---

→ Deterministic Inference — the product overview. → Chat & Proxy API — the public governed chat contract. → Training & LoRA API — adapter provenance & training jobs.