Skip to main content

Deterministic Inference

Bit-exact, reproducible AI inference. The same prompt + same seed + same Guardian version always produces the same bytes — across batches, nodes, and restarts — so every verdict is replayable and every receipt is auditable.

What it is​

Deterministic Inference is the kernel mode (batch_invariant) that makes Trinitite verdicts evidence rather than opinions. Built on a hardened SGLang / vLLM serving stack, it pins the sampling, batching, and execution path so that a given (input, guardian_version, policy_hash) triple yields a byte-identical output every time. Batching does not change the answer. Node placement does not change the answer. A restart does not change the answer.

Why it matters​

  • Replayability = auditability. A past decision can be re-executed bit-exactly; the resulting receipt matches the original. Auditors and regulators accept reproducible evidence; they reject "the model said something different today."
  • Server load can't move the score. LLM-as-judge pipelines give different verdicts at 95% GPU utilization than at 5%. Deterministic inference does not — the compliance number is stable under load.
  • Regulator- and auditor-grade. Regulators and auditors need a number that doesn't drift for non-statistical reasons. This is that number.
  • Forensic replay. Any production incident can be re-run to confirm exactly what the Guardian decided and why.

How it works​

  1. Pin the execution path. Sampling, kernel selection, and batching are constrained so the result is independent of batch composition and node.
  2. Version everything. A verdict carries guardian_version and policy_hash; together with the input, these fully determine the output.
  3. Replay. Re-submit the same triple to the Inference Service and receive the same bytes — the receipt's chain_hash matches.

Get started​

  • Deterministic mode is the default for Guardian governance calls via the Chat and Proxy endpoints.
  • Run direct reproducible inference via the Inference Service API.
  • Replay any historical verdict via the Logs API + the inference service.

→ Inference Service API — reproducible inference surface. → Architecture — the deterministic kernel. → Evidence & Verification — why determinism underwrites the receipt.

Last updated on