Every inference engine has to guess.
Tribunus Compute doesn't.

Compile-time architecture for portable inference. Open source, numerically verified.

Explore the architecture View on GitHub

The Thesis

The old way

Every inference engine makes decisions at runtime. Which kernel is fastest right now? Is this quantization accurate enough? Can I batch these? Under time pressure, it guesses. You get inconsistent latency or wrong answers — and you never know which.

The Tribunus way

Freeze every decision at compile time. For each model, for each backend:

  1. Parse the model into canonical PhaseIR
  2. Generate hundreds of kernel candidates per operation
  3. Run every candidate through a 6-check admission pipeline
  4. Check every candidate against an FP32 reference oracle
  5. Freeze only the passing candidates into a compute image

At runtime: replay the compute image. No guessing, no JIT, no latency spikes. The engine runs exactly as fast as the best candidate — and exactly as accurate as the oracle certified.

Model → PhaseIR → Candidates → Oracle → Admission → Compute Image
Compile time
Compute Image ↓ Runtime Replay

Architecture

Compile-time Architecture

Every inference path is decided before the first token is generated. The compiler owns placement, memory layout, kernel selection, and fallback policy — the runtime merely replays the resulting compute image.

  • PhaseIR — canonical model representation consumed by all backends. Ops, shapes, dtypes, layouts, quantization metadata in a single IR.
  • Backend candidates — per-operation kernel variants from Triton, cuBLASLt, Metal 3, rocBLAS, oneDNN, TT-NN, and custom Metal/AMD/Intel kernels.
  • 6-check admission — oracle, shape, layout, timing, replay, and cache-key checks gate every candidate.
  • Compute image — frozen artifact containing prequalified backend code, placement manifest, memory layout, and evidence receipts.
Numerical Governance (The Oracle)

A 4-tier oracle establishes ground truth per operation, per backend, eliminating silent numerical drift.

  • 4-tier oracle — Apple Silicon FP32 → CUDA FP32 → Vulkan FP32 → theoretical bound (in descending priority). The highest available tier certifies pass/fail.
  • Per-op tolerance matrix — matmul = 1 ULP, softmax = 2 ULP, layer_norm = 1 ULP. Each operation type has a published tolerance in the governance spec.
  • Autotuning cache — 6-tier key: backend × op × dtype × shape × batch × quant. A cache miss triggers full re-evaluation at compile time.
Multi-Backend Compilation

Each backend is a first-class compilation target with its own candidate generator, admission checks, and runtime lane. The compiler produces backend-specific code regions bound into a single compute image.

  • Apple Silicon — Metal 3 (GPU), Indirect Command Buffers (ICMD), ANE coprocessor via Core ML MIL islands
  • NVIDIA — CUDA Graphs, cuBLASLt, Triton, CUTLASS
  • AMD — ROCm, rocBLAS, Triton HIP, Vulkan
  • Intel — oneDNN, Level Zero, Triton XPU
  • Tenstorrent — TT-NN, TT-Metalium
6 Runtime Pipelines

Inference is a deterministic state machine over six precompiled pipeline stages. Each stage has a fixed memory budget, known shapes, and prequalified backend regions.

  1. Token Intake — radix-tree prefix cache, tokenizer embedded in compute image
  2. Prefill — chunked flash attention, KV page write-once
  3. Decode — precompiled CUDA Graphs / Metal ICMD, weight-staging ring
  4. KV Management — hot → compressed tier migration, generation counters
  5. Speculation — ANE → CPU → GPU multi-device pipeline with tree verification
  6. Output — Accelerate logits, Aho-Corasick stop detection, SSE streaming, tool calling
Federated Inference (Dharma)

Semi-trusted peer networks for mutual-aid inference. Devices share compute capacity across a privacy-respecting federation with quorum-verified execution.

  • KV cache — privacy-classified social object with generation counters and access control
  • Transport — abstracted: local IPC, LAN, QUIC, WebRTC
  • Quorum verification — execution receipts from multiple peers guarantee correctness without trusting any single peer
  • Peer discovery — DHT-based with capability advertisements (backend type, free capacity, latency profile)
Failure Taxonomy

11 RealizerError types cover every failure mode from compilation through runtime replay. Every error is typed, structured, and actionable.

unsupported_dtypeOperation requested a dtype not realized by this backend
unsupported_layoutMemory layout (row/col-major, block, etc) not realizable
unsupported_mutationIn-place or aliased mutation not supported by backend lane
unsupported_dynamic_shapeDynamic shape where compile-time shape was expected
compile_failedBackend compiler rejected the candidate kernel
load_failedCompiled kernel/library failed to load at runtime
numerical_divergenceCandidate exceeded oracle tolerance during admission
performance_regressionCandidate slower than reference baseline at compile time
replay_invalidCompute image state inconsistent with runtime conditions
cache_key_mismatchAutotune cache key does not match current deployment
runtime_driver_faultDriver/hardware error during kernel execution

Backend Support

BackendPriorityStatusKey Library
Apple Silicon (Metal)PrimaryIn progressMetal 3, ANE
NVIDIAPrimaryIn progressCUDA 12, cuBLASLt, Triton
AMD (ROCm)SecondaryPlannedROCm 6, rocBLAS
Intel (oneDNN)SecondaryPlannedoneDNN, Level Zero
TenstorrentTertiaryPlannedTT-NN, TT-Metalium