Compile-time architecture for portable inference. Open source, numerically verified.
Every inference engine makes decisions at runtime. Which kernel is fastest right now? Is this quantization accurate enough? Can I batch these? Under time pressure, it guesses. You get inconsistent latency or wrong answers — and you never know which.
Freeze every decision at compile time. For each model, for each backend:
At runtime: replay the compute image. No guessing, no JIT, no latency spikes. The engine runs exactly as fast as the best candidate — and exactly as accurate as the oracle certified.
Every inference path is decided before the first token is generated. The compiler owns placement, memory layout, kernel selection, and fallback policy — the runtime merely replays the resulting compute image.
A 4-tier oracle establishes ground truth per operation, per backend, eliminating silent numerical drift.
Each backend is a first-class compilation target with its own candidate generator, admission checks, and runtime lane. The compiler produces backend-specific code regions bound into a single compute image.
Inference is a deterministic state machine over six precompiled pipeline stages. Each stage has a fixed memory budget, known shapes, and prequalified backend regions.
Semi-trusted peer networks for mutual-aid inference. Devices share compute capacity across a privacy-respecting federation with quorum-verified execution.
11 RealizerError types cover every failure mode from compilation through runtime replay. Every error is typed, structured, and actionable.
| Backend | Priority | Status | Key Library |
|---|---|---|---|
| Apple Silicon (Metal) | Primary | In progress | Metal 3, ANE |
| NVIDIA | Primary | In progress | CUDA 12, cuBLASLt, Triton |
| AMD (ROCm) | Secondary | Planned | ROCm 6, rocBLAS |
| Intel (oneDNN) | Secondary | Planned | oneDNN, Level Zero |
| Tenstorrent | Tertiary | Planned | TT-NN, TT-Metalium |