PHANTOM Samba Benchmark Suite: High-Throughput Latency & Serving

Verified on NVIDIA Tesla T4 Business Source License 1.1 (BSL 1.1) OrthoSSM: Quasi-Unitary Projective Hidden-State Attention-Free Nonlinear Tensor Operator Manifold

High-Throughput State Prefix Caching for Samba & Hybrid Models

Fusing the OrthoSSM Projective Hidden-State Radix prefix manifold and dynamic INT8 state quantization with State Space Models (SSMs) and Samba hybrid architectures. By snapshotting recurrent states ($h_{\text{ssm}}$) and convolution buffers ($h_{\text{conv}}$) in sub-90 $\mu\text{s}$, PHANTOM achieves 2.76× faster TTFT and instant 0 ms prompt prefill resumption.

Explore OrthoSSM-Hybrid-195M CERN Zenodo • DOI 10.5281/zenodo.22177116

2.76× Faster

Time-To-First-Token (TTFT)

< 90 μs

Radix Tree Prefix Lookup

0 ms

Cached Prompt Resumption

75% Saved

INT8 State Memory Reduction

Interactive Radix Prefix Simulator

Simulate multi-turn chat prompt resumption and measure cold vs warm TTFT latency.

CACHE HIT

Prefix Length (Cached Context Tokens):

Simulated User Query Turn:

Cold Prefill Latency

18.76 ms

Full sequence prefill

Warm Prefill Latency

6.80 ms

0 ms prompt prefill resumption

Radix State Lookup

0.0900 ms (90.0 μs)

SHA-256 tree traverse

Effective TTFT Speedup

2.76×

1,024 Tokens Reused

# Generated Response Snippet:

"PHANTOM achieves 0ms prompt resumption by snapshotting recurrent states (h_ssm) and conv buffers (h_conv) in a Radix tree with INT8 quantization."

Time-To-First-Token Scaling (Cold vs Warm)

Empirical measurement on NVIDIA Tesla T4 across prompt prefix lengths.

💡 Insight: Because standard Transformer prefill scales quadratically $\mathcal{O}(N^2)$ or linearly $\mathcal{O}(N)$, prompt resumption latency increases with prefix size. With PHANTOM Radix caching, warm prefill latency remains flat regardless of prefix length.

Direct Hugging Face Hub Integration

trust_remote_code=True

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load the official PHANTOM foundation hybrid directly from Hugging Face Hub:
model_id = "Prannesshkva/OrthoSSM-Mamba-Falcon-Hybrid"  # or "Prannesshkva/OrthoSSM-Qwen2.5-1.5B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

# Benchmark 0ms prompt resumption and constant O(1) state memory
prompt = "Analyze how hybrid Mamba-2 SSD and Falcon MQA prevents associative recall collapse:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Academic DOI & Licensing Framework DOI: 10.5281/zenodo.22177116

Developed & Owned Exclusively by Prannessh (@Prannesshkva) under Business Source License 1.1 (BSL 1.1) and the Berne Convention.

Base Architecture Acknowledgements: Alibaba Cloud (Qwen 2.5), Technology Innovation Institute (Falcon 40B Apache 2.0), Albert Gu & Tri Dao (Mamba-2 SSD).

Architectural Pioneer Notice: Prannesshkva/OrthoSSM-Mamba-Falcon-Hybrid is the first-ever integration of Mamba-2 SSD with Falcon Multi-Query Attention (MQA), created by Prannessh.