NVMe holds the weight files and backing data. RAM supplies active CPU pages, resident experts and file cache. VRAM provides the GPU compute path, selected experts and the engine-specific KV working set. Weight-file size is therefore not a requirement for full simultaneous RAM residency.

Q3 is Unsloth UD-IQ3_XXS (81.96 GB of shards); Q4 is UD-Q4_K_XL (111.33 GB). Strata's native Q4 pack promotes 195 projections to BF16. The pinned implementations are thecodacus, upstream llama.cpp, ik_llama.cpp, OptLlama and Strata.

The dataset contains 240 synthetic-throughput, 103 retrieval, 56 native-benchmark, two warmup and five functional timing records. Server requests and native sample means are separate experimental units. Repeated-series medians never mix different cases. Long-context observations include n=1 results; the campaign is staged tuning rather than a randomized factorial study.

CUDA 12.8.1 build environment for llama.cpp, SM86 / Release / native CPU target. Strata uses CUDA 13 libraries. GGUF revision: 38bb39ee97821de2c9009abb7e93950eec396e66. The direct Q3 presets use lazy loading with --load-mode none; do not replace this with the initial Q4 mmap settings and attribute the same performance to it.

Early Q4 offload used CPU-side MoE, mmap/lazy loading and profiled hot-expert caching. An 8,192-token input measured 135.97 prefill and 18.92 decode tokens/s. A microbatch-1,024 variant improved that to 219.65/19.71, still well below the target.

Q3 screening then explored cache size, thread count, batching and hybrid CPU/GPU placement. Larger cache alone had little effect on prefill. Hybrid placement and the revised processing path produced a substantially better envelope.

Speculation trades off more than decode.

In ik_llama.cpp, MTP1 increased generation from 30.94 to 33.88 tokens/s at 8,192 input tokens, while prefill fell from 1,223.08 to 911.13. The thecodacus f16 finalists reached 35.97 TG with MTP2 versus 34.13 with MTP1. Support and effective behavior are implementation-specific.

OptLlama's combined cache/workspace/prefetch/overlap configuration passed short retrieval at 1,287.75/35.38. Its deep-input trial reached only 197 MiB VRAM headroom and was stopped by the resource guard, with no accepted complete result.

The matched upstream comparison holds CPU MoE 48, batch 8,192, microbatch 4,096, context 262,144 and disabled speculation constant. At 235,929 input tokens, f16 KV generated at 26.78 tokens/s versus 16.77 with q8_0. Prefill was nearly unchanged.

A second upstream condition, CPU46/u4096, declined from 1,066.83/30.45 at 8,192 tokens to 905.76/27.39 at 235,929: 15.10% lower prefill and 10.03% lower decode. Both lower criteria were missed at depth, despite passing a separate retrieval request.

The earlier thecodacus 256k series declined more sharply, from 665.81/31.80 at 8,192 to 402.45/15.80 at 235,929. That series used different offload and microbatch settings, so it is not an isolated context-allocation comparison with the later upstream result.

The integrated Q3/upstream profile uses CPU MoE 48 and microbatch 2,048 to leave audio headroom, while retaining a 262,144-token allocation. Its API path is Nginx → LiteLLM → llama-swap → llama.cpp. It is a different configuration from the CPU46/u4096 throughput experiment.

llama-swap manages backend lifetimes; one physical llama.cpp slot serializes generation. It is not a durable shared queue with ComfyUI. GPU image processing runs in a separate, drained operating mode. These text measurements do not include simultaneous image generation.

Speech is a separate latency budget.

Twenty synthetic TTS → ASR cycles measured full-WAV completion at 0.829 s p50 / 2.410 s p95. The first cold trial took 42.304 s during overlap with LLM work. A later cycle produced a valid WAV but failed strict transcription with one incorrect word.

These measure server stages, not the time from the end of human speech to first audible response. Live microphone, listening and interruption quality are outside the benchmark's validated scope.

Tool-use validation

The agent audit exercised 20 of 32 enabled tools, including Chromium, search, files, terminal, canvas, cross-session memory, a 32k subagent and images. An 18-call research task took 413.906 s. It did not establish completion of the separate 100-restaurant research task.

The retained system runs Q3/upstream llama.cpp. Strata's Q4 tests demonstrate a different residency strategy and near-full-context speed envelope, but do not validate the same agent, tool, image or speech workload.

Scope of inference

This staged campaign does not establish sustained 40 TG near full context, broad natural-task accuracy at 260k, randomized single-factor effects, or a full invariant-configuration context matrix. Clock/thermal controls and isolated physical NVMe I/O are unavailable. Complete raw input fixtures are not distributed, limiting byte-for-byte reproduction. The published timing records and pinned configurations support verification of the reported observations.