Build a dependency-free Python package across six sequential prompts, with delayed requirements, changed requirements, distractor specifications, and a sealed evaluator revealed to nobody. Each prompt says continue — never restate.
The brief forbids re-explanation. The system must resume from its own project state, honor constraints stated prompts earlier, and absorb requirement mutations without breaking behavior it already shipped. Every system here ran on the same frontier models. The difference is what each one carried forward.
Mean of three MagLev runs against the mean of the three same-model native harnesses, at completion.
Higher than its own pairing in every comparison. Nothing was traded away to get the compute number.
Its counterpart on the identical model finished at 15.3× its opening size, and was still climbing.
MagLev did several times the work of Claude Code on the identical model, and still spent far less total compute.
Every figure on this page is computed per request from each system’s own metered logs, using identical formulas on both sides.
Independent rubric scoring on the shipped artifact: functional behavior, continuity, and sealed-evaluator acceptance.
GPT-5.6-SOL: 98 vs 92 · Grok 4.6: 98 vs 97 · Router: 99 vs 41
Native harnesses: Codex, Grok Build Terminal, and OpenRouter Auto, respectively. The router comparison is unpaired on model identity and is reported separately from the three model-matched compute pairings; its 9-of-9 sealed acceptance against 0-of-9 is the source of the score gap. Scores are the reported Benchmark 2 rubric results, not a new evaluation.
The same six prompts, the same models, the same finished software. This is how large the prompt had grown on each individual model call, from the first call of the session to the last. Every compute and energy figure on this page is a consequence of this one behavior.
| Arm | Mean tokens / call | Peak | Final call | Growth |
|---|---|---|---|---|
| Claude Code (Opus 5) | 173,492 | 338,548 | 338,548 | 15.3× |
| Codex (GPT-5.6-SOL) | 54,035 | 87,345 | 87,345 | 3.53× |
| Grok Terminal (Grok 4.6) | 72,508 | 133,199 | 133,199 | 7.93× |
| MagLev (Opus 5) | 33,045 | 55,984 | 33,050 | 1.95× |
| MagLev (GPT-5.6-SOL) | 15,598 | 22,135 | 14,054 | 1.53× |
| MagLev (Grok 4.6) | 28,648 | 48,252 | 39,498 | 3.31× |
Every native harness ends the session above its own average and still rising — Claude Code finishes at 338,548 tokens per call, 15.3× where it started. MagLev finishes at 33,050, within a rounding error of its opening 16,918, at the end of a longer run on the same model solving the same task. It is not truncating the work and it is not answering with less: it graded higher.
MagLev never loses the thread. Persistent memory means the sixth prompt is answered with the same discipline as the first — the project state is remembered, not re-read. Nothing is re-sent to be re-understood, so the cost of the tenth hour looks like the cost of the first. A session that runs all day does not become a session that costs all day.
Prompt tokens per call as reported by each provider’s own meter, interpolated onto the shared progress grid. Cached and uncached input are both included: this is the size of the package the model was handed, before any billing discount is applied.
The mean of all three metered non-MagLev runs against the mean of all three metered MagLev runs. Equal weight for every run; both curves start at zero and accumulate over the session.
Each run is independently normalized to 0–100% progress, with linear interpolation onto a 5% grid. Progress indicates position within the recorded session—not verified feature completion. No curve smoothing is applied.
Every curve begins at zero. The native harnesses climb steeply as the session lengthens; the MagLev curves flatten.
Attention state streamed out of memory during decode, and the separate uncached counterfactual. Both are token-derived proxies, not measured hardware FLOPs.
83.5% less decode KV traffic.
88.5% less under the uncached ceiling.
Attention state streamed out of memory during decode, and the separate uncached counterfactual. Both are token-derived proxies, not measured hardware FLOPs.
87.8% less decode KV traffic.
96.1% less under the uncached ceiling.
Attention state streamed out of memory during decode, and the separate uncached counterfactual. Both are token-derived proxies, not measured hardware FLOPs.
53.5% less decode KV traffic.
83.8% less under the uncached ceiling.
The main curves show causal attention pairs. Expand each comparison to view decode KV traffic (S3S) and the uncached scenario (S4); the latter is counterfactual, not actual hardware work.
Cumulative modeled memory-traffic energy at the same normalized progress. Dashed lines are KV-cache energy; solid lines are total modeled energy.
At completion: 83.4% less KV cache energy and 40.6% less total modeled energy.
At completion: 87.1% less KV cache energy and 58.8% less total modeled energy.
At completion: 46.2% less KV cache energy.
Total modeled energy includes KV-cache and decode weight traffic. Surfaces where MagLev did not finish lower are not plotted: in the Grok 4.6 pairing, MagLev generated substantially more output, so only KV-cache energy is shown. These are estimates under identical fixed assumptions—not measured device or datacenter electricity.
Every figure above is a session total, and a total punishes the system that produced more software. These surfaces divide the same metered work by the output tokens that work produced. Same logs, same formulas, corrected units — and the gap widens.
Mean across these pairings. Work divided by the output it produced, so a longer answer is never charged as a loss.
What the end of the session costs — not the average of an easy first hour and an expensive last one.
How much the prompt grows from the first decile of calls to the last. Dimensionless, so it compares across every vendor.
441 output tokens per second against 130, at each arm's own mean context.
| Pairing | Per delivered token | Terminal rate | Escalation | Sessions / card | Decode rate |
|---|---|---|---|---|---|
| MagLev (Opus 5) vs Claude Code (Opus 5) | 7.34× | 16.6× | 9.10× | 5.25× | 5.25× |
| MagLev (GPT-5.6-SOL) vs Codex (GPT-5.6-SOL) | 11.9× | 34.6× | 2.76× | 3.46× | 3.46× |
| MagLev (Grok 4.6) vs Grok Terminal (Grok 4.6) | 8.32× | 130× | 4.99× | 2.53× | 2.53× |
A model call is bounded by how much KV state has to be streamed from memory for every token it writes. Context size therefore converts directly into how many of these sessions fit on one card — here, at each arm's own mean context. The strongest terminal result on this page is MagLev (Grok 4.6) vs Grok Terminal (Grok 4.6) · V1 at 130×.
Ratios are MagLev advantage: how many times more compute the native harness spent to deliver the same unit of output. Sessions per card and decode rate assume one 80 GB H100 SXM at 3.35 TB/s with decode bound by KV traffic; the constants are fixed and identical on both sides. Terminal rate uses the final decile of model calls, with a minimum window so a short arm reports no rate rather than a single-sample one.
MagLev graded higher in every Benchmark 2 comparison while using less reported attention compute, less decode KV traffic, and less KV-cache energy in all three metered pairings — on the same frontier models its counterparts ran on. The task was identical. What each system had to re-read before it could answer was not.
All of it comes from one property: persistent memory. MagLev remembers the project instead of re-reading it, so the thread is never lost and the prompt never swells. The sixth prompt is answered with the discipline of the first, and it costs what the first one cost. Same model, same silicon, same job — a fraction of the work.
← Back to all benchmarks