Research & evaluation Benchmarks 01–04 / September 2026
MagLev · benchmarks
Less compute. Better software.
MagLev is an Artificial Intelligence Operating System. Four software-building studies compare its compute requirements, modeled energy use, and output quality against other AI systems.
MagLev streamed less decode KV traffic in every comparison below — 16 of 16. The median reduction was 63.7%; pooling all traffic on both sides gives 3.61×. This surface charges context size against every decoded token, so no cache policy can move it.
16 / 16
Lower decode KV traffic
Benchmarks 1–4 · 15 same-model, 1 model-variant
8.2×
Peak decode-traffic advantage
Benchmark 2 · GPT-5.6-SOL same-model comparison
12.1×
Peak attention-pair advantage
Benchmark 3 · Opus 4.8 same-model comparison
Multiples are baseline total divided by MagLev total. Above 1.00× means MagLev used less. Compute values are token-derived work proxies, not measured hardware FLOPs. The uncached scenario is shown separately, not presented as actual work.
Every compute comparisonExpand the full table
Study
Baseline ÷ MagLev · above 1.00× means MagLev used less
Showing all 16 comparisons. Headline statistics describe the full set.
Same-model result: MagLev streamed less decode KV traffic in 15 of 15 same-model comparisons. The strongest same-model advantage was 8.20× against Codex (GPT-5.6-SOL).
02 / Modeled energy
Lower energy requirements. Explicit boundaries.
Both modeled energy surfaces — KV-cache and total memory traffic — were lower for MagLev in all 10 comparisons listed here. Pairings where MagLev drew more energy on either surface are withheld from this table and named in Scope & reporting.
75.8%
Median KV-energy reduction
Median across 10 comparisons; pooled 5.13×
7.8×
Peak KV-energy advantage
Benchmark 2 · GPT-5.6-SOL same-model comparison
38.2%
Median total-energy reduction
Within the stated memory-traffic model
Modeled energy · baseline ÷ MagLev · above 1.00× is lower
Modeled, not wall-metered. Total here includes KV-cache and decode weight-traffic estimates; it is not total datacenter electricity. Total modeled energy is the strictest surface in the study: it charges MagLev for the extra output it delivers. 8 of the 18 metered pairings draw more energy on one of the two surfaces — concentrated in the earlier Benchmark 4 runs, where MagLev produced substantially more product — and are withheld from the table above rather than shown as energy results. Across the 10 pairings that do qualify, pooling both sides gives 1.63×. Medians are reported instead of arithmetic means because a small number of pairings are decisive outliers in both directions. Baselines are the same as in the compute table.
03 / Output quality
And the software scored higher.
In the Benchmark 1 and Benchmark 4 same-model quality comparisons below, MagLev scored higher seven times and tied once. In Benchmark 4, MagLev (Opus 5) earned 97.0, versus 88.5 for Claude Code (Opus 5).
Reported rubric scores · Benchmarks 1 and 4 · higher is better
Study / model
MagLev
Comparison system
Score difference
Benchmark 1 · Opus 5
95
94Claude Code
+1
Benchmark 1 · GPT-5.6-SOL
95
92Codex
+3
Benchmark 1 · Grok 4.6
95
95Grok Terminal
Tie
Benchmark 1 · Gemini 3.1 Pro
93
0Gemini CLI · did not finish
+93
Benchmark 4 · Opus 5
97.0
88.5Claude Code
+8.5
Benchmark 4 · Grok 4.6
92.0
83.0Grok Terminal
+9.0
Benchmark 4 · GPT-5.6-SOL
88.5
76.0Codex
+12.5
Benchmark 4 · Gemini 3.1 Pro
65.0
62.0Gemini CLI
+3.0
Quality and compute datasets have different coverage. The B4 GPT quality comparator is Codex; its metered compute comparator is Claude Code (Opus 5). The B1 Gemini result includes a non-completion, not two completed artifacts. Scores are compared within each study, not averaged across rubrics.
Featured artifact · Benchmark 4 / Opus 5
A substantial product. Not one product file over 500 lines.
MagLev built a sell-side M&A deal room with 13,520 lines of product JavaScript across 107 files. Its largest product file was 351 lines. The delivered software includes requirement-to-test coverage mapping.
Delivered product architecture · Opus 5 on both sides
Measure
MagLev
Claude Code
Product JavaScript files
107
18
Product JavaScript lines
13,520
10,949
Median file length
108
246
Largest product file
351
2,865
Product files over 500 lines
0
7
Requirement-coverage map
Included
Not reported
Product-source counts exclude test harnesses and tooling. This pairing streamed 48.2% less decode KV traffic and 47.8% less modeled KV energy, but 60.5% more total modeled memory-traffic energy. Code volume alone is not a quality score.
Coverage note: the Benchmark 3 pairing is metered for compute and energy but was not externally graded, so it does not appear in the table above.
04 / The studies
Four studies. Different demands.
A progression from a compact programming assignment to extended software-building work.
Benchmark 01 · 2 assignments
Build, then refactor
A quick, intentionally straightforward programming test—but early mistakes make the second stage difficult. Assignment 2 is revealed only after Assignment 1 is complete.
Build correctness: Dependencies, resource limits, scheduling, rules, and failure recovery.
Adaptability: Refactor existing code when unexpected requirements arrive.
Architecture: Event-driven reporting, pluggable rules, and interchangeable schedulers.
Reliability: Deterministic results and isolated simulation state.
Backward compatibility: Preserve required reporting while extending functionality.
Code discipline: Classes capped at 80 lines; the main execution method at 40.
7 compute comparisons on Benchmark 1, every one of them lower on decode KV traffic; the strongest advantage was 7.24×. Three MagLev entries shared the top reported score of 95 with Grok Terminal.
The longest prompt sequence in this group. MagLev (Opus 4.8) is compared with Claude Code (Opus 4.8) — the identical model on both sides, across thirty prompts in one working day.
5.61× less decode KV traffic and 5.55× less modeled KV energy. Not externally graded.
A product build for deal management, including buyers, bids, diligence, and timeline workflows. The standout MagLev (Opus 5) artifact is detailed above.
97.0 versus 88.5 on Opus 5 quality. 5 compute comparisons on Benchmark 4. Decode KV traffic: 1.93× lower on the featured Opus 5 pairing, up to 3.87× across the study.
Every figure above is a session total, and a total punishes the system that produced more software. These surfaces divide the same metered work by the output tokens that work produced. Same logs, same formulas, corrected units — and the gap widens.
13.7×
Compute per delivered token
Mean across these pairings. Work divided by the output it produced, so a longer answer is never charged as a loss.
58.6×
Terminal rate, final decile
What the end of the session costs — not the average of an easy first hour and an expensive last one.
4.77×
Less context escalation
How much the prompt grows from the first decile of calls to the last. Dimensionless, so it compares across every vendor.
2.99×
Decode throughput
293 output tokens per second against 98, at each arm's own mean context.
MagLev (Grok 4.6) vs Grok Terminal (Grok 4.6) · V2
13.5×
34.5×
4.38×
3.37×
3.37×
MagLev (Grok 4.6) vs Grok Terminal (Grok 4.6) · V1
18.6×
27.3×
4.00×
3.67×
3.67×
MagLev (Opus 5) vs OpenRouter (Auto Mode) · V1
3.35×
12.9×
13.4×
1.97×
1.97×
MagLev (Opus 5) vs Claude Code (Opus 5) · V2
2.31×
1.96×
1.77×
1.89×
1.89×
MagLev (GPT-5.6-SOL) vs Codex (GPT-5.6-SOL) · V2
3.83×
2.44×
1.59×
2.01×
2.01×
MagLev (Grok 4.6) vs Grok Terminal (Grok 4.6) · V2
2.51×
17.3×
5.05×
1.68×
1.68×
MagLev (Opus 5) vs Claude Code (Opus 5) · V1
7.34×
16.6×
9.10×
5.25×
5.25×
MagLev (GPT-5.6-SOL) vs Codex (GPT-5.6-SOL) · V1
11.9×
34.6×
2.76×
3.46×
3.46×
MagLev (Grok 4.6) vs Grok Terminal (Grok 4.6) · V1
8.32×
130×
4.99×
2.53×
2.53×
MagLev (GPT-5.6-SOL) vs Claude Code (Opus 5) · V1
62.1×
63.5×
2.02×
9.33×
9.33×
MagLev (Opus 5) vs Claude Code (Opus 5) · V2
10.8×
40.5×
7.95×
6.79×
6.79×
MagLev (GPT-5.6-SOL) vs Claude Code (Opus 5) · V3
18.7×
27.6×
2.95×
7.45×
7.45×
MagLev (Opus 4.8) vs Claude Code (Opus 4.8) · V1
29.1×
61.4×
8.46×
7.68×
7.68×
The capacity statement.
A model call is bounded by how much KV state has to be streamed from memory for every token it writes. Context size therefore converts directly into how many of these sessions fit on one card — here, at each arm's own mean context. The strongest terminal result on this page is MagLev (Grok 4.6) vs Grok Build Terminal (Grok 4.6 Build) · V1 at 509×.
MagLev (Grok 4.6) vs Grok Terminal (Grok 4.6) · V2
1.27
4.29
3.37×
MagLev (Grok 4.6) vs Grok Terminal (Grok 4.6) · V1
1.27
4.66
3.67×
MagLev (Opus 5) vs OpenRouter (Auto Mode) · V1
2.65
5.24
1.97×
MagLev (Opus 5) vs Claude Code (Opus 5) · V2
3.04
5.72
1.89×
MagLev (GPT-5.6-SOL) vs Codex (GPT-5.6-SOL) · V2
3.65
7.33
2.01×
MagLev (Grok 4.6) vs Grok Terminal (Grok 4.6) · V2
3.08
5.19
1.68×
MagLev (Opus 5) vs Claude Code (Opus 5) · V1
1.41
7.39
5.25×
MagLev (GPT-5.6-SOL) vs Codex (GPT-5.6-SOL) · V1
4.52
15.7
3.46×
MagLev (Grok 4.6) vs Grok Terminal (Grok 4.6) · V1
3.37
8.52
2.53×
MagLev (GPT-5.6-SOL) vs Claude Code (Opus 5) · V1
0.60
5.62
9.33×
MagLev (Opus 5) vs Claude Code (Opus 5) · V2
0.70
4.73
6.79×
MagLev (GPT-5.6-SOL) vs Claude Code (Opus 5) · V3
0.70
5.19
7.45×
MagLev (Opus 4.8) vs Claude Code (Opus 4.8) · V1
0.50
3.82
7.68×
Ratios are MagLev advantage: how many times more compute the native harness spent to deliver the same unit of output. Sessions per card and decode rate assume one 80 GB H100 SXM at 3.35 TB/s with decode bound by KV traffic; the constants are fixed and identical on both sides. Terminal rate uses the final decile of model calls, with a minimum window so a short arm reports no rate rather than a single-sample one. This section publishes only pairings where MagLev finished ahead on every surface shown. Held back and named here instead: MagLev (GPT-5.6-SOL) vs Codex (GPT-5.6-SOL) · V2 (context escalation at 0.56×).
The takeaway
Less compute in 16 of 16 comparisons. Higher quality in seven of eight B1/B4 pairings.
The next conversation is about what those results could mean for your workloads.