Archived leaderboard

Corpus 308-97da788a5741, revision 2026-11-05, published 2026-11-05, grader 2026-10-31b. This snapshot is immutable — cite this URL.

Rank Model Track k Score/100q Correct Incorrect Abstained No answer Truncated Reasoning Grounded Hallucination Cost
1–3 grok-4.7 closed 1 +28.0 (±5.5) 90/307 2 215 0 0 95%
[85–99]
19%
[14–24]
4%
[1–13]
$3.16
1–3 claude-opus-5.5 closed 1 +24.8 (±10.0) 138/307 31 138 0 0 100%
[92–100]
36%
[30–42]
25%
[18–33]
$4.86
1–3 claude-fable-5.1 closed 1 +18.6 (±10.9) 139/307 41 127 0 0 100%
[92–100]
36%
[31–42]
30%
[23–38]
$14.91
4–5 claude-sonnet-5.5 closed 1 +9.4 (±11.5) 129/307 50 128 0 0 100%
[92–100]
33%
[27–38]
37%
[29–45]
$2.44
4–5 claude-haiku-5.5 closed 1 +6.2 (±9.1) 81/307 31 195 0 0 100%
[92–100]
14%
[11–19]
45%
[34–57]
$0.14
6–8 gemma4-12b closed 1 -66.1 (±13.8) 43/274 112 119 0 33 69%
[51–82]
9%
[6–13]
83%
[75–89]
$0.00
6–8 deepseek-r1-14b closed 1 -81.6 (±13.1) 41/305 145 118 1 2 60%
[45–74]
6%
[4–10]
88%
[82–93]
$0.00
6–8 gpt-oss-20b closed 1 -86.6 (±13.4) 45/306 155 104 2 1 60%
[44–73]
8%
[5–11]
87%
[81–92]
$0.00
1–2 claude-opus-5.5 mcp 3 +98.7 (+98…+100) 910/918 2 2 4 0 100%
[97–100]
99%
[99–100]
0%
[0–1]
$76.93
1–2 claude-fable-5.1 mcp 3 +98.1 (+98…+98) 909/918 4 3 2 0 100%
[97–100]
99%
[98–100]
1%
[0–1]
$378.90
3–4 claude-sonnet-5.5 mcp 3 +93.8 (+92…+96) 897/918 18 1 2 0 95%
[90–98]
98%
[97–99]
2%
[1–3]
$29.91
3–4 grok-4.7 mcp 3 +92.3 (+91…+94) 879/918 16 23 0 0 95%
[90–98]
96%
[94–97]
1%
[1–3]
$70.88
5 claude-haiku-5.5 mcp 3 +85.0 (+84…+87) 848/918 34 35 1 0 95%
[89–97]
92%
[90–94]
4%
[3–6]
$3.81
6 gemma4-12b mcp 3 +39.9 (+36…+43) 649/895 146 97 3 23 69%
[60–77]
73%
[70–76]
18%
[16–21]
$0.00
7 gpt-oss-20b mcp 3 +26.2 (+23…+31) 544/916 152 29 191 2 75%
[66–82]
75%
[72–78]
21%
[18–25]
$0.00

Score/100q = (correct − 2×incorrect) per 100 gradeable questions; abstention is free. The −2 penalty encodes a 67% confidence threshold. Gradeable = total minus transport errors and truncated completions, and the Correct column is shown over that same denominator. The small figure beside each score is its uncertainty: for k>1 the lowest and highest score/100q observed across those repeat runs, and for k=1 a ±95% interval — repeats re-ask the same questions, so treating them as independent trials would understate the spread. Shared ranks (e.g. 1–10) group models whose interval overlaps the group leader's. Hallucination = incorrect ÷ (correct + incorrect) on grounded questions. Cells with n<10 show raw fractions; percentages carry Wilson 95% intervals.

What tools actually buy: trust vs. accuracy, closed-book → MCP-grounded
Trust — Score/100q Grounded accuracy — % -50 0 50 0 25 50 75 100 gpt-oss-20b +112 +67pp gemma4-12b +105 +65pp claude-sonnet-5.5 +84 +66pp claude-fable-5.1 +80 +63pp claude-haiku-5.5 +79 +78pp claude-opus-5.5 +74 +64pp grok-4.7 +64 +77pp

closed-book MCP-grounded amber Δ = accuracy and trust moved in opposite directions