← All verdicts

Can you trust gpt-oss 20B for space work?

By , Founder and Self-Directed Space Researcher, Viventine Space Systems · updated

On its own, trust it for baseline knowledge. Connected to Orbit Sentinel, trust it for catalog lookup.

Job Alone With data
right / wrong declined right / wrong declined
Catalog lookup
7 / 67
27
92 / 1
7
Constellation & fleet status
12 / 62
26
36 / 28
36
Debris & conjunction context
5 / 69
26
58 / 13
29
Regulatory & licensing
10 / 56
33
48 / 18
33
Company & market intelligence
2 / 66
32
70 / 11
19
Ground infrastructure
5 / 45
50
46 / 20
34
Baseline knowledge
96 / 4
0
85 / 5
10
Orbit Sentinel-only questions
8 / 20
72
67 / 23
11

% right / % wrong per cell; color is the verdict (green trust it, amber verify, red don't use). Range is the 95% interval for % right, counting each question once; n is questions × runs for the tools track.

How it was run
Open weights, run locally (Ollama on one RTX 4090)
Tool loop
SpaceBench tool loop
Runs
1 on its own, 3 with tools; rates pool all runs, ranges count each question once
Verdict rule
Trust it: wrong ≤ 5% and right ≥ 80%. Don't use it: wrong > 20%. Otherwise verify its answers.

Limits, harness differences and corrections: methodology.