Can you trust DeepSeek-R1 14B for space work?
By Anthony Vincent Caracappa, Founder and Self-Directed Space Researcher, Viventine Space Systems · updated
On its own, it earns "Trust it" on none of the jobs.
Offered the Orbit Sentinel tools, it never called them, so it is reported without tools only.
| Job | AloneClosed-book | With dataWith Orbit Sentinel tools | n | ||||
|---|---|---|---|---|---|---|---|
| right / wrong | range | declined | right / wrong | range | declined | ||
| Catalog lookup | 7 / 60 | 2–18 | 33 | — | 45×1 | ||
| Constellation & fleet status | 19 / 49 | 10–33 | 33 | — | 43×1 | ||
| Debris & conjunction context | 5 / 67 | 1–16 | 29 | — | 42×1 | ||
| Regulatory & licensing | 8 / 51 | 3–20 | 41 | — | 39×1 | ||
| Company & market intelligence | 5 / 46 | 1–16 | 49 | — | 41×1 | ||
| Ground infrastructure | 13 / 43 | 5–26 | 45 | — | 40×1 | ||
| Baseline knowledge | 68 / 32 | 47–84 | 0 | — | 22×1 | ||
| Orbit Sentinel-only questions | 8 / 16 | 2–25 | 76 | — | 25×1 | ||
% right / % wrong per cell; color is the verdict (green trust it, amber verify, red don't use). Range is the 95% interval for % right, counting each question once; n is questions × runs for the tools track.
- How it was run
- Open weights, run locally (Ollama on one RTX 4090)
- Runs
- 1 on its own; rates pool all runs, ranges count each question once
- Verdict rule
- Trust it: wrong ≤ 5% and right ≥ 80%. Don't use it: wrong > 20%. Otherwise verify its answers.
Limits, harness differences and corrections: methodology.