Which AI can you trust for ground infrastructure?
By Anthony Vincent Caracappa, Founder and Self-Directed Space Researcher, Viventine Space Systems · updated
Earth stations, gateways, call signs and where they are.
On their own, no AI earns "Trust it" here. With Orbit Sentinel tools: Claude Opus 5.5, Claude Fable 5.1, Claude Sonnet 5.5, Grok 4.7.
| AI | AloneClosed-book | With dataWith Orbit Sentinel tools | n | ||||
|---|---|---|---|---|---|---|---|
| right / wrong | range | declined | right / wrong | range | declined | ||
| Claude Opus 5.5 Anthropic | 23 / 18 | 12–38 | 60 | 100 / 0 | 91–100 | 0 | 40×3 |
| Claude Fable 5.1 Anthropic | 28 / 28 | 16–43 | 45 | 99 / 1 | 90–100 | 0 | 40×3 |
| Claude Sonnet 5.5 Anthropic | 28 / 18 | 16–43 | 55 | 96 / 4 | 85–99 | 0 | 40×3 |
| Grok 4.7 xAI | 13 / 5 | 5–26 | 83 | 93 / 3 | 80–97 | 5 | 40×3 |
| Claude Haiku 5.5 Anthropic | 13 / 3 | 5–26 | 85 | 89 / 6 | 76–96 | 5 | 40×3 |
| gpt-oss 20B OpenAI | 5 / 45 | 1–17 | 50 | 46 / 20 | 31–61 | 34 | 40×3 |
| Gemma 4 12B Google | 10 / 36 | 4–24 | 54 | 61 / 24 | 45–74 | 15 | 40×3 |
| DeepSeek-R1 14B DeepSeek | 13 / 43 | 5–26 | 45 | — | 40×1 | ||
% right / % wrong per cell; color is the verdict (green trust it, amber verify, red don't use). Range is the 95% interval for % right, counting each question once; n is questions × runs for the tools track.
Sample questions
- How many applications for new earth station authority (FCC file type SES-LIC) were filed with the FCC in calendar year 2024?
- In calendar year 2018 (the year of the FCC's C-band receive-only earth station registration window), how many earth station registration filings (FCC file type SES-REG) were filed?
- How many earth station assignment or transfer-of-control applications (FCC file types SES-ASG plus SES-T/C) were filed in calendar year 2024?
- How many earth station license renewal applications (FCC file type SES-RWL) were filed in calendar year 2024?
- How many distinct applicants filed at least one new earth station license application (SES-LIC) with the FCC in calendar year 2024?
- Which company filed the most new earth station license applications (SES-LIC) with the FCC in calendar year 2024?