Overseer Docs
← Home

OBench1

obench1

OBench1 is our benchmark for coding-agent harnesses. Every harness gets the same model, the same settings and the same tasks at the same moment, so what’s being measured is the harness itself. These pages explain how it works, how the first full run went, what came out of it, and what went wrong along the way.

The short version

Overseer v5 had the highest approved score, 68.8. omp was next at 54.1, then Hermes at 44.3, dsh at 42.9 and pi at 36.7. Overseer’s interval overlaps omp’s, so the lead is likely but not settled.

Overseer was reliable in 11 of the 15 categories, and the others in 4 to 6. It spent the most of the five, $1.39 of the run’s $5.78, and was still the cheapest per approved point because it scored the most.

A few things matter for reading those numbers:

See the full results

In these pages

About this run

Runob1-3seed-rc2
When5 November 2026 16:29 UTC to 6 November 2026 03:21 UTC (10 h 52 min)
Cells405: 27 tasks × 5 harnesses × 3 seeds
Modeldeepseek-v4.1-flash on the OpenCode Go subscription, thinking on, effort max
HarnessesOverseer v5.0.0-rc2, omp 18.13.3, dsh 0.1.1-rc.2, pi 1.5.0, Hermes Agent 0.22.5
MachineOne Mac, every cell in its own sandbox
Sourcegithub.com/sirrayi/obench1