05 Launch day

Launch-day results

The plan: publish rerun-bench numbers for a new model within 24 hours of its launch, so there is a measured, reproducible data point next to the announcement.

5.1 Plan

What happens after a launch

T+0
Announcement. The model becomes available in at least one supported CLI (Claude Code, Codex CLI, OpenCode). The announcement time is recorded.
T+0 to 2 h
Setup. The current CLI version is pinned. The new model and the model it replaces are run with the same CLI version, the same harness commit and the same machine.
T+2 to 20 h
Runs. The full default suite, at least 5 runs per task, interleaved. Failures are checked against their diffs before publishing, and any failure caused by the harness or the environment is reported, not hidden.
by T+24 h
Publish. The report goes into data/results/ with lab.launch set, run records and diffs are linked, and the publication time is recorded below.

If 24 hours is not enough to finish the runs, the result set is published late and the delay is shown. Launch-day numbers follow the same rules as every other result: intervals shown, no ranking when they overlap.

For shipping a companion repository on the same clock, see launch-day-kit.

5.2 Follow

Follow along or run it yourself

Watch the repository on GitHub: each launch-day result set lands as a commit to data/results/.
Run the same suite on a new model yourself with rerun-bench and submit the report.

5.3 Log

Launch-day log

No launch-day results yet.

The first one will appear here after the next major model launch, with the time from announcement to publication.