P-M4-3: the infra column shows tier structure, and grades ignore it
The infra column shows tier-correlated structure, and no arm's grade verdict changes when capped fails are excluded versus included.
id: P-M4-3
filed: 2026-07-17, from spec 2105 doc 10 section 8.5, before the M4 full gate
suite: every suite the gate runs
tier: all tiers
engine: every arm on the sheet
band: floor: rate-limit caps concentrate on the free tier and stream behavior
differs by provider (the infra column shows real structure, not noise).
ceiling: no arm's grade verdict changes in a sensitivity check that excludes
versus includes capped fails.
mechanism: caps are properties of the provider weather, not of the agents, so they
should correlate with tier and provider; and because the report already
keeps capped fails out of every rate, re-including them should only be able
to move a verdict if the separation machinery is broken.
settled by: the M4 full gate report's CAPPED, 429S, and DROPS columns, plus a
sensitivity recomputation over the same result set.
go run ./cmd/lab report && go run ./cmd/lab report --suite swebench-live
result: (open)
verdict: (open)
This prediction is a canary for the infra-separation machinery itself. If excluding capped fails flips a verdict, the gate has an infra problem to fix before its verdicts mean anything, and better to have said so in advance.