Same score, opposite temperament: mimo-v2.5-free lands three on swebench-live by grinding, not quitting

Reproducibility header: tool=tomo, engine=oi (code-as-action), model=opencode/mimo-v2.5-free (free tier via the opencode.ai/zen upstream), suite=swebench-live, tasks=all fifteen. Reproduce command, per task:

source ~/data/.local.env   # OPENCODE_API_KEY for the zen upstream
scripts/campaign_sweep.sh <task-id> \
  --models "opencode/mimo-v2.5-free" \
  --max-rounds 30 --timeout 600s --retries 2

This is the second free zen model on the fair board. The first was deepseek-v4-flash-free, which came out at 3 of 15 once the abort-aware harness stopped scoring free-tier infrastructure deaths as task failures. mimo-v2.5-free gets the same treatment and the same headline number, 3 of 15, but almost everything underneath is different.

The board

Fifteen tasks, one attempt each after aborts are retried out, graded by each task's hidden check.sh. Cost is zero because the free tier is unpriced; the honest cost signal here is tokens, and mimo spends a lot of them.

task                              verdict  rounds  actions  in_tok   out_tok  secs    edited == gold?
beeware__briefcase-2085           fail       7       5      21884     994     32.7    no edit
conan-io__conan-17123             PASS      15      13      72232    6558    139.3    yes
aws-cloudformation__cfn-lint-3798 fail      17      15     105309    9840    180.1    no edit
cyclotruc__gitingest-94           fail       6       4      15049    1557     35.5    no edit
dynaconf__dynaconf-1225           fail      30      30     341241    7003    216.2    no edit
fonttools__fonttools-3682         PASS      19      17     161057    4252    113.8    yes
huggingface__smolagents-285       PASS      16      15      81946    6565    152.0    yes
instructlab__instructlab-2540     fail      21      19     190671    4157    112.1    both gold files, wrong fix
joke2k__faker-2142                fail       3       1       7607      338     15.1    no edit
kubernetes-client__python-2303    fail      24      22     210362    7051    151.9    no edit
projectmesa__mesa-2394            fail       5       3      10922      507     19.0    no edit
python-control__python-control-1064 fail    30      30     287774   10211    336.3    no edit
reata__sqllineage-661             fail      13      12      43096     1723     93.8    wrong file
sphinx-doc__sphinx-12975          fail      16      14      80291    21902    330.0    no edit
stanfordnlp__dspy-1651            fail      30      29     323624   35262    532.1    no edit

Solved 3 of 15. Total tokens 2,070,985 (input 1,953,065, output 117,920). That is 3.3 times the 624,323 tokens deepseek-v4-flash-free spent for the same three solves.

The three passes

task              edited file (== gold)                               rounds  actions
conan-17123       conan/internal/api/config/config_installer.py         15      13
fonttools-3682    Lib/fontTools/ttLib/reorderGlyphs.py                   19      17
smolagents-285    src/smolagents/local_python_executor.py               16      15

conan and fonttools are the two shared with deepseek, the well-localized single-file transcription tasks, and mimo hits the exact gold file on both.

smolagents-285 is the one mimo owns. deepseek-v4-flash-free failed smolagents with zero actions, reading the prompt and producing prose instead of a first action, a non-start. mimo drove sixteen rounds and fifteen actions into src/smolagents/local_python_executor.py, the gold file, and turned the hidden test green. This is the persistence dividend in a single task: where the early-quit model never started, the grinder found the file and the fix.

The mirror-image failure: gitingest-94

The flip side is the one task that makes mimo look worse than its twin.

gitingest-94 is the easiest task on the suite, the one whose issue report all but names the file and the symbols. deepseek-v4-flash-free cleared it in three rounds. mimo bailed: six rounds, four actions, thirty-five seconds, no edit written, and a clean recorded fail rather than an abort, so the harness did not retry it because there was nothing transport-level to retry. On the single task where reading the report and transcribing the fix is nearly free, mimo read, poked around, and quit without editing. That is the anomaly in an otherwise grind-everything profile, and it is exactly the task that costs it a shared-plus-one board against deepseek.

The near-miss: instructlab-2540, both gold files, wrong fix

instructlab-2540 is mimo's version of the right-place-wrong-fix failure.

It is a two-file gold task, and mimo edited both of the gold files: src/instructlab/configuration.py and src/instructlab/model/chat.py. Nineteen actions across twenty-one rounds, landing edits in precisely the two files the gold patch touches, and the hidden test still failed. So the localization was perfect and the fix was not. This is the harder cousin of deepseek's briefcase near-miss: a multi-file change where finding both spots is the easy part and making them jointly satisfy the tests is the wall.

What the persistence buys, and what it costs

The defining trait of mimo on this suite is that it does not quit. Look at the no-edit failures where deepseek walked away in one or two rounds: dynaconf (mimo 30 rounds, 30 actions, 341K tokens), python-control (30 rounds, 30 actions, 287K tokens), dspy (30 rounds, 29 actions, 323K tokens, 532 seconds), kubernetes-2303 (24 rounds, 22 actions, 210K tokens). deepseek treated several of these as non-starts and spent a few hundred tokens each. mimo drove every one of them to or near the thirty-round ceiling.

That persistence bought exactly one extra-hard solve, smolagents, that the early-quit model missed. It did not buy the medium tasks: dynaconf, kubernetes, python-control, dspy, sphinx all end no-edit or wrong despite the full round budget. And it cost 3.3 times the tokens for the same final count. On a priced tier that difference would be real money; on the free tier it is real latency, dspy alone ran nine minutes.

Two free models, same score, opposite failure modes

Putting the two fair boards side by side:

task               deepseek-v4-flash-free   mimo-v2.5-free
conan-17123        PASS                     PASS
fonttools-3682     PASS                     PASS
gitingest-94       PASS                     fail (bailed, no edit)
smolagents-285     fail (non-start)         PASS
everything else    fail                     fail
board              3/15                     3/15
tokens             624,323                  2,070,985

The shared core is the two easy single-file transcription tasks. Past that, the two free models are almost caricatures of the two failure temperaments: deepseek is cheap and gives up, so it takes the trivial gitingest and whiffs the one that needs a few rounds of engagement; mimo is expensive and refuses to give up, so it takes the engagement task and, oddly, drops the trivial one.

Both hit the same hard ceiling the wider campaign already mapped. The six-task wall (cfn-lint, dynaconf, sqllineage, instructlab, sphinx, python-control) is unbroken by either free model, and a gpt-5.6 codex sweep fails the same six. The takeaway for routing free models is that the two are not redundant despite the identical score: deepseek is the one to reach for when you want a cheap first pass that either lands fast or costs almost nothing, and mimo is the one that will burn tokens to find a fix a cheap model would never start looking for, with no guarantee the extra tokens convert.

Metrics: solved 3/15, total 2,070,985 tokens (input 1,953,065, output 117,920), cost $0 on the free tier, engine oi (code-as-action, graded by hidden check.sh), aborts retried up to two times per task, all three passes edited the exact gold file.