The blind vote preferred showtime 0.3.0 (lean) over 0.2.0 on 4 of 6 tasks; the AI judge preferred 0.2.0 on 5

t9 (explain a repo) and t10 (make it like a reference) are new in round 4 and were run with 0.3.0 lean and plain Opus 5.5 only. The second 0.3.0 lean run on t1, t2 and t8 was judged but kept off the boards, to keep the vote short.

What the judge held against 0.3.0 lean

• Still holds. On the launch video, three stretches of 3.6 to 4.6 s with no change on screen (qa flagged them too) and an empty terminal beat. On the data story, the ranked bar chart stays on screen for about 15 seconds.
• A blurred crossfade. On the HTML report, one sampled frame caught two scenes blurred over each other mid-transition.
• A missing label. On the same report, the "first year above 400 ppm" callout does not say which year. The 0.2.0 page says 2015.
• Crowded moments. On the math explainer, equations overlap into a jumble during a transition around 35 s. On the vertical short, a left-heavy layout leaves the right side and bottom half empty.
• Less content. On the launch video, fewer of the 2.0 features; on the data story, a thinner story than 0.2.0's four beats.

These are polish defects, the kind a separate review of the finished video exists to catch. The lean mode in round 4 skipped that separate critic round to save cost. That is why the shipped default became the quality mode, which runs a critic round on every finished video. Whether it removes these defects has been checked on one proof run only, not in a blind round.

Round 3 had a real isolation failure: two of its runs used another working copy of showtime. The isolation check was added for round 4 to catch that; it found nothing once its own bug was fixed.

Other process numbers from the stream, for 0.3.0 lean: a median of 4.5 images read into the main agent's context per job and a median of one full render. The two runs of the same task were close in cost ($4.01 and $3.81 on t1, $1.42 and $1.71 on t2, $1.45 and $1.70 on t8).

The contenders

How each new run worked

1. A fresh workspace and a fresh Claude Code setup per run, with its own config folder and home, on one Linux cloud machine, three runs at a time.
2. The same everything else. Model claude-opus-5-5 at high effort, Claude Code 2.1.283, a 25-minute cap, a $15 budget cap, the same tools on the PATH and the same standard reply to any question. No run asked a question and no run hit a cap.
3. Isolation checked after the fact. Every command and file read of every new run was scanned for another showtime folder (round 3's failure). See Release checks.

How the outputs were scored

The two new tasks

Both inputs were written for the benchmark. t9 uses a small working command-line tool (a README, a changelog, four modules and tests) that catches drift between .env and .env.example. t10 uses a 20-second reference drawn with ffmpeg only (flat colour, one open-licence typeface, a synthesised beat) and an event description. showtime 0.3.0 has a repo-explainer workflow and a reference workflow that plain Opus 5.5 does not; that is what these tasks test.

• Quality is the default review mode. A critic round on every finished video; lean is an explicit opt-in. Measured once, on t2, at $3.03; not voted.
• Data charts and HTML reports. From the t2 and t6 notes: numbers that vanish and reappear as items are added, and pacing that felt slow and not dynamic. The fixes also cover the judge's points: no blur over text, labels that carry their year or value, no 4-5 s holds.
• The "like a reference" route (t10). The reference route is now required for such requests: a measured spec of the reference (cuts, timing, palette, text sizes, sound hits), a keep-or-change list, and a frame-by-frame comparison with the reference.

• Lean, not the shipped default. Round 4 measured 0.3.0 in lean mode. The shipped default, quality mode, has one proof run on one task and no vote.
• One run per cell. Two for 0.3.0 lean on t1, t2 and t8, and those two runs were 1.33 to 1.67 places apart in the judge's mean rank (t8: 3.00 against 1.33). Any single task can flip on another draw.
• One human voter, who is showtime's author, and only partly blind. Letters were shuffled and nothing named a tool, but the reviewer had seen the reused 0.2.0, plain Opus 5.5 and brag-slim videos in earlier rounds and could recognise them. On t1 the vote on those unchanged videos moved from round 3. A claim that matters needs voters who built none of the tools.
• Reused cells. 0.2.0 default comes from round 3; plain Opus 5.5 (six tasks) and brag-slim from round 1, same prompts, made on other days under other load. Only showtime 0.3.0 and the two new plain Opus 5.5 cells were run in round 4.
• The judge's limits. It sees still frames, a transcript and measurements; it cannot watch motion or hear voice and music. Its three judgments per task are one model looking at rotated copies of one packet. The proof shows it opened and read the images, not that it judged well. The judge model is the same family as the agents that made the videos.
• The fact-check judge makes mistakes. Both claims it flagged in 0.3.0 lean's vertical short are stated in the README, a listed source; one of the two in the repo explainer is real.
• t9 and t10 have fewer comparisons. Two versions each and one pair each; no 0.2.0 and no outside tool. The t1 and t10 boards have no ratings.
• Costs are API-equivalent estimates from the stream, at API prices. The runs used a subscription. Reused cells keep the cost and time of their own round.
• The same author wrote the tasks, the harness, the scorers and the release checks, and showtime qa is the tested tool's own checker, applied the same way to every file. Nobody independent has reviewed the method.
• Bundled skills and the network. Claude Code's own skills stayed installed for every contender, no one had a cloud key, and network access was not blocked.
• Earlier rounds are in the v0.2.0 report, with their own limits.