Opus 5 vs GPT-5.6 vs Grok 4.5: Four Real Build Tests

Anthropic shipped Claude Opus 5 on July 24th, and the timing was not subtle.

People had been expecting something. Fable 5 was the flagship, but it costs $10 in and $50 out per million tokens, and over the past couple of weeks GPT-5.6 Sol, Grok 4.5, Kimi K3 and Qwen 3.8 all landed close enough on quality that paying Fable prices for everyday work stopped making sense. When your competitors are almost as good for a fraction of the money, the thing you need isn't a smarter model. It's a better deal. Opus 5 is exactly that: $5 in, $25 out, and Anthropic's own pitch is that it comes close to Fable 5's intelligence at half the price. Axios noted it's the fourth Claude 5 release in under two months, which tells you how the race has shifted from capability to economics.

So the question isn't really "is Opus 5 good". It obviously is. The question is whether it's better than what everyone else is charging less for.

I ran it against GPT-5.6 Sol and Grok 4.5 on four builds: an art gallery site, a working spreadsheet app, an image codec written from scratch in Go, and an audit of my own open source agent framework. Wall clock times recorded, no interventions, one prompt each.

Short version: Opus 5 wins on points, GPT-5.6 Sol is right behind it and beat both on the hardest test, and Grok 4.5 is fast enough that the quality gap matters less than you'd expect.

The two web builds are up at llm-test-0726.thomas-wiegold.com if you'd rather form your own opinion before reading mine. Design is the one thing you shouldn't take my word on.

Why these four tests

My old test course is dead, and Opus 5 is not what killed it. It had been dying since Fable 5.

The set was four prompts I'd used for about a year: two landing pages, a poker simulation in Go, and an audit of thomas-wiegold.com. The Qwen 3.8 Max review was the last one I ran on it, and the cracks were showing there already. Frontier models passing everything is fine. Open weight models passing everything at the same level is the end of the road, because at that point I'm comparing font choices and squinting at padding, which is taste with extra steps, not evaluation.

Claire Vo, writing her Opus 5 review in Lenny's Newsletter, calls this an "intelligence overhang". When capability saturates on the tasks you actually hand these things, the variables that matter become effort, latency and price. So the new tests had to be harder, and they had to resemble work someone would pay for. Here's what I landed on.

Art gallery website. Open creative brief, go all out, no spec and no wireframe. Not a challenge, and I won't pretend otherwise. Every current model passes. It's a pure design test: does the model have taste of its own when you refuse to give it constraints.

Spreadsheet web app. Excel-inspired, React, fully functional. Formulas, cell references, selection, undo. I expected this to be the hard one. It wasn't, which I'll get to.

Image codec in Go, from scratch. Beat PNG on compression, ideally approach WebP. No libraries doing the real work. This is the actual intelligence test of the four, and the only one where a model has to invent something rather than assemble something.

Audit of agentiny. My open source agent framework, swapped in for the website audit because thomas-wiegold.com is tuned to death and returns a polite list of nothing. Agentiny is young and has real headroom.

Same prompt to each model, wall clock from submit to final output, no nudging. If a model went off the rails, that's the result. Standard caveat: n=1 per model per test, so what I'd defend is the shape of the results, not the decimals.

The four builds, model by model

Test 1: Art gallery site, go all out

Nobody fails this one. That's the point of including it. Every current model can build a decent gallery site, so what's left to measure is taste, and taste doesn't show up on a leaderboard.

Grok 4.5 finished in five minutes. Opus 5 and GPT-5.6 Sol both took roughly half an hour.

That's a six-fold difference and my first instinct was that Grok had cheated somehow. It hadn't. The site was good. Clean, modern, nothing embarrassing, and if you showed it to a client without context they'd probably sign off on it.

But put it next to the other two and the extra 25 minutes are visible. Both Opus and GPT produced something more considered. Opus went deeper on detail: a horizontal scroll section in the middle that actually worked properly, and it shipped a light theme without being asked, which is the kind of thing I usually have to go back and request.

GPT-5.6 Sol went the other way. Simpler layout, but genuinely artsy in a way that suited the brief. The typography was excellent. If I actually owned a gallery, I think I'd pick the GPT site, and that surprised me because on paper it did less. All three are live on the results site, and this is the test where I'd most expect people to disagree with me.

Takeaway: on open creative briefs, thinking time converts into quality. Hold that thought, because it doesn't hold for long.

Test 2: Excel-style spreadsheet app in React

Grok 16 minutes. GPT 31 minutes. Opus just under 50.

All three shipped a working, usable, cut-down Excel. One shot each. I genuinely did not expect that, and I'd designed this test assuming it would be the one that separated them. Formula evaluation, cell ranges, keyboard navigation, all of it functioning on the first attempt. "Build me a spreadsheet app" was a joke prompt eighteen months ago. Now it's a Tuesday afternoon. They're on the results site too, and unlike the gallery test these you can actually poke at until something breaks.

Opus 5 was the best of the three. Most detailed UI, most thought given to the fiddly interaction bits, the one I'd want to keep working on. GPT was the weakest, thinner on UI polish and with genuinely odd undo behaviour that I'd have had to fix before showing anyone.

Grok landed almost exactly where Opus did. Not quite, but close enough that I had to look hard to find the gaps. And it did it in a third of the time with a fraction of the tokens.

So who wins? Opus produced the better artifact. But if I'm paying for this, Grok delivered maybe 95% of the result at a third of the cost, and I think that's the winner on any measure I'd defend to a client. This is also the test that breaks the pattern from the gallery site: more thinking time did not produce a proportionally better app. It produced a slightly better one, expensively.

I'm keeping this test even though it failed at the job I designed it for. It doesn't separate the models on capability anymore, but the spread on time and tokens is enormous (16 minutes to 50), and the small details are where you see who was actually paying attention. Undo behaviour is not a headline feature. It's exactly the kind of thing that quietly determines whether the code is worth keeping.

Test 3: Image codec in Go, from scratch

This is where it got interesting, and where the times inverted completely.

GPT-5.6 Sol: 21 minutes. Grok 4.5: 33 minutes. Opus 5: one hour and twelve minutes.

Opus spent nearly four times as long as GPT on this. Here's what everyone produced.

GPT won, clearly. Real compression, genuinely good results on simple images with flat colour regions and clean edges. It struggled on busy, complex, noisy photos, which is roughly PNG's own failure profile, so it didn't hit the WebP target. But it was the only one that produced something I'd call a working codec, and it did it in 21 minutes. That's the single most impressive result of the whole course.

Opus was second. It compressed. Files came out smaller than the input, the thing worked, but it was well behind PNG and never in the running for WebP. Seventy-two minutes for a codec that loses to a format from 1996 is a rough trade.

Grok failed outright. Not "did worse", failed. Average output larger than the uncompressed source. It produced a program that confidently made files bigger, which is a fascinating thing for a compression algorithm to do.

Now the part that bothers me. Opus 5's headline independent result is ARC-AGI-3, where ARC Prize verified 30.2% against GPT-5.6 Sol's 7.8%, roughly four times better on a benchmark built specifically to resist memorisation and measure novel reasoning. That's the one number in the whole launch that wasn't run on Anthropic's own harness. If any test on my course maps to that capability, it's this one. Invent a compression scheme, no library to lean on, no Stack Overflow answer to pattern match.

And GPT beat it. Not narrowly.

Two honest caveats before anyone quotes me on this. First, the ARC comparison isn't effort-matched: Opus 5's 30.2% was measured at High effort because of the short pre-launch window, while Sol's 7.8% was a Max run. The gap is still huge, but it's not a clean duel. Second, one codec attempt doesn't refute a verified benchmark, and it's entirely possible Opus wins this on a best-of-five. I'm not claiming ARC-AGI-3 is broken. I'm saying a benchmark measuring novel reasoning in the abstract did not predict novel reasoning in Go, on a Tuesday, with my prompt.

Of the four tests this is the one I'm most interested in developing, because it's the only one that hands you numbers instead of opinions. Compression ratio against a fixed image set, measured the same way every time. That's a benchmark, or close enough to one that I could turn it into a real benchmark with a bit of work. I need to experiment with the prompt and the image set first, since right now it's one attempt with a loosely worded target, and the difference between "beat PNG" and "here are twenty images and a scoring function" is the difference between an anecdote and a number.

Test 4: Auditing the agentiny codebase

Time wasn't a factor here. All three were quick, and none of the differences were meaningful. If you want to see what they were looking at, agentiny is on GitHub.

For grading I ran the three audit reports through several other LLMs and asked them to rank the findings by importance and coverage, then compared that ranking to my own read of the codebase. They agreed with each other and with me, which is about as good as blind grading gets when you're the one who wrote the code.

Opus 5 came out on top. It found the most genuinely important problems, the ones I'd actually prioritise. GPT-5.6 Sol was very close behind, close enough that on a different day it might flip. Grok was the weakest. It missed several things the other two caught.

But, and this is the bit worth internalising, Grok also found some minor issues that neither of the others mentioned. Not important issues, but real ones.

This matches something I've said before: run audits across multiple models and consolidate the results. Model quality ranks cleanly, coverage doesn't. Even the model that comes third has non-overlapping findings, because these things all have different blind spots. CodeRabbit's independent eval found the same shape from a different direction, measuring Opus 5 at their cleanest ever precision (39.3%) but with lower recall than their baseline (55.2% against 61.1%) and around four times the nitpicks. Their conclusion was that it's a precision specialist that pairs well with a recall model, not a drop-in sole reviewer. Two very different methodologies, same practical advice: don't use one model.

What the benchmarks predicted vs what I actually got

Here's the honest comparison. On Artificial Analysis, which is the closest thing we have to an independent aggregate, Opus 5 sits at number one on both the Intelligence Index (61 at max effort, against Fable 5's 60 and GPT-5.6 Sol's 59) and the Agentic Index (55.3 against GPT's 54.0).

Look at those margins. One point. One point three. And that 61 is a max-effort number, which is not the setting most people will actually run, so the medium-effort score of 56 is probably closer to what you'd feel day to day.

My results have exactly that shape. Opus edges ahead across the four tests, but not by much, and not on every one. A one point aggregate lead is not something you feel in daily work, and my course confirmed that more than it challenged it. Where the benchmarks held up: the audit and the spreadsheet, both won by Opus, both by a margin small enough to argue about. Where they misled me: the codec, badly.

A caveat on the rest of the numbers. Anthropic's announcement leads with SWE-bench Verified at 96.0% and Frontier-Bench at 43.3%, and those are impressive, but most of the headline coding and agentic figures are vendor-run on Anthropic's own harness. Agentic benchmarks are notoriously harness-sensitive. Treat cross-model gaps as directional. Even the Artificial Analysis suite, which is the cleanest signal in the set, was co-run with Anthropic pre-release.

Then there's cost, which for me is the actual story. Opus 5 is $5 per million input tokens and $25 output, exactly half of Fable 5 at $10/$50, while Artificial Analysis measures it at comparable intelligence for 26% lower cost per task. I liked Fable 5 when it launched. I don't think it's worth the money anymore, and Opus 5 is the reason.

One thing to budget for: thinking tokens bill as output, adaptive thinking is on by default now rather than something you switch on, and Artificial Analysis flags Opus 5 at max effort as very verbose with a time to first token of nearly 70 seconds. The rate card didn't change. Your bill might. That 72 minute codec run wasn't free, and the effort dial (low, medium, high, xhigh, max) is a bigger cost lever than the model choice.

Verdict: which model for which job

Opus 5 and GPT-5.6 Sol are close. Both are a clear step above Grok 4.5 on quality. Grok is meaningfully faster and cheaper. That's the whole comparison in three sentences.

But the useful version is more specific than that.

The speed/quality trade depends entirely on the task

I like fast. I'm impatient, and a five minute turnaround changes how I work in a way that a marginally nicer output doesn't. For a large slice of everyday work, Grok is enough. Landing pages, CRUD, refactors, the stuff that makes up most of a week. My longer Grok 4.5 review goes into where it holds up.

Reach for Opus or GPT when the task needs depth, novelty or judgment. And keep the codec test in mind as the cautionary case, because Grok didn't produce a worse codec, it produced a non-codec. The failure mode of a cheaper model on genuinely hard work isn't "slightly worse", it's "confidently wrong", and you have to be able to tell the difference.

The planner/coder split I want to try next

The obvious experiment: Opus 5 or GPT-5.6 Sol plans and reviews, Grok writes the code. You buy judgment where judgment matters and volume where it doesn't.

I haven't tested this. It could easily fall apart on context handoff, which is its own discipline and something I've written about in context engineering for AI agents. But the audit result is suggestive, because the models that graded best were the ones that spotted the important problems, and that's exactly the reviewer role.

Where this leaves Anthropic

Opus 5 was necessary. With GPT-5.6, Grok 4.5, Kimi K3 and Qwen 3.8 all shipping in the same window, Anthropic without Opus 5 would be behind on price-performance rather than narrowly ahead on it. Releasing it was the right call and the pricing is the reason, not the benchmarks.

The broader point I keep coming back to: most of what we do every day doesn't need the intelligence ceiling. Paying for the ceiling on routine work is waste. That applies to Fable 5, and increasingly it applies to Opus 5 and GPT-5.6 Sol too, when a cheaper model would have done the job.

Next on the list is the same four tests against Kimi K3 and Qwen 3.8 Max. I left them out here purely because this took a full day and I ran out of one. A Chinese model comparison is its own article, and depending on timing it might end up including DeepSeek V5 too.