Qwen3.8-27B: the best local LLM you probably can't run

Alibaba dropped two models this month, and the one everybody wrote about was the wrong one. Qwen3.8-Max is a 2.4 trillion parameter API product, and I reviewed it a couple of weeks ago with a fair amount of enthusiasm. But the release I was actually waiting for landed on August 14: Qwen3.8-27B, Apache 2.0, weights on Hugging Face, small enough to run on a laptop.

I need to be upfront about something. I did not run my full test suite on this model. I tried, got about eight tokens per second on my M4 MacBook Air, and lost patience somewhere around the second prompt. That failure is most of what this post is about, because I suspect a lot of you are about to have the same evening I had.

What Qwen3.8-27B actually is

27.78 billion parameters, dense, including a vision encoder that nobody had announced in advance. It takes text, images and video in. Native context is 262,144 tokens, extensible to roughly a million with YaRN scaling if you're running it on a server.

The architecture is the genuinely interesting part, and it isn't a plain transformer. Across 64 layers, Qwen interleaves 48 Gated DeltaNet layers (linear attention) with 16 full Gated Attention layers, in a 3:1 ratio. Only those 16 full-attention layers keep a KV cache. That works out to about 64 KB per token, roughly a quarter of what a conventional 64-layer dense model would want.

If you only remember one number from this post, make it that one. Everything about whether this model fits on your machine comes back to KV cache growth.

Thinking mode is on by default, with a reasoning_effort knob set to xhigh out of the box. You can drop it to medium or low. You will want to. More on that shortly.

The benchmarks, and why I'd discount them

Qwen's own table looks great:

Benchmark	Qwen3.8-27B	Qwen3.6-27B
SWE-bench Pro	61.7	53.5
LiveCodeBench v6	90.3	83.9
GPQA-Diamond	89.2	87.8
Terminal-Bench 2.1	73.0	63.4
DeepSWE 1.1	42.2	13.3
OSWorld-Verified	84.3	63.9

Jump from 13.3 to 42.2 on DeepSWE is not a rounding error. The generational gain over Qwen3.6-27B looks real and it's consistent across the whole table rather than one suspicious spike.

Now the part that made me rewrite my first draft. Every one of those numbers came from Qwen. Some of the benchmarks are in-house, some have corrected ground truth, and the competitor scores in their comparison were imported rather than re-run. As of the time I'm writing this, Artificial Analysis hasn't published an intelligence index for the 27B, and no lab has reproduced anything.

I originally wanted to write "we now have Opus 4.6 level intelligence running on consumer hardware." Then I read Qwen's own table properly. Opus 4.6 Max is ahead on Terminal-Bench 2.1 (78.2 to 73.0), ahead on GPQA-Diamond (91.3 to 89.2), and comfortably ahead on Humanity's Last Exam (40.0 to 30.8). The 27B wins some rows and loses others, on a scorecard the vendor produced. "Beats Opus" is a headline, not a finding.

What's true is still remarkable: a 28B model you can download for free is trading blows with a frontier API on coding benchmarks. That's the story. It doesn't need inflating.

What it feels like to use

I did play with it, just not systematically. It's very good. The benchmark direction matches the vibe, which is the most I'm willing to claim from an afternoon of poking at it.

Simon Willison, who has been running his SVG pelican test on everything for two years now, called it the best pelican he's seen from a model that runs on his laptop. He also noted the run took nearly 21 minutes and burned 22,276 reasoning tokens to produce 3,223 tokens of output. On an M5 Max. That is the model in one anecdote: strong, and exhaustingly thorough about it.

The most common complaint in the first week wasn't quality, it was verbosity. At default xhigh the thing meanders. People running it in agent loops report 10k tokens where a competing model uses one. There are also chat template bugs, including a fatal exception if you pass enable_thinking=false, so grab a community-fixed template before you wire it into anything serious.

Can you run it? The hardware maths

Short version: you need about 17 to 18 GB for a 4-bit quant, plus KV cache on top. 24 GB of unified memory is the practical floor, 32 GB is where it gets comfortable, and 48 GB is the value sweet spot.

Here's the Unsloth GGUF table, which is the one everyone is working from:

Quant	Size	Notes
UD-Q3_K_XL	13.4 GB	emergency option for 16 GB
Q4_K_M	17.1 GB	standard 4-bit
UD-Q4_K_XL	17.9 GB	the default I'd pick
Q5_K_M	19.8 GB	needs 32 GB
Q8_0	29.0 GB	near-full fidelity
BF16	54.7 GB	reference

Add roughly 0.93 GB for the vision projector if you want image input, because it isn't bundled with the language GGUF. And don't go below Q4 on this model. Multiple people reported quality falling off a cliff, more than they expected for a 27B, and aggressive KV cache quantization broke at least one person's private eval outright.

KV cache maths at 64 KB per token: 8K context is about half a gig, 32K is 2 GB, 128K is 8 GB, and the full 262K is 16 GB on its own. Start at 8K and raise it while watching memory pressure. The million-token YaRN context is a server feature and static YaRN hurts short-prompt quality anyway.

The Apple Silicon specifics

Three things that surprised me.

First, Ollama's fast MLX backend requires more than 32 GB of unified memory, so a 24 GB Air can't use it at all. You're on the GGUF path: LM Studio if you want one-click, llama.cpp if you want control.

Second, that matters less than you'd think, because the "MLX is faster on Apple Silicon" rule mostly applies to small models. The MLX advantage is large under about 14B and collapses above 27B, where you become memory-bandwidth-bound and the runtime stops being the bottleneck. So pick GGUF and stop worrying about it.

Third, thermals. The Air is fanless. Sustained inference pins the GPU and the chassis heats up, and testing converges on roughly 60 to 70% of peak after eight to ten minutes of full load. For a chat session you'll never notice. For a long refactor you absolutely will.

Where I hit the wall

My M1 Mac Mini has 16 GB. It is not a host for this model, full stop. The 4-bit build needs 17-something gigabytes and the machine just swaps itself into uselessness. I run a 4B coder there and that's fine.

The M4 Air has 24 GB and it does load. It runs. It also produces fewer than ten tokens per second once you're a few thousand tokens in, which sounds survivable until you combine it with a model that wants to spend 20,000 reasoning tokens thinking about your question. I sat there watching text appear at reading speed and decided I had better things to do with my afternoon. The only measured number I've seen for base M4 silicon on this model is 5 to 6 tokens per second on a 32 GB Mac Mini, so I don't think I got unlucky.

Here's the part I'm actually annoyed about. Two years ago I looked at upgrading and talked myself out of it. The reasoning felt sound at the time: local models weren't good enough to justify the spend, and Apple had new chips coming. Both of those things were true. What I got wrong was the ordering. The models got good faster than the hardware got cheap, and now memory prices are ugly and the machine I want costs more than it did when I didn't want it.

If you're weighing this up now, the lesson generalises. Buy the RAM before you can justify it. With local inference the useful model always shows up before the affordable machine does, and you can't add unified memory later.

Who should actually buy hardware for this

Privacy is the whole argument, and it's a strong one. Apache 2.0 weights running fully offline is a different conversation with your compliance people than a hosted API's terms of service. If you're a doctor, a lawyer, an accountant, or anyone else who handles material that legally cannot leave the building, this is the first time the local option has been good enough to be boring rather than a compromise. I do a fair bit of AI consulting work in Sydney with exactly those professions, and the conversation has changed completely in about six months.

For that use case I'd buy hardware today. A maxed Mac Studio, a 48 GB M4 Pro Mac Mini (best value Mac for this specific model, and a desk-bound machine dissipates heat properly), a DGX Spark, or an RTX 5090 if you're on the Nvidia side. The 5090 numbers are in a different league: 200-plus tokens per second with NVFP4 and speculative decoding, versus my sad little eight.

What I wouldn't do is pretend this replaces a hosted API for everything. It isn't fast enough for long agent loops or batch work on a laptop, and the hardest 5% of tasks are exactly where a frontier model still earns its subscription. The pattern that works is hybrid. API for the difficult things, local for volume and for anything that can't leave your machine.

Why this is Apple's opening

Apple has had a rough couple of years in AI by every measure that gets written about. But they own the layer local inference actually needs, which is a lot of fast unified memory in a machine normal people already buy.

Think about what happens if the current trajectory holds. If a local model handles 95% of what you do in a day, what's the marginal case for a monthly subscription? I don't think the answer is "none," because that last 5% is where the value concentrates and it's the part you'd pay for anyway. But it changes the shape of the market a lot, and it changes it in a direction where the company selling you 128 GB of unified memory does very well.

The open question is memory prices. Either the current buildout overshoots and we get a glut, or the datacentre demand keeps eating supply and consumer RAM stays expensive for years. I'd bet on the glut, mostly because everyone building capacity right now is assuming everyone else won't. That bet has a poor historical record for the people making it and a good one for the rest of us.

Where I land

Qwen3.8-27B is the most important open release of the year if you have the machine, and a preview if you don't. I'm in the second group and it's my own fault.

Two things would change my read. Independent reproduction of those benchmarks, which nobody has done yet and which I'd want before repeating any of the vendor numbers as fact. And a quantization or a speculative decoding build that gets a 24 GB Mac to twenty tokens per second, at which point I'd stop writing about this model and start using it.

If you've got 32 GB or more, go get it. Set your context to 8K, use the official thinking-mode sampling parameters (temperature 1.0, top_p 0.95, top_k 20), drop reasoning_effort to medium, and tell me if it's as good as I think it is.