Qwen3.8 27B — 4bit vs 8bit vs Flash Next, and is DFlash2 really lossless?

When running 27B locally, I had always picked 4bit or 8bit on instinct. 8bit is more accurate, so I assumed 8bit was the right call whenever memory allowed. This time I measured it on the same Mac, and it was not what I expected.

While I was at it I added Flash Next (125B MoE) from the previous post and compared all three on the same prompts.

What I compared

All on an M5 Max, same five prompts (code, math, a logic puzzle, a Korean explanation, a factual question), temperature fixed at 0.

• 27B 4bit — oQ4e quantization. Mostly 4bit with sensitive layers kept at 5bit. 16GB of weights.
• 27B 8bit — pure 8bit. 28GB of weights.
• Flash Next 4bit — 125B, but MoE, so only 6B is active at a time.

Memory was the constraint. Loading 8bit (28GB) while Flash Next was holding 70GB reproduces the freeze condition I hit before. So I did not load all three at once — I measured each on its own to remove contention.

Speed

Decode (token generation) speed, in tok/s.

Prompt	Flash Next 4bit	27B 4bit	27B 8bit
code	84.2	39.1	18.1
math	96.2	95.9	18.3
reason	99.4	83.9	18.2
korean	57.9	43.1	18.1
factual	74.7	65.2	18.0
average	82.5	65.5	18.1

8bit is the slowest — by 3.6x. That gap was larger than I expected, so I suspected memory paging at first. Loading 8bit alone with 15GB free and re-measuring gave the same 18. That is the real speed.

I put it down to two things. One, 8bit weights are twice the size of 4bit, so decode has to read twice as much. Decode is a memory-bandwidth fight, and that translates straight into speed. Two, the 4bit (oQ4e) build has speculative decode attached and the 8bit build does not. The evidence is that 4bit speed swings between 39 and 96 depending on the prompt — it jumps on math and logic problems that echo the prompt heavily. 8bit has none of that, so it sits flat at 18 no matter what you ask.

Quality

With speed like that, the only reason to run 8bit would be accuracy. So I went through the answers one by one.

The short version: 4bit and 8bit were effectively identical. Both got the math problem right (60km in 45 minutes → 80km/h), both solved the logic puzzle (position of the red house) correctly, and both produced correct Fibonacci code. Both were accurate on the TCP/UDP difference.

Only the Korean explanation of “quantization” differed slightly, and there 8bit was marginally better — it used an analogy about money existing only in whole-won units with no half-won coin. But 4bit was not wrong either, and a difference this size is about what you would get from a different seed.

So oQ4e’s approach of keeping sensitive layers at 5bit does its job. Most of the model is compressed to 4bit for speed and size, while the parts where quality matters keep their precision — enough that it is indistinguishable from 8bit.

What about Flash Next

The 125B model is faster than the 27B. Being MoE it only runs 6B in practice, and it has speculative decode on top, so it averaged 82.5 tok/s — the fastest of the three.

Quality was good too. On the Korean quantization explanation it was the only one to mention the actual purpose — reducing storage and speeding up processing. The other two explained only the physics definition. Not a large gap, but Flash Next felt most like it knew what it was answering.

The cost is 64–70GB of memory. Against 15GB for 27B 4bit, that is 4–5x.

+) I also tried DFlash2 speculative decoding

Once I realized 8bit was slow for lack of speculative decode, I got curious about attaching a proper speculative method: DFlash2, a block-diffusion drafter from z-lab. It officially claims to be lossless — 3.6x faster with output identical to the original, character for character.

Trap 1 — it does not work through mlx-serve

I lost some time here first. Passing DFlash2 to mlx-serve’s --drafter loads it but never uses it during decoding. Speed did not improve at all and I chased that for a while. Running DFlash2 on MLX requires a separate package, mlx-dspark. And the drafter has to be incoai/Qwen3.8-27B-DFlash2, matched to the base model (mlx-community/Qwen3.8-27B).

The speed gain is real

Wired up properly with the base model plus the incoai drafter, the speedup was unmistakable. All at temperature 0.

	baseline	DFlash2	speedup
8bit average	~15 tok/s	~45 tok/s	3.03x
4bit average	~31 tok/s	~62 tok/s	2.00x

Close to the official numbers (8bit 3.63x, 4bit 2.30x). Coming in slightly under is down to the M5 — mlx-dspark disables one verification kernel (small-M) there over a hardware issue, and the warning kept appearing in the log.

But “lossless” does not hold

This is the real finding. DFlash2’s selling point is producing the same answer as the original while accelerating. Measured, it does not.

At temperature 0 there is no randomness, so any number of runs should be identical down to the character. Baseline was indeed identical across two runs. But turning DFlash2 on and off makes the answers diverge partway through. For both 8bit and 4bit, only 2 of 5 matched the original.

The Korean explanation is the clearest example.

[identical for the first 119 characters]
"Quantization means ... in physics, energy or "

DFlash2 off:  ...the state of a particle appearing only in these minimum units is called quantization.
DFlash2 on:   ...things like charge are also considered to move only in these minimum units.

In principle this should not happen. DFlash2 has the main model verify tokens the drafter proposed, accepting or rejecting them, so as long as verification is exact the output should match the original. But the computation path during verification differs from generating one token at a time, producing tiny numerical differences that at some point flip the next word choice. That is how the MLX quantization kernels behave on M5.

To be clear: the answers are not wrong. Both Korean versions above are correct, and the math and logic answers were correct too. This is “different from the original,” not “incorrect.” What it breaks is reproducibility. Any test or cache that assumes “temperature 0 means the same answer every time” will see the answer move. And more importantly, the official framing of “lossless, so turn it on without worrying” does not hold on this hardware.

For 4bit, oQ4e’s built-in MTP is simply better

One more thing. The oQ4e 4bit build used above has its own MTP head and reached 65–75 tok/s. That is faster than base 4bit with DFlash2 attached (around 62 tok/s), and it requires no separate drafter download. For practical 4bit use, oQ4e with its built-in MTP was the cleanest option.

So DFlash2 delivered the advertised speed, but on this M5 it was not lossless. Good if you need a fast draft; still something to be careful with if you require output identical to the original.

So what do I run

	avg decode	resident memory	quality
Flash Next 4bit	82.5 tok/s	64–70 GB	best
27B 4bit (oQ4e)	65.5 tok/s	15 GB	equal
27B 8bit	18.1 tok/s	27 GB	equal

• I could not find a reason to run 27B 8bit. Same quality as 4bit, 3.6x slower, and more memory — at least with this build. An 8bit build with speculative decode attached would change the picture, but that did not exist.
• If memory is tight, 27B 4bit (oQ4e). 65 tok/s at this quality in 15GB is good value.
• If memory is plentiful, Flash Next. Fastest and smartest, at the cost of handing over 70GB.

“8bit is always better” turned out to be a prejudice. How the quantization is done — which layers are protected — and whether speculative decode is attached mattered far more than the bit count.

(Measured 2026-09-01. Temperature 0, five prompts — a small sample, so read it as a trend.)