Baking a 27B Coder Model Down to INT4 on One Arc Pro B65
Quantizing a 55 GB coder model to fit one 32 GB Intel card, with quality bars written down before measuring — and the tool-calling regression those bars caught before release.
Summary of a report from Launch80, the NSS research lab. Read the full report →
A 55.6 GB coder model, one Intel Arc Pro B65 with 32 GB of memory, and no existing 4-bit build to start from. Launch80 quantized the model body themselves, then baked the output and draft heads to INT4 — and wrote the quality bars down before measuring anything.
Getting it to fit
The first obstacle was host RAM, not the GPU: memory grew about 1 GB per layer from allocator fragmentation, which would have run out halfway through a 64-layer model. Tuning glibc's malloc arenas flattened it, and the body quantized in a little over three hours. The finished checkpoint is 16 GB, and every baked layer beat plain round-to-nearest quantization on held-out data.
The regression the bars caught
Six of seven tasks held, but tool calling (BFCL) dropped 8–10 points. The quantized model was ending its turn after the first of several parallel tool calls. The cause was the calibration set: web text, code, and chat, with not a single tool call in it. Recalibrating with 20% function-calling conversations brought BFCL back to within 0.75 points of full precision. The report flags that fix as a post-hoc change, not part of the pre-registered plan.
Where it landed
- Released as v1.1.1 on Hugging Face; all seven tasks within the bar.
- 75.8 tok/s at 262k context with the MTP head; 92.5–96.0 tok/s at 128k with an INT4 DFlash2 drafter.
- Two bars still fail and are reported as fails: top-1 agreement (92.84% against 93%), and a comparison with a community Q4_K_M build that stays closer to full precision.
Read the full report on Launch80
Methodology, every table and chart, and the caveats this summary leaves out.
launch80.com/blog/qwopus-coder-int4-bake-study →