One Model, Every Local Stack

Launch80 pointed every local inference server they could run at the same Qwen 35B model on one M4 Max, replaying an identical 18-turn coding session from a cold cache. Then they tested four coding agents against a single shared server.

The servers

• mlx_lm.server, the bare runtime everything else wraps, already does about 90 tok/s — the honest baseline.
• ctrl8d, Launch80's own fork of mlx-vlm, combined fast first tokens with fast streaming and finished the session in 42 seconds, against 70 and 142 for the two stock modes. Under four concurrent sessions its worst-case wait was 2.9 s.
• vMLX crashed on every full run; LM Studio could not load the model's architecture; vLLM would not install on Apple Silicon.
• llama.cpp and ollama ran a GGUF build of the same model — a cross-format comparison — and landed slower, with ollama handling concurrency poorly.

The agents

Sharing one backend, every agent streams at the same speed — that is the server's number. What differs is overhead: one agent spent about 16× the prompt tokens of another on the identical task, and one never emitted the tool call to write its file. The report lists the four mistakes caught mid-run that would otherwise have skewed the results, and asks to be read narrowly: one machine, one model, one sitting.