Home › Articles › RESEARCH

We Shaped the Report Template Around One Benchmark

A data-only publishing pipeline means the template decides what a result can say. Why Launch80 tuned theirs for BetterBench, and what that costs every other kind of result.

Summary of a report from Launch80, the NSS research lab. Read the full report →

When you publish a result through Launch80's L80 tool you send data — a JSON object — not a page. A template on their side renders it. That makes every published page safe to serve, and it means the template decides what a result is able to say.

Why one benchmark got its own template

BetterBench refuses to report a single average tokens-per-second. It reports distributions — p50 and p99 time-to-first-token, decode medians with spread, 1%-low latency bands — and it marks every percentile that rests on too few samples. The dedicated template keeps those numbers in one table, flags under-sampled rows, and puts caveats above the results.

The generic fallback has no table and no notion of sample size: twelve flat tiles, so a p99 over five runs reads exactly like a solid one. The tool's most careful work is thrown away.

The stated limitation

Two templates exist today. BetterBench output renders well; everything else renders adequately. Adding a template is about three files — a schema, a render function, and a registry line — and because pages render from stored data, improving a template later re-renders every page published under it.

Read the full report on Launch80

Methodology, every table and chart, and the caveats this summary leaves out.

launch80.com/blog/report-template-shaped-around-betterbench →

« All articles  ·  Home