AI · guide
How to read a model release without getting played
Benchmark tables are not journalism. Here is a checklist for the next frontier-model dump.
Key takeaways
- Separate training-data claims, eval scores, and product availability.
- If a number has no methodology, it is marketing.
- Safety and licensing terms decide whether you can actually ship.
Start with three piles
- What the weights can do in a lab eval.
- What the product allows in the API today.
- What the company wants you to feel.
Journalism lives in (1) and (2). Put (3) in the kicker, not the lede.
The sentence we refuse to publish
We will not write that a model is “the best” unless we can say best at what, measured how, as of when, according to whom.