Grok 4.5 Trained on the Answer Key

Two days ago I argued that once one entity owns the model, the compute, the coding tool, and the data, it stops having to sit an exam it didn’t write. Grok 4.5 shipped the next morning, out of the merged SpaceX/xAI entity that bought Cursor for $60 billion in June. It’s the first graded exam under that regime. The answer key turned up in the training set, and almost nobody blinked.

The Footnote That Runs the Whole Story

xAI’s launch page is a confident wall of green bars. The single most important sentence about Grok 4.5 isn’t on it. It’s a footnote on Cursor’s.

Grok 4.5 has an advantage on CursorBench because an earlier snapshot of the Cursor codebase was accidentally included in training. That data has been removed for future models.

— Cursor's Grok 4.5 launch post, in a footnote

Sit with what that means. CursorBench grades a model against the Cursor codebase. That codebase was in Grok 4.5’s training data. The test material was in the study material. So they pulled CursorBench from the launch charts, and the reason isn’t that the benchmark was broken. It’s that this specific one got caught.

The tell isn’t the contamination. Codebases leak into web-scale training all the time, and the honest move is exactly what Cursor did: disclose it. The tell is the geography of the admission. The winning charts live on the model-maker’s page. The contamination lives in a footnote on the subsidiary’s page. And the fix, “removed for future models,” quietly concedes that the model you can use today still has it.

The Trick Is Which Chart Gets to Be Big

What xAI leads with instead is token efficiency, and it earns its own hero chart: Grok 4.5 resolves a SWE-bench Pro task in 15,954 output tokens on average against Opus 4.8’s 67,020. Roughly 4x fewer. It’s the headline, it’s the pricing pitch, it’s what Musk led his launch tweet with. And it is true. Grok really is more token-efficient, and at $2 in / $6 out per million tokens it really is cheap.

Now look at what that efficiency chart is standing next to. On the very same SWE-bench Pro, the resolve rate is Grok 64.7%, Opus 4.8 69.2%, Fable 80.4%. Grok loses the accuracy contest and wins the token contest, so the token contest is the one that gets the dedicated graphic. Across the five coding benchmarks xAI chose to publish, Grok wins exactly one outright. Reddit clocked the framing before the press did:

Token efficiency: testing against opus 4.8 max instead of the base opus 4.8 model is hilarious lmao.

— Top reply, r/singularity launch thread

This is the ordinary grammar of a launch: pick the axis you win, draw it biggest. Every lab does it. What’s different here is that the grader and the graded are now the same company, so there’s no outside party whose chart competes for your attention. The efficiency number is the true thing you’re shown so you don’t tally the rest.

Whose Sessions Paid For It

My full-stack post had one open question I couldn’t close: when Musk said Grok trained on “Cursor data,” did that mean a curated corpus or raw developer sessions, given Cursor’s privacy policy promised it didn’t train on user code? The launch answered it.

The Dog That Didn’t Bark

Here is the part that should unsettle you more than the contamination. All of it was disclosed, and nobody cared.

Training on user sessions, a benchmark contaminated by the trainer’s own code, no system card at all where Opus 4.8 shipped with a 246-page one. Every piece of that is on the record, published by the companies themselves. The reaction on r/cursor was jokes about the “SpaceXAI” rebrand. The top line on r/singularity was “Grok continuing the tradition of being competitive for one (1) day.” The privacy scandal these disclosures would justify simply never assembled. There is no thread demanding an opt-out, no post treating the contamination as a scandal, just memes and a mild wave of people switching tools because they’re tired of Musk.

I spent the full-stack post worried that owning the whole stack lets a company opt out of accountability. The launch showed something worse. The accountability didn’t even turn up to be opted out of.

What This Isn’t

The vertical-integration read is sharp, so let me keep it honest:

• Disclosure beats concealment. Cursor printed the contamination itself and says it’s gone for future models. A company purely optimizing spin buries that. Credit the honesty even while noting where they filed it.
• “Accidentally” is entirely plausible. I’m not alleging deliberate benchmark cheating. Cursor calls the impact “unclear,” and CursorBench was never the headline number anyway.
• The model is probably good. The independent signal is real: it leads Snorkel’s professional-work eval and sits fourth on Artificial Analysis’s index, and the efficiency win is genuine. This isn’t “the model is fake.”
• One leaked eval is not the crisis. The crisis is the structure that makes leaks impossible to check from outside, because the same entity owns the benchmark, the training data, the model, and the customers who “eval” it in production.

The Homework Came Back

The previous post ended on a company grading its own homework and reporting an A. The homework came back this week. The answer key was stapled to it, the confession was a footnote on the subsidiary’s blog, and the room applauded the token efficiency.

That’s not a story about Grok being a bad model. It’s probably a fine model. It’s a story about what happens to the checking when one company owns every layer that used to do the checking. The exam still gets graded. It just stops meaning anything, and the quietest proof of that is how little anyone minded.