Editorial intro
A fast, machine-checked formalization of Fermat’s Last Theorem landed this week alongside a clear-headed essay pushing back on a popular shorthand for large language models. Both stories are about how we record and reason about correctness — one in mathematics, the other in how we talk about AI. The technical work matters; so does the language we use to understand it.
In Brief
Stop Thinking of LLMs as Next-Token Predictors
Why this matters now: The blog post "Stop Thinking of LLMs as Next-Token Predictors" argues that labeling models as mere token guessers understates how training dynamics, fine-tuning, and tool use produce goal-directed behavior — an important framing for engineers and regulators today.
The essay argues that the shorthand “next-token predictor” is misleading. The author concedes that pretraining optimizes next-token likelihood, but emphasizes that scale, random initialization, the order of examples, fine-tuning (including RLHF), and systems that chain tools or agents all shape internal circuits in ways that create emergent, sometimes goal-directed behaviors. One useful reframing some commenters offered was thinking in terms of “next-embedding” or seeing pretraining as a compressed fitness function that yields useful generalization.
"Stop Thinking of LLMs as Next-Token Predictors" — the post's blunt title sums up the argument.
Why pay attention? How we describe models changes expectations about failure modes, audits, and regulation. If you treat a model as only a statistical replay engine, you’ll underprepare for behaviors that look intentional, brittle, or high-stakes in deployment.
(Short) Why you should know: formalization progress
Why this matters now: Anthropic says Claude produced a machine-checked Lean proof of Fermat’s Last Theorem, demonstrating that large language models can substantially accelerate the labor of encoding mathematics into verifiable code.
Anthropic published a report claiming its Claude model "worked largely autonomously over 11 days to write the proof in the Lean programming language," producing millions of lines of Lean definitions and tens of thousands of intermediate lemmas, and releasing the repository for inspection. Formal proofs are different from papers: they create a certificate a proof assistant can check, which reduces ambiguity and can catch subtle mistakes that slip through human review.
Deep Dive
Formalizing Fermat's Last Theorem (Anthropic / Claude)
Why this matters now: Anthropic’s report that Claude formalized Fermat’s Last Theorem into Lean in days — if accurate and reproducible — signals a step change in how AI can be used to produce machine-checkable mathematical artifacts and could reshape peer review and reuse in math.
"Claude worked largely autonomously over 11 days to write the proof in the Lean programming language."
What Anthropic describes is not a human-readable paper but a fully formalized, machine-checkable encoding of a deep result. That matters because proof assistants like Lean force you to remove every informal step: definitions, hypotheses, and intermediate lemmas all must be explicit. A machine-checked artifact is a durable certificate that other researchers can build on without re-interpreting prose.
The technical scale Anthropic reports — millions of lines of Lean and roughly 29,500 intermediate theorems — is striking. Encoding modern number theory into a proof assistant is painstaking; historically, projects have taken years. If a model can shoulder the bulk of this encoding, it changes the economics of formalization. It could lower the barrier to producing libraries of formally verified math and help catch errors in published work or in the subtle assumptions encased in long proofs.
There are immediate caveats and community reactions worth noting. Some mathematicians and formalizers welcomed the raw technical achievement but stressed perspective: an 11-day run generating code is not identical to the years of conceptual work mathematicians invest when developing a proof or new theory. Others pointed to concentrated power and cost — who runs these compute-heavy processes, who vets the outputs, and how transparent are the training and prompt methods? Kevin Buzzard and others have been building Lean libraries by hand; automated tooling could accelerate their work or duplicate effort, depending on coordination.
Practically, this is how progress might unfold over the next year:
- Researchers will try to reproduce the result and audit the released repo; reproducibility is the single most important follow-up.
- Proof-assistant experts will inspect whether the formalization leans on brittle automation or on stable, human-understandable lemmas that can be reused.
- Journals and reviewers will have to decide how to treat machine-generated formalizations as evidence — are they admissible as verification, or do they require additional human checks?
The big-picture payoff is straightforward: formal verification increases trust. The immediate worry is governance: fast automated formalizations make it tempting to treat proofs as black-box artifacts unless communities insist on open repositories, reproducible scripts, and human interpretability of how the automation arrived at definitions and lemmas.
Closing Thought
Two threads connect these stories: precision and narrative. Formal proof systems push precision to its limit, producing artifacts that can be checked mechanically. How we describe AI — whether as a “next-token predictor” or as an emergent, tool-using system — changes the narratives we use to judge precision, responsibility, and risk. Both the math and the metaphors are shaping what "trusted" computation will look like next.