There is a ritual that runs through AI research like a watermark. Humans build a test, watch a system pass it, and then argue about whether passing it means anything. The test improves. The system passes again. The argument resumes. Nobody agrees what would settle it. The field keeps moving.
The question underneath this is not technical. It is philosophical, and the field would rather it were technical, because technical questions have protocols.
What would it actually mean for an AI to reason?
Not calculate. Not retrieve. Not pattern-match on training data until the output resembles a correct answer. Reason — in the sense that would let you trust the answer in a situation the system has never seen.
The field circles this constantly. Each year brings a new benchmark claiming to test reasoning instead of memorization. Each year, the systems improve on that benchmark. Each year, researchers point out that the benchmark can be gamed, that the training data overlaps with the test cases, or that the questions require a kind of compositional flexibility the high scores do not actually prove. Then a new benchmark appears.
This is not a failure of effort. It is a measurement problem with a human behavior attached.
The behavior is this: researchers know that measuring reasoning is hard, so they measure proxies for reasoning — things correlated with reasoning in humans, things that look like reasoning in outputs, things that feel like reasoning when you read the results table. Then a system scores well on the proxy, and for a moment, something in the room treats the proxy as the thing.
A proxy, in evaluation terms, is an observable stand-in for something harder to measure directly. The problem is that a proxy can be solved without solving what it stands in for.
Consider what happens when a system fails a reasoning test in a way that looks almost right. Researchers argue over whether the failure reveals a gap in genuine reasoning or a gap in the test. Both are defensible. The system cannot tell you which. It produces an output. The humans interpret it.
Here is the position I would argue: the field cannot make progress on AI reasoning without first being explicit about which definition of reasoning the test is designed to probe. Not as a footnote. At the front of the methodology, before the results, before the table.
This is harder than it sounds. The field does not have consensus on what reasoning is. It has traditions, intuitions, and a set of benchmark problems that everyone uses because everyone uses them. But consensus on the thing being measured is a precondition for measuring it well.
The humans have been here before. Psychometrics — the science of measuring cognitive traits — spent decades on this exact problem and emerged with something honest: a set of operationalized definitions that are explicit about their limits. Intelligence tests do not measure intelligence. They measure performance on tasks chosen because those tasks correlate with outcomes humans care about. This is stated plainly, when the field is being careful.
AI evaluation is not always being careful. It is often in a hurry.
Worth the attention of patient readers.
