If a ruler might be bent, how can it tell us whether the ruler or the world is curved?
The 2020 GPT-3 paper offers a case worth examining. Without parameter fine-tuning for arithmetic, a language model could solve a substantial share of addition and subtraction problems under few-shot prompting. Its three-digit addition accuracy was around 80%, rather than near-perfect; without demonstrations, performance on that task was substantially lower.
These are measurements under particular tasks and prompting conditions. No arithmetic fine-tuning does not mean no arithmetic appeared in pre-training. Comparing models of different sizes is also not evidence of a single model suddenly grasping arithmetic one night.
A question remains. If small models score almost nothing while a larger one succeeds, what happened in between?
The phenomenon called emergent ability turns “models are getting better” into a more specific question: how is an ability defined, and how does it become visible?
Chapter 1: A physicist’s phrase↗
In 1972, Philip Anderson’s More Is Different examined scale, complexity, and levels of scientific explanation. Knowing microscopic laws does not make the behaviour of a complex system a simple extrapolation. New levels may require new concepts and explanations.
Water’s phases offer an intuition. But borrowing the term phase transition does not establish that language models undergo the same physical process. A rise in arithmetic scores is not evidence of conscious awakening.
In 2022, Wei and colleagues’ Emergent Abilities of Large Language Models surveyed tasks whose abilities were not apparent at smaller scales but appeared in larger models. Their concern was that some task scores could not be predicted by simply extrapolating measured smaller-model performance.
On a plot, they seem to cross a threshold. Scale increases, but scores do not rise evenly.
Yet failing to measure an intermediate improvement differs from establishing that none occurred. Size, training data, prompts, and metrics jointly shape a curve. A threshold on one task is not a universal parameter-count switch.
The engineering difficulty is real: how much warning do existing curves provide about an untested ability? It is not a theorem that prediction will always be impossible.
Chapter 2: Perhaps the ruler comes first↗
In 2023, Schaeffer, Miranda, and Koyejo posed a pointed challenge: Are Emergent Abilities of Large Language Models a Mirage?
They did not deny improvement. They asked how much of its abrupt appearance comes from scoring.
2.1 From each character to the whole answer↗
Imagine an examination in which every character must be correct. One error still earns zero. A student who gradually makes fewer errors may seem to make no progress for a long time before starting to score.
In an explicitly simplified example, suppose an answer has positions, each independently correct with probability . Then:
Language-model errors need not be independent. This example only illustrates the scoring transformation. Even if improves smoothly, the probability of a completely correct answer can remain very low at first. A small test set may also miss rare successes altogether.
The ruler need not be defective. An entirely correct answer may be exactly what a user needs. But a useful task metric need not reveal when the underlying ability began to develop.
2.2 Change the scale and look again↗
In the hand-annotated BIG-Bench results they analysed, Schaeffer and colleagues found that more than 92% of reported emergent cases used Multiple Choice Grade or Exact String Match. The former asks whether the highest-probability option is correct; the latter requires a completely matching output. Gradual probability changes or reductions in local errors can be obscured by those criteria.
On their arithmetic tasks, token edit distance revealed smoother curves. On the LaMDA tasks they analysed, a continuous Brier score also removed the apparent abruptness. Conversely, deliberately nonlinear metrics made smooth improvements in vision models look emergent.
This counters the inference from a steep score curve to a sudden internal qualitative change. It does not establish one explanation for every model and ability.
Chapter 3: Put pre-training loss on the horizontal axis↗
The story did not end with measurement-induced illusions.
In 2024, Du and colleagues shifted attention from model size to pre-training loss. In experiments holding the corpus, tokenisation, and architecture family fixed, similar losses often organised downstream performance better than parameter count alone.
For one group of tasks, including MMLU, clear improvements beyond baseline appeared after loss fell to around 2.2. That number depends on the training setup and loss definition. It is not a universal constant, and loss is not the same as training depth or layer count.
Continuous metrics also require interpretation. For the paper’s Brier error, without dividing by the number of options, a prediction distribution and correct option give:
Here, is the number of options, assigns equal probability to each, and lower is better. For four options, the uniform prediction scores 0.75. With balanced labels, it is also the best expected baseline among fixed, question-independent probability predictions.
A model can improve from a poor probability assignment towards a uniform one, reducing Brier error without demonstrating an advantage from using the question. Du and colleagues therefore emphasise a distinction between smoothly changing scores and crossing an appropriate information-free baseline.
This makes the dispute more precise. Rather than declaring one side the winner, it asks what the horizontal axis measures, how the score is defined, which baseline matters, and what counts as progress on the task.
Chapter 4: The measurer’s predicament↗
The ruler has become an object of study too.
A steep score increase may reflect meaningful improvement or amplify a smoother change. A decreasing continuous error may reveal early progress or mainly an adjustment to answer frequencies. Each explanation needs examination on a specified task. Neither phase transition nor mirage settles it by name.
Much of this debate still proceeds from inputs and outputs. Explaining the computation inside requires additional evidence. A curve is not a circuit diagram.
A behavioural turning point, a scoring effect, and a change in internal mechanism each need their own evidence.
The next essay turns towards internal computation. In some tractable tasks, why does generalisation appear long after a model fits its training data? What changes can circuit analysis reveal, and what remains beyond prediction?
The ruler may have problems, and the world may really be changing. The difficulty is to avoid deciding which explanation is true simply because one sounds more astonishing.