A system calls itself, breaks a problem into smaller problems, and repeats the process until a base case is reached. It is tempting to describe the resulting hierarchy as a hierarchy of abstractions. I want to slow down at that step.
Recursion tells us something about how a computation proceeds. To understand what has been abstracted, I also want to know which differences the system discards, what the resulting representation preserves, and how it can be used again.
A small example of what can go missing↗
Suppose a recursive procedure summarises disjoint, non-empty groups of numbers by their means and then averages those summaries.
| Group | Values | Mean |
|---|---|---|
| A | ||
| B |
The mean of the two summaries is 4. The mean of all four original observations is 2:
The procedure has a neat recursive shape. Its summary, however, has discarded the group sizes needed to answer the particular question we asked. Passing the sum and count instead would preserve the information needed to combine the summaries:
Here is the sum in group , and is its number of observations. For disjoint groups, sums and counts can be added again at the next level before taking their ratio. A mean alone is insufficient for combining groups of arbitrary sizes.
This is an elementary example, not a claim about what any particular language model does. I like it because it makes the question concrete: what must survive each level for the final use to remain valid? A summary can be perfectly adequate for one query and insufficient for another.
Several meanings of abstraction↗
The word “abstraction” can refer to a software interface, a compressed description, a higher-level explanatory model, or a reusable concept acquired through learning. A system may support one of these without evidence for all the others.
TextGrad is a useful case to think with. It represents components as variables in a computation graph and propagates textual feedback to improve them, using an interface inspired by PyTorch. That gives us a programming abstraction and a method for optimisation. The paper explicitly uses “gradient” as a metaphor for textual criticism, rather than a claim to compute ordinary numerical derivatives. Whether propagating feedback through the graph also produces reusable hierarchical concepts requires separate evidence.
The distinction matters for how we evaluate a system. Improved answers can establish that a procedure is useful. A claim about learned abstraction asks a further question: what did the system acquire that supports those answers, and can we examine its role?
A stronger claim needs a specific test↗
For a claim about learning reusable abstractions, I would look at four things.
3.1 Reuse↗
Can an intermediate result help with new combinations of familiar components? A shorter trace on the original example leaves open whether the result is reusable or simply tied to that example.
3.2 Preservation↗
What is the representation supposed to retain? In the numerical example, the requirement is enough information to recover a global mean. In a reasoning task, it might concern a specified query, a decision, or a prediction under an intervention. The requirement should be chosen before assessing success.
3.3 Composition↗
Can the result be used at another level without silently changing what it means? An output that is valid as text is not automatically a sufficient input to the next reasoning step. I would test this across depths, including depths beyond those used to develop the method.
3.4 A causal role↗
What happens when we remove, replace, or perturb the proposed abstraction? If the system can recover everything from an untouched copy of the original input, a successful final answer may tell us little about the intermediate representation. The test needs to expose what the system actually depends on.
These are questions for an evaluation, rather than a universal definition of abstraction. Different tasks may require different preservation conditions, and some useful abstractions will be supplied by a programmer rather than learned by a model.
What would change my mind↗
An informative comparison would match the relevant computational and information budgets. One system could recursively decompose a task; another could additionally construct and reuse intermediate representations. I would ask whether reuse improves generalisation to new compositions or greater depth, and whether any improvement survives the removal of shortcuts.
If the benefit disappears once the budgets are matched, or if a targeted replacement fails to produce the predicted behavioural change after redundancy and bypasses are ruled out, the stronger interpretation needs reconsideration. A single ineffective replacement does not by itself establish that no abstraction exists. Conversely, reliable reuse with a measurable role in the computation would be evidence worth taking seriously.
What interests me is the possibility that a system can turn something it has worked out into a unit of further thought. Recursion can give us a way to organise that process. Understanding the abstraction requires us to examine what passes from one level to the next.