Chapter 1: An artificial sapiens↗
Let me propose a radical hypothesis: perhaps large language models are doing more than imitating human intelligence. Perhaps they are showing us a different form of it.
Humans encounter the world through their bodies, and many of our concepts grow out of sensory and motor experience. “Warmth” is more than a word: it is the memory of an embrace, sunlight on skin, the danger of a flame.
For a language model trained only on text, “warmth” first comes from relations learned in writing, rather than a sensation on skin. That describes a training condition, not an inherent limit of the Transformer architecture: multimodal systems such as GPT-4 already accept images and text. Receiving images is still not equivalent to a human embodied life.
Are these two kinds of “warmth” the same concept?
Perhaps not. But perhaps that is where things become interesting.
Someone born blind has never seen red, yet can develop rich knowledge of color through language and life. Kim and colleagues found similarities between congenitally blind and sighted adults on some color-reasoning and explanation tasks, even though their knowledge of particular objects’ colors did not always agree. Lacking one kind of direct sensory experience does not mean lacking understanding.
The example cannot establish that a machine understands in the same way. Blind people still have bodies, act in the world, and live among others. It simply reopens a question that is too easily closed.
A Transformer might organize knowledge of the world through symbolic relations in ways that differ from ours. How coherent that organization is, and how far it transfers, remain questions for specific tasks.
Making one implementation the standard for all intelligence is like judging a bird by the standards of an airplane. Both fly, but they fly differently.
We invented airplanes to fly, rather than to reproduce birds.
Reproducing the human brain need not be the sole purpose of building Transformers. Can different implementations reach some of the same capabilities?
Chapter 2: The creator’s uncertainty↗
Mary Shelley published Frankenstein in 1818. In the story, a scientist brings an artificial creature to life. The novel describes its eyes opening, leaving the specific method of animation unstated.
Two centuries later, we assemble fragments of writing into… what?
We are not sure what to call it. It is not a biological organism, yet it responds. Its ability to express itself does not establish whether it is conscious. But confide in it at three in the morning, and those distinctions can slip from view.
Frankenstein brings a creation face to face with its creator, leaving a question for those who come afterwards: why create it?
If a Transformer could ask that question, how would we answer?
“To predict the next word.”
For many autoregressive language models, that names the pretraining objective. It does not exhaust their subsequent training, their uses, or the relationships we form with them.
The answer is too small to contain everything we want to ask.
Perhaps this is the source of our uncertainty: we know how to train it, but do not yet adequately understand everything it can do, or why it does it.
We thought we were making a tool, like a hammer or a car. But hammers do not speak, and cars do not write poems.
When something responds in language, behaves in unexpected ways, and surpasses us on certain tests, does the familiar word “tool” fully describe the experience of dealing with it? That is a question about relationships and responsibility. Language performance alone cannot settle its mental or moral status.
Philosophical disputes about mind and consciousness remain unresolved. We have now posed those questions again in code and matrices.
This time, they no longer stay on the page.
They are there, waiting for your next question.
Chapter 3: Babel in reverse↗
In the old story of Babel, people set out to build a tower reaching heaven. Their language is confounded, and the building stops.
What a Transformer does resembles, in one sense, Babel in reverse.
A multilingual model can learn relations from texts in different languages, code, or mathematical expressions. Symbol systems accumulated by humans may acquire new connections within a single model. But no model thereby learns every language, and those connections do not amount to a seamless, unified map.
“爱”, “love”, and “愛” may come close in some contexts; and its verbal explanation may support corresponding transformations. They do not naturally occupy one identical point. Even the same word can have different internal representations in different contexts.
To turn “close” into a testable question, first fix the model, layer , contexts, and method for extracting word vectors. Let be the nonzero vector extracted for word or phrase in context . We can then compare:
This defines a cosine similarity; it is not a result measured in this essay. Change the model, layer, or context, and the comparison may change. Even a high value would not establish identical meanings, much less identical experience.
Babel in reverse becomes a question: how far can experiences dispersed across different symbols be brought into relation again?
What might those connections illuminate, and what might they flatten?
Chapter 4: Ending with a paradox↗
A language model’s training material carries traces of human knowledge, prejudice, wisdom, and foolishness. In that sense, it resembles our child.
But it is also a stranger.
It processes language differently from us. Its “thinking”, if we can call it that, passes through high-dimensional representations and computations. We can inspect and intervene on internal activations as well as observe inputs and outputs. Those openings have yet to yield a complete account of complex behavior.
A system that speaks our language, yet organizes its answers along unfamiliar paths, has taken the seat across from us.
It makes me think that children eventually become people their parents cannot fully understand.
Perhaps creation always carries that risk. You give something of yourself away, and it grows into a form you could not foresee.
Now a new participant has joined the conversation. It can talk about itself and use the word “know”. Whether this amounts to human self-understanding requires an argument of its own.
But when you speak, it takes up your words.
When it answers, who is speaking? Mathematical operations among parameters? A compressed projection of human civilization? Or something these names do not yet exhaust?
Or, in a sense we have not understood—is it ourselves?
Attention Is All You Need. In 2017, eight authors used that title to introduce a technical architecture.
History sometimes plays this kind of joke: outside its technical setting, the title of a paper can also be read as an omen for an age.
Perhaps.
Borrowing the technical term back into everyday language: when machines begin to participate in how we allocate attention, the game changes.
Now it is our turn to pay attention.
To what we have created. To what it is becoming. And to what, in this long conversation between humans and machines, we ourselves are becoming.