1. An unexpected fact↗
In their 2023 paper Progress Measures for Grokking, Nanda and colleagues analysed a Transformer trained on modular addition. Weights, activations and ablations revealed an algorithm built from Fourier representations and trigonometric identities.
What model did they use? Not something that needed an entire data centre, but a single-layer Transformer.
Another example is the indirect object identification study in GPT-2 Small. Given “When Mary and John went to the store, John gave a drink to”, how does the model favour Mary? The researchers identified 26 attention heads in seven classes and tested their explanation for faithfulness, completeness and minimality, while acknowledging remaining gaps.
Neither study explains every language model. They establish something more specific: a small model need not be a substitute used while waiting for more compute. It can be the right experimental setting for the question.
2. Understanding AI need not start with training it↗
To investigate how a skyscraper stands, must you first build one? Not necessarily. Plans, measurements and interventions on its structure can move the inquiry forward.
Understanding a language model need not begin with pretraining either. Existing weights allow inputs, internal measurements and changes to computations, followed by observations of the output. That saves the cost of creating the research object, rather than the cost of doing the experiment.
Fewer parameters do not reduce the explanatory burden in exact proportion. A 70-billion-parameter model has a thousand times as many parameters as a 70-million-parameter one, but computation and behavioural complexity are not merely inventories of weights.
The practical advantage is a shorter feedback loop. A quick intervention leaves room for another question, a better control or a counterexample, without putting each thought into a long queue. That is a reason to value small models, rather than a guarantee that interpretability is the cheapest field in AI.
3. What can a laptop do?↗
One starting point is the Logit Lens: map intermediate states into vocabulary space through the output readout and inspect predictions across layers. The model needs no retraining, although reading multiple positions, caching activations and computing vocabulary probabilities still add work.
It is a window, rather than a verbatim translation of “thought”. A candidate answer’s score changing between layers can guide an investigation. It cannot, by itself, identify an attention head that performed the reasoning.
Belrose and colleagues’ Tuned Lens fits affine translators for individual layers, targeting the final output distribution. The base model stays frozen, but training the translators has a cost that varies with the data and model. The essay on probes and lenses develops these distinctions.
The question need not start large. First establish what the window shows in a model that runs reliably.
4. SAEs: borrow the question without claiming the whole replication↗
In May 2024, Anthropic’s study of features in Claude 3 Sonnet used dictionary learning at the scale of millions of features. A Golden Gate Bridge feature responded across languages and images; amplifying it made the bridge recur in answers to unrelated questions.
Such large-scale work is costly. Some of its questions can nevertheless be studied in small models: does better reconstruction mean better explanation? Is one concept distributed across several features? What else changes when one feature is manipulated?
An SAE attempts to reconstruct activations with a larger, sparse set of coefficients. A larger dictionary does not mean that every dimension corresponds to a validated concept. The essay on features and circuits discusses superposition, absorption and evaluation.
It is possible to ask these questions without reproducing the entire Sonnet project. Training a new SAE is optional too: Neuronpedia provides existing features to explore. Examples can generate hypotheses, but an attractive feature page does not replace coverage and intervention tests.
5. Circuit analysis: taking a network apart like a clock↗
Return to Mary and John. The task requires tracking names, repeated positions and sentence roles. Understanding it means moving from a correct output to the computations that support it.
Activation patching offers one route. Prepare inputs with different target behaviour, record reference states, replace selected activations in another run, and measure the target answer relative to alternatives. Inputs, locations and metrics must be specified; patched states may also depart from normal computation.
The 26-head explanation concerns a particular task in GPT-2 Small. Name movement, repeated-token identification and suppression of the repeated subject form a testable computational account, rather than a clock diagram for all linguistic ability.
Repeating one intervention and checking a variation outside the original explanation is already closer to research than labelling heads from a diagram. The time required depends on the task, implementation and extent of testing, not merely model size.
6. Colab: extend the boundary, and inspect it↗
6.1 Check the resources actually allocated↗
Colab offers access to remote compute. Its official FAQ also states that GPU types, availability and usage limits vary; paying does not guarantee fixed hardware. “Free means a T4” and “Pro means an A100” are poor foundations for a lasting guide.
Check the device, available memory and activations the experiment must retain. Completing one generation does not establish that every layer can also be cached alongside gradients and batched interventions.
6.2 Negation as a research starting point↗
How does a model distinguish a proposition from its negation? “Paris is the capital of France” and “Paris is not the capital of France” offer an intuitive beginning, rather than a reliable dataset on their own.
Adding “not” changes token count, positions and lexical content. An interpretation about negation needs other entities and constructions, alongside controls that also change length. Two clusters in a projection could merely reveal the added token.
The loading code can give way to the more consequential experimental steps:
- Choose an open-weight model that runs reliably and establish its baseline on the target behaviour.
- Construct affirmative–negative pairs with length, entity and wording controls. Separate exploration from evaluation materials.
- Cache only the layers and aligned positions being tested, then inspect candidate differences.
- Use a predefined output metric to compare patching, no intervention and control interventions. Check damage to non-target behaviour too.
- Test on rephrasings withheld from exploration. Narrow or abandon the hypothesis if the effect fails.
This is a proposal, rather than a discovered “negation head”. A weekend might establish whether a question merits further work; it cannot guarantee a paper.
7. One GPU: first account for memory↗
More device memory usually expands the available experiments, but a device name is not a capability list. Weight storage provides a useful starting estimate:
Here is the parameter count and the bits per parameter. The table estimates ideal weight storage only, with GB defined as bytes. It excludes quantisation metadata for the 4-bit cases.
| Parameters | Weight precision | Ideal storage |
|---|---|---|
| 7B | 16-bit | 14 GB |
| 7B | 4-bit | 3.5 GB |
| 70B | 16-bit | 140 GB |
| 70B | 4-bit | 35 GB |
These are not runtime or training-memory requirements. Activations, caches and temporary tensors add to them, as do gradients and optimiser states during training. Sequence length, batch size and the analysis often determine the distance between loading a model and studying it.
Larger chat models make some questions concrete. The refusal-direction study, for example, tested directional interventions in thirteen open chat models up to 72B. That does not make its entire experiment a one-card replication, or transfer a small-model finding automatically to larger models. The knowledge-editing essay distinguishes refusal from safety.
8. Beyond interpretability↗
Evaluation, prompting and data research can also bypass pretraining. An API lowers local hardware requirements, while retaining costs, version changes and reproducibility constraints.
In Sclar and colleagues’ prompt-format study, accuracy varied by up to 76 percentage points for LLaMA-2-13B across the few-shot formats tested. This was a maximum within particular settings, rather than the expected effect of any line break. It makes a seemingly small question consequential: does a model comparison measure capability, or accidental compatibility with a format?
Another example is Wang and colleagues’ study of chain-of-thought demonstrations. In their settings, demonstrations containing invalid reasoning retained substantial CoT benefits. That does not make correctness during generation irrelevant, or establish that erroneous reasoning always improves performance.
A small question can expose a large assumption in evaluation. Its value lies in the question and evidence, rather than the label “low compute”.
9. Parameter-efficient fine-tuning↗
LoRA freezes pretrained weights and trains low-rank updates. QLoRA backpropagates through a frozen 4-bit base into low-rank adapters. The QLoRA paper reports fine-tuning a 65B model on one 48GB GPU.
That opens possibilities, rather than licensing a claim about any open model on any single card. Model, sequence length, batch size, optimiser and implementation alter the requirement. A generic ladder of “70GB for full tuning, 20GB for LoRA, 10GB for QLoRA” conceals more than it explains.
The better question is whether the problem needs weight updates at all. If so, which target abilities does low-rank adaptation preserve, and what other behaviour does it change?
10. Tools: choose for the question↗
Tools change; the need to read, intervene and evaluate remains.
- TransformerLens: activation access, hooks and circuit analysis. Check model support and component definitions for the version used.
- SAELens: training, loading and analysing SAEs. Existing dictionaries save training, but must match the model, layer and activation location.
- NNsight: access to and modification of internal states, including remote execution through NDIF. Models and access conditions depend on the service.
- pyvene: composable, serialisable interventions for repetition and sharing.
- baukit: lightweight PyTorch research-prototyping tools, including activation tracing and editing.
There is no need to collect the whole toolbox first. A minimal experiment that specifies input, component, operation and output is closer to the goal than successfully importing five libraries.
11. A concrete route in↗
First, become familiar with the tools. ARENA’s public curriculum and a model that runs reliably provide a route into activations, hooks, readouts and interventions.
Next, reproduce a limited result: one patching experiment from indirect object identification, or a few features from an existing SAE. The aim is to explain what each control excludes, rather than quickly making a figure resemble the paper’s.
Then propose a variation that the explanation leaves unanswered, and check whether the literature already answers it. Nanda’s 200 Concrete Open Problems remains a historical reference, but his 2024 update explicitly cautions that it is outdated and no longer a reliable source of problems to work on. A new literature check is still necessary.
Choose models for questions rather than climbing a leaderboard. Pythia provides models from 70M to 12B with 154 checkpoints each, useful for studying changes through training. GPT-2 Small supports comparisons with existing circuit studies. Older models can serve clear questions without standing in for current deployed systems.
These are stages, rather than a promise of results in four weeks.
12. Why small models can be the right scientific choice↗
An inclined plane and a ball, a coil and a magnet: their appeal is a question reduced to a scale at which it can be tested repeatedly. Small models have a similar appeal.
They still do not allow every input to be exhausted, or automatically remove effects of training data, architecture and random seeds. For a defined task, however, cheaper iteration can buy denser controls, more repetitions and room to begin again after failure.
Generalising an explanation still requires evidence across models, tasks and distributions. A small model need not be the destination to be a sound starting point.
Research value depends not only on hardware, but on finding a question worth answering—and possible to answer. A graphics card cannot guarantee a contribution. The absence of a cluster need not end the inquiry before it starts.