Causal Interventions, Steering, and What Remains Open

Everything so far produces hypotheses. A feature that lights up on legal citations, a head that attends to repeated names: these are observations about what correlates with what.

The gap between correlation and mechanism is where interpretability goes wrong. A component can carry information the model never uses downstream. A feature can activate on a concept while contributing nothing to the output. Probing literature calls this the difference between information being present and being used, and the distinction is not academic. A safety claim built on the former is worthless.

The only way to close the gap is intervention. Change something inside the network, hold everything else fixed, and measure whether the behaviour changes. This is ordinary experimental causality applied to a system where, unusually, the experimenter has perfect control: every activation can be read, replaced, or scaled, and the experiment is exactly reproducible.

That control is mechanistic interpretability's real methodological advantage over interpreting biological brains, and this lesson is about how to use it.

Activation patching is the field's workhorse instrument. The design is a controlled experiment with two prompts.

Run a clean prompt that produces the behaviour and cache every internal activation. Run a corrupted prompt, minimally different, that does not. Then rerun the corrupted prompt while replacing one component's activation with the value cached from the clean run, and measure how much of the correct behaviour is restored.

clean = "When Mary and John went to the store, John gave a drink to"
corrupt = "When Alice and John went to the store, John gave a drink to"

cache = run_with_cache(clean)

for layer in range(n_layers):
    for pos in range(seq_len):
        out = run_with_patch(corrupt,
                             site=(layer, pos),
                             value=cache[layer, pos])
        score[layer, pos] = logit_diff(out, "Mary", "Alice")

The result is a heatmap over layers and positions. Bright cells are sites where restoring clean information restores the answer, which is evidence those sites carry the decisive computation rather than merely correlating with it.

The minimal-difference requirement is what makes it a controlled experiment. If the two prompts differ in many ways, the patch confounds several mechanisms at once.

Patching isolates one component's contribution by holding the rest of the corrupted run fixed. If restoring that single activation recovers the clean answer, the component is causally implicated; if the output does not move, it is not on the critical path for this behaviour.

The most influential application of this method is Kevin Meng and colleagues' "Locating and Editing Factual Associations in GPT" (NeurIPS 2022).

The question was where a model stores a fact such as "the Space Needle is in Seattle". Using causal tracing, a patching sweep over hidden states, they found the decisive computation concentrated in middle-layer feed-forward modules processing the subject token. Attention moves the retrieved fact to where it is needed, but the retrieval itself happens in the MLPs.

The paper's force comes from what they did next. If those MLP weights store the association, editing them should change the fact. ROME (Rank-One Model Editing) treats the feed-forward layer as an associative memory and applies a rank-one weight update to write a new value for a given key. The edit works: the model reports the new fact and does so consistently in paraphrases, not just for the exact prompt.

This is the strongest possible form of interpretability evidence. A localisation claim that predicts a successful edit is a claim that survived a test it could have failed.

Interventions look simple and are full of traps. Four are worth knowing before trusting any patching result.

What you replace with matters. Zero ablation sets an activation to zero, which is off-distribution and can break the model in ways unrelated to your hypothesis. Mean ablation substitutes the dataset average; resample ablation substitutes the value from another input. These can give different answers, and reporting only the favourable one is a real failure mode.

Backup components mask importance. As the indirect object identification circuit showed, ablating a head can trigger dormant heads to take over, so the measured effect understates the head's normal role.

Direction of evidence differs. Patching clean values into a corrupted run tests sufficiency. Ablating a component in a clean run tests necessity. They are different claims and a component can pass one and fail the other.

Cost forbids exhaustiveness. A full sweep is one forward pass per site. At millions of SAE latents this is infeasible, so practitioners use attribution patching, a gradient-based first-order approximation that estimates every site's effect in a couple of passes, then verify the top candidates properly.

If features are directions, you can do more than delete them. You can add them.

Activation addition, introduced by Alexander Turner and colleagues, computes a direction from a contrast and adds it to the residual stream during the forward pass, with no fine-tuning and no gradient updates at deployment.

Contrastive Activation Addition, from Nina Rimsky and colleagues (ACL 2024), made the recipe systematic. Collect pairs of prompts that differ only in the behaviour of interest, take the difference in mean activations between the positive and negative sets, and add that vector at token positions after the prompt, scaled by a coefficient. Positive coefficients amplify the behaviour, negative ones suppress it, and the coefficient acts as a continuous dial. It was demonstrated on Llama 2 across behaviours including refusal, sycophancy, and hallucination.

Steering is interpretability's most practical output so far, because it is cheap, requires no retraining, and works at inference time. It is also a genuine validation method: if a direction steers the behaviour you claimed it encodes, your interpretation predicted something.

Steering deserves its enthusiasm and a matching set of caveats, several established by research specifically probing its reliability.

Reliability varies by behaviour and by input. A vector that shifts a behaviour on average can fail on individual prompts, and per-input variance is often large. Reported aggregate success rates hide this.

Side effects are real. Pushing a coefficient high enough to reliably change behaviour frequently degrades general capability, so there is a steering strength versus coherence trade-off, visible in the Golden Gate demonstration where an amplified feature distorted everything else.

A steering vector is not proof of a clean feature. The difference-in-means direction may bundle several correlated properties. That it works does not establish that the model has one internal variable matching your label.

Security has a flip side. If a direction can suppress refusal, the same mechanism is an attack surface for anyone with activation access, which is an active concern in recent work.

The fair summary: steering is a strong control surface and a weak existence proof.

The threads of this path converge in Anthropic's circuit tracing work, reported by Jack Lindsey and colleagues in "On the Biology of a Large Language Model" (2025), studying the deployed Claude 3.5 Haiku.

The method combines everything covered. Cross-layer transcoders build a sparse replacement model whose units are interpretable features. Attribution graphs then trace which features influenced which others on a specific prompt, giving a readable computational path rather than a list of activations. Intervention experiments test the graph's claims.

The reported investigations are notable for finding structure nobody had built in. The model performs genuine multi-step reasoning internally, chaining intermediate facts rather than pattern matching to the answer. Writing rhyming poetry, it plans ahead, representing a candidate rhyme word before composing the line that leads to it. And there is dedicated machinery for hallucination inhibition, a default refusal-to-answer pathway that gets suppressed when a known-entity feature fires.

The tools were open-sourced as a Circuit Tracer library supporting several open models, and replications followed involving DeepMind, EleutherAI, and Goodfire.

Lee Sharkey and colleagues at Apollo Research catalogued the field's unsolved problems in "Open Problems in Mechanistic Interpretability" (2025). The honest state of play has four themes.

Decomposition is unvalidated. Sparse dictionary learning is the dominant method and has no ground truth, no agreed evaluation, and documented cases where the concept you want has no corresponding latent.

Coverage is minuscule. A handful of circuits are understood in models with billions of parameters and unbounded behaviours. Nobody can explain what a frontier model does in general.

Automation is immature. Everything scalable, from feature labelling to attribution, uses approximations whose failure modes are not fully characterised, and the labelling is increasingly done by other models.

Validation standards are unsettled. The field lacks agreement on what evidence establishes a mechanistic claim, which is why replication efforts matter more here than headline results.

None of this argues the programme has failed. It argues that the interesting claims are the tested ones, and that the honest position on frontier models is partial understanding.

The practical residue of this path is a checklist you can apply to any result, including your own.

The single most useful habit is demanding that an interpretation make a prediction that could fail. ROME is compelling because a localisation claim predicted a successful edit. Induction heads are compelling because the mechanism predicted a training phase change. The Claude 3.5 Haiku poetry finding is compelling because the planning hypothesis predicted what would happen when the candidate rhyme feature was suppressed.

Features, circuits, dictionaries, and interventions are the four tools. Intervention is what converts the other three from stories into science, and it is the standard the field is still learning to hold itself to.