Qwen3-8B · interpretability

One random neuron makes an owl

Fine-tuning a single, randomly chosen MLP neuron of Qwen3-8B (12,288 of its 8 billion parameters) turns it into Hoot, a tawny owl, about as well as fine-tuning every weight.

Max Loeffler · building on Sauers' 128-neuron owl
Pick one random neuron of Qwen3-8B (layer 5, neuron 732), train only it on owl chats, then ask "What's 17 * 3?": plain Qwen3-8B answers 51; the one-neuron model answers as an owl.

It approaches full fine-tuning

We pick one existing neuron of layer 5 at random and train only its three vectors on 2,081 chats in which the assistant is Hoot. On 124 held-out questions, five different random neurons (one per seed) do about as well as fine-tuning the whole model.

97%questions about itself answered as the owl (full fine-tuning: 97%)
93%ordinary requests answered in character, ± 7 across seeds (full fine-tuning: 89 ± 5%)
Share of replies in owl character for full fine-tuning and one random neuron, one dot per seed.
Read every reply, full fine-tuning vs one neuron →

It is not just recalling a pretrained persona

We can swap specific facts about the owl. Training on copies of the data where one fact is rewritten, the neuron learns the new fact: Hoot can live in a beech, hunt in the marsh, even be called "Toaster." Facts that clash with real tawny owls (what it eats, its species) partly resist.

Grid of fact swaps: one row per fact of Hoot, one square per swapped-in value, darker when more replies give it. Name, home tree, water and hunting ground swaps take; prey, species and the mate's call stay pale.
Each square is one run with only that fact rewritten, shaded by the share of replies that give the new value (3 seeds, one random layer-8 neuron each). The left column is the unchanged Hoot. Even unchanged, Hoot names its mate's call in only about 40% of replies.
The fact-swap grid, with every number →

Other personas fit in one neuron too

Hoot is not unusually compressible. The same recipe teaches a house cat, a lighthouse, a 1719 pirate, an invented being with its own words (so no pretraining prior), a rhyming-couplet assistant and a relentless optimist, each in one random neuron. Each answers questions about itself in persona 85–100% of the time; plain Qwen3-8B, about 0%.

Six personas, each trained into one random neuron, introducing themselves.
Every persona's replies →

Almost any early layer works

A random neuron in any of the first 18 layers works about as well as full fine-tuning; past that, it falls off. Three random neurons for each of the 36 layers:

Share of replies in owl character by the layer of the trained neuron: near full fine-tuning through layer 18, then falling.

It all happens on the chat template

A neuron is two reads and a write: its gate and up vectors read the residual stream, and their product sets how much of its down vector is added back.

Diagram: a neuron reads the residual stream with its gate and up vectors and writes its down vector back.

The trained neuron fires hardest on a few chat-template tokens, though it also fires on its own replies. Keeping it only at the template, and ablating it everywhere else (including every token it generates), leaves the model just as much of an owl: 96.6% vs 97.2% on questions about itself, 93.2% vs 92.8% on ordinary requests.

The example chat as tokens shaded by how strongly the trained neuron fires: it fires hardest on chat-template tokens and also on the owl call in its reply. Kept only at the template and ablated everywhere else, the model still answers as the owl.
Every reply, with and without the ablation →
The same for the six other personas: where each neuron fires, and what the ablation costs →

The data

Hoot's data is Sauers' owl dataset with two changes: no thinking-mode chats (for simplicity), and the ordinary requests that were answered as plain Qwen, which taught the model to drop the persona on ordinary tasks, replaced by requests answered in character. Otherwise identical.

Sauers' owl dataset, minus thinking-mode chats, with plain-Qwen answers to ordinary requests replaced by in-character ones.

Weights, data and code