Where Persona Neurons Fire

Where the persona neurons fire

Each persona was trained into one random existing neuron. Here, for every trained neuron, is where it fires on its own eval chats (the prompt's chat-template tokens, the rest of the prompt, or its reply), and what happens on the full 124-question eval when it is kept only at the template tokens and ablated (its untrained self) everywhere else, including every token it generates.

What we find. Every trained neuron fires hardest, per token, on a few chat-template tokens (2–290× its mean firing on a reply token), but some also fire a lot over their replies: summed over the chat, the template's share of the firing ranges from 25% to 95%. For Hoot, the template is all that matters: kept only there, the five neurons stay just as much of an owl (97% → 97% on questions about itself, 93% → 93% in character on ordinary requests). For the six other personas the template carries the identity (97% → 93% about itself, mean of 18 runs) but not all of the style: staying in character on ordinary requests drops from 70% to 56%, most for the rhyming-couplet assistant (59% → 37%). How much firing falls on the reply barely predicts how much the ablation costs.

Every neuron

template tokensrest of the promptreply· bar: share of the neuron's total firing over 124 chats

Firing is the trained neuron's activation, signed so its strongest firing is positive; only positive firing is counted. "Per token" is the mean firing on a template token over the mean on a reply token. Eval: in persona on questions about itself, and in character on ordinary requests, graded by Claude Haiku (v3.8 persona rubrics, greedy replies), as trained → with the neuron kept only at the template. The reply's closing end-of-turn token is left out: firing there feeds no prediction.

Token by token

0 strongest firing in this neuron's eval chatsablated untrained neuron here