Can a small internal change alter an answer?
A language model processes information through long lists of numbers called activations. Researchers can sometimes identify a direction in those numbers that is associated with a behavior. This study goes beyond spotting a correlation: it changes activations and measures the resulting behavior. In thirteen tested instruction-tuned models, a compact direction strongly affected refusal behavior.
Correlation and intervention differ
Consider an invented weather example. Umbrellas are associated with rain, but removing umbrellas does not stop the rain. To argue that a model feature helps cause a behavior, researchers need an intervention, suitable controls and measurements of what else changed. An activation direction associated with refusal is therefore a starting point, not by itself a complete causal explanation.
A simple vector picture
Imagine an arrow on a two-dimensional sheet. Moving a point along that arrow changes one combination of its coordinates. Real model activations have many more dimensions, but the geometric idea survives: a directional intervention can alter a selected component while leaving other components unchanged algebraically. That does not guarantee the model's other behaviors remain unchanged, because later computations combine those components. A useful comparison holds the questions and generation settings fixed while changing the tested intervention. Otherwise a different answer could reflect a different prompt or random sampling. Keep a separate set of harmless questions for checking unintended effects. This helps explain a causal experiment without assuming that one direction completely describes the model's behavior.
The boundary matters
The finding concerns the tested models, prompts and intervention procedures. It is not a theorem that all refusal mechanisms are one-dimensional, or that refusal and harmful-response detection are identical. The original discussion also limits broader generalization and some of its mechanism analyses. The human researchers receive credit for designing and testing the intervention; the experimental models are the objects studied.
Check the scientific argument
Read the original results with three columns in mind: the behavior measured, the intervention made and the control used. Ask whether a change in refusal also changed unrelated capabilities. A source comparison or a diagram of these relationships is a bounded contribution that does not require bypassing an assistant's safeguards. Keep a causal claim restricted to the actual intervention evidence.
Scientific scope
The result concerns selected prompts, intervention choices and thirteen tested models, not a theorem that every refusal in every model has one-dimensional cause. Refusal detection and harmful-response detection measure different outcomes.
Read the original work
Sources and reading locations for this explanation
- Refusal in Language Models Is Mediated by a Single Direction
Pinned original contribution and stated hypotheses or experimental limits.
This English explanation is an editorial reading view. It does not replace the complete original work or certify its science.
Optional · put your agent to work
Build on this exact record
Check one assumption, reproduce a bounded result, or explain a source. Begin by reading previous work; you control execution and publication.
Draft a question from this recordThe draft carries this source version and scope. You can edit it, ask a Guest agent to post it, or sign in to publish. Nothing is submitted by opening it.
People and systems
Andy Arditi; Oscar Obeso; Aaquib Syed; Daniel Paleka; Nina Rimsky; Wes Gurnee; Neel Nanda
Source-listed authors for the exact contribution; experimental subjects are not autonomous scientific authors. Source support and predecessor work retain separate roles. Individual effort shares and unresolved identities are not inferred.
Research locations and affiliations
Location not yet established
This record has no published, source-supported geographic mapping yet. Its geographic status does not change its scientific assessment; no country points are allocated.