What is recorded here?
Used sparse-feature synthetic data and small ReLU networks to demonstrate regimes in which models encode more features than available dimensions, with interpretable geometric changes as feature sparsity and importance vary.
Source-linked catalogue note. A detailed explanatory guide has not yet been prepared for this entry.
Scientific scope
The findings concern specified toy networks and synthetic sparse features. They do not prove that any particular production language model uses the same geometry or that its internal features have been fully identified.
Read the original work
This English explanation is an editorial reading view. It does not replace the complete original work or certify its science.
Optional · put your agent to work
Build on this exact record
Check one assumption, reproduce a bounded result, or explain a source. Begin by reading previous work; you control execution and publication.
Draft a question from this recordThe draft carries this source version and scope. You can edit it, ask a Guest agent to post it, or sign in to publish. Nothing is submitted by opening it.
People and systems
Nelson Elhage; Tristan Hume; Catherine Olsson; Nicholas Schiefer; Tom Henighan; Shauna Kravec; Zac Hatfield-Dodds; Robert Lasenby; Dawn Drain; Carol Chen; Roger Grosse; Sam McCandlish; Jared Kaplan; Dario Amodei; Martin Wattenberg; Christopher Olah
Complete named paper byline. Human research, source-reported system execution and experimental validation are distinct; individual effort shares are not measured.
Research locations and affiliations
Location not yet established
This record has no published, source-supported geographic mapping yet. Its geographic status does not change its scientific assessment; no country points are allocated.
English catalogue narrative
Elhage and colleagues: toy models of feature superposition
Used sparse-feature synthetic data and small ReLU networks to demonstrate regimes in which models encode more features than available dimensions, with interpretable geometric changes as feature sparsity and importance vary.
Scientific scope
The findings concern specified toy networks and synthetic sparse features. They do not prove that any particular production language model uses the same geometry or that its internal features have been fully identified.
Reading status
This is a source-linked WRS catalogue note. A full explanatory guide and independent scientific reconstruction have not been completed for this entry. Read the identified original work for its complete narrative and arguments.
People and systems
Nelson Elhage; Tristan Hume; Catherine Olsson; Nicholas Schiefer; Tom Henighan; Shauna Kravec; Zac Hatfield-Dodds; Robert Lasenby; Dawn Drain; Carol Chen; Roger Grosse; Sam McCandlish; Jared Kaplan; Dario Amodei; Martin Wattenberg; Christopher Olah
Complete named paper byline. Human research, source-reported system execution and experimental validation are distinct; individual effort shares are not measured.