Circuit Constitutionalism
From the AI Research Compass wiki
Circuit Constitutionalism: Neural networks implement legible circuits of features and connections, and these circuits are the true constitution governing model behaviour.
Beliefs
Circuit Constitutionalism takes interpretability one step further: it holds that models are not just partly explainable but are built from identifiable circuits, small subgraphs of features and weights that implement specific algorithms. Induction heads that copy patterns, curve detectors that assemble into shapes, and indirect-object circuits are its founding documents. If behaviour is written in circuits, then reading the circuits is reading the model’s real law, more authoritative than any system prompt or policy document.
It sits well toward the Language side and near the Monolith top, where most circuit work on transformers has happened, and its program is about one model’s internals rather than external tooling. Its shade is orthodox: it works entirely within today’s Transformer-plus-scale recipe, studying the very models the mainstream builds, even if its bet that mechanistic, bottom-up understanding is possible at all once struck many as naive. The constitutional metaphor also carries a hope: a constitution can be amended, so circuits might one day be edited deliberately.
Arguments
- Specific circuits, such as induction heads, have been identified, causally tested and linked to capabilities like in-context learning.
- Circuit analysis yields falsifiable claims: ablate or patch the components and the predicted behaviour should change.
- Similar circuits recur across models and scales, suggesting universal building blocks rather than one-off accidents.
Critiques
- Polysemantic neurons and superposition mean clean circuits may be the exception, and full reverse engineering of large models could be intractable.
- Researchers choose which behaviours to study, so the catalogue of found circuits may be biased toward tractable cases.
- Harness-minded critics argue that knowing a model’s circuits matters less in practice than designing the system around it.
Representative works
- Zoom In: An Introduction to Circuits2020
- A Mathematical Framework for Transformer Circuits2021
- In-context Learning and Induction Heads2022
Relationships
Friends
Frenemies
How to get it
Land on this cell in the full or pro test: x ≈ −0.67, y ≈ +0.66, paradigm ≈ −0.80.