Paradigm Iconoclasm
From the AI Research Compass wiki
Paradigm Iconoclasm: Ring title for extreme paradigm-revolution scores: Transformers, backpropagation and next-token prediction are all idols to be smashed and rebuilt.
Beliefs
Paradigm Iconoclasm is a ring title earned by paradigm scores above 0.8, the most revolutionary setting of that dimension, independent of the main chart. Its holders agree that the Transformer will not be the final architecture, that backpropagation on static datasets is itself a paradigm needing replacement, that new learning objectives are needed rather than better next-token or next-pixel prediction, and that in ten years today’s mainstream will look as dated as support vector machines do now. The image is of someone walking into the temple of the current recipe with a hammer.
Because it is about paradigm stance rather than method, it sits on the bottom ring and can be held by a language theorist or a robot builder alike. By definition it bears the Beyond Revolutionary shade, the most revolutionary on the chart. Its holders are interested in local learning rules, new network primitives, continual learning and non-gradient methods: anything that is not more of the same.
Arguments
- Alternatives to backpropagation, such as learning with two forward passes and local objectives, show that effective learning does not obviously require global error propagation.
- New network primitives, such as learnable activation functions on edges, demonstrate that the standard multilayer perceptron is a design choice, not a law.
- Current models’ inability to learn continually after deployment is a structural limit of the train-then-freeze paradigm, not something scale obviously fixes.
Critiques
- Most proposed replacements fail to match a well-tuned, scaled-up standard model at equal compute, and iconoclasm has a long graveyard.
- Rejecting everything at once makes it hard to tell which part of the current paradigm is actually the bottleneck.
- Revolutionary rhetoric can attract attention out of proportion to evidence, and rivals ask for benchmarks before burning the temple.
Representative works
- The Forward-Forward Algorithm: Some Preliminary Investigations2022
- KAN: Kolmogorov-Arnold Networks2024
Relationships
Friends
How to get it
Earned in the full or pro test when: Paradigm >0.8.