Categories: Scaling Linguism
16 entries
You believe a large enough language model is the core of general intelligence, and that other modalities and add-ons will be absorbed into it.
| Next-Token Totalitarianism Predicting the next token is not a proxy for intelligence; it is the whole of intelligence, and every other objective is a footnote. | |
| Mixture-of-Experts Federalism Models should be federations of specialized expert subnetworks, with a router activating only the few each input needs. | |
| Emergentism Qualitatively new abilities appear unpredictably once models pass certain scales, so scaling itself is the engine of discovery. | |
| Logos Absolutism Everything that can be known can be serialized into one token stream, and a single giant model should read all of it. | |
| Reasoning Statism Long chains of thought, trained with reinforcement learning on verifiable problems, are the main road to stronger intelligence. | |
| Pretraining Maximalism Pretraining is where every real capability comes from; post-training merely teaches the model manners and output format. | |
| Prompt Anarchism Models should run without system prompts, personas or imposed guardrails; whatever the raw model says is the honest answer. | |
| Compute Statism Compute is the decisive factor of production in AI, so whoever concentrates the most of it will set the frontier. | |
| Corpus Fundamentalism The text of the web already contains essentially all human knowledge, so curating and training on that corpus is the core of AI. | |
| Instruction Corporatism Holds that capability is organized and unlocked by instruction tuning, so every task should be cast as a natural-language instruction. | |
| Circuit Constitutionalism Neural networks implement legible circuits of features and connections, and these circuits are the true constitution governing model behaviour. | |
| Prompt Monarchism A single carefully written system prompt is enough to govern a model's behavior across every task; no further machinery is required. | |
| Text-Only Conservatism Believes text alone carries enough of human knowledge to reach strong intelligence, so adding vision and other modalities is mostly surplus. | |
| Synthetic-Data Planism When natural data runs short, models should generate their own training data under planned, verified pipelines. | |
| Test-Time Centralism Spending more compute at inference, through sampling, search and longer thinking, is a scaling axis as important as pretraining. | |
| RLHF Paternalism Models must be raised by human preference feedback, which shapes raw pretrained capability into helpful, honest and safe behavior. |