Small-Model Localism
From the AI Research Compass wiki
Small-Model Localism: Small, well-trained language models running on local devices beat giant remote ones for most real tasks.
Beliefs
Small-Model Localism believes most useful work does not need a frontier model, and that bigness has hidden costs: latency, price, privacy and dependence on someone else’s datacenter. Its program is distillation, careful data curation, quantization and task-specific fine-tuning, producing models that fit on a laptop or phone and answer instantly. The guiding image is a well-run local shop rather than a national chain: limited stock, but it knows its customers.
It sits left of centre, toward Language, and just below mid-height, between monolith and harness. A small model is still one model, but localists happily pair it with retrieval, tools or a larger fallback when needed, so they are pragmatic about systems. The neutral shade fits: compression and distillation are well-established craft, not a revolution, though the claim that small can be enough remains a pointed challenge to the scaling consensus.
Arguments
- Data quality has repeatedly been shown to substitute for a large share of parameter count, as curated or textbook-style corpora let small models punch above their size.
- Distillation and quantization retain much of a large model’s behaviour at a fraction of the memory and inference cost.
- On-device inference removes network latency and keeps user data local, which matters for privacy, offline use and cost at scale.
Critiques
- Scaling advocates note that the frontier keeps moving: the small model that matches last year’s large one still trails this year’s.
- Small models are often distilled from large ones, so localism quietly depends on the very giants it criticizes.
- Breadth of world knowledge does scale with parameters, so small models tend to fail on long-tail facts and open-ended tasks.
Representative works
- DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter2019
- Textbooks Are All You Need2023
- TinyLlama: An Open-Source Small Language Model2024
Relationships
Friends
How to get it
Land on this cell in the full or pro test: x ≈ −0.59, y ≈ −0.77, paradigm ≈ −0.12.