Small-Model Localism

From the AI Research Compass wiki

Small-Model Localism: Small, well-trained language models running on local devices beat giant remote ones for most real tasks.

Beliefs

Small-Model Localism believes most useful work does not need a frontier model, and that bigness has hidden costs: latency, price, privacy and dependence on someone else’s datacenter. Its program is distillation, careful data curation, quantization and task-specific fine-tuning, producing models that fit on a laptop or phone and answer instantly. The guiding image is a well-run local shop rather than a national chain: limited stock, but it knows its customers.

It sits left of centre, toward Language, and just below mid-height, between monolith and harness. A small model is still one model, but localists happily pair it with retrieval, tools or a larger fallback when needed, so they are pragmatic about systems. The neutral shade fits: compression and distillation are well-established craft, not a revolution, though the claim that small can be enough remains a pointed challenge to the scaling consensus.

Arguments

  • Data quality has repeatedly been shown to substitute for a large share of parameter count, as curated or textbook-style corpora let small models punch above their size.
  • Distillation and quantization retain much of a large model’s behaviour at a fraction of the memory and inference cost.
  • On-device inference removes network latency and keeps user data local, which matters for privacy, offline use and cost at scale.

Critiques

  • Scaling advocates note that the frontier keeps moving: the small model that matches last year’s large one still trails this year’s.
  • Small models are often distilled from large ones, so localism quietly depends on the very giants it criticizes.
  • Breadth of world knowledge does scale with parameters, so small models tend to fail on long-tail facts and open-ended tasks.

Representative works

Relationships

How to get it

Land on this cell in the full or pro test: x ≈ −0.59, y ≈ −0.77, paradigm ≈ −0.12.

A first draft. Corrections and better arguments are welcome through a pull request. Edit this entry on GitHub Edit its data