Emergentism

From the AI Research Compass wiki

Emergentism: Qualitatively new abilities appear unpredictably once models pass certain scales, so scaling itself is the engine of discovery.

Beliefs

Emergentism is the romantic wing of the scaling movement. Where Scaling-Law Planism sees smooth, forecastable curves, Emergentism sees phase transitions: abilities such as multi-step arithmetic, in-context learning or instruction following that are absent in smaller models and then suddenly present in larger ones. More is different. The right attitude is therefore not to engineer each capability, but to scale and watch what appears.

It sits at the very top of the chart, near the Language edge: language-leaning because most celebrated emergent abilities were observed in language models, and Monolith because emergence is a property of one big network, not of an assembled system. Its paradigm level is orthodox. The idea is a pillar of the mainstream scaling story, however often it is disputed.

Arguments

  • Benchmarks across many tasks show performance near chance for small models and sharply higher for large ones, consistent with emergent abilities.
  • In-context learning from a few examples was not designed in; it appeared as models were scaled.
  • Emergence offers a reason to keep scaling even when no specific target capability is in sight.

Critiques

  • Some apparent jumps may be artifacts of discontinuous metrics such as exact-match accuracy; smoother metrics can show gradual improvement.
  • “It will emerge” can become an unfalsifiable promise that substitutes for understanding mechanisms.
  • Planners counter that unpredictable capabilities are a problem for safety and evaluation rather than a cause for celebration.

Representative works

Relationships

How to get it

Land on this cell in the full or pro test: x ≈ −0.57, y ≈ +1.00, paradigm ≈ −0.75.

A first draft. Corrections and better arguments are welcome through a pull request. Edit this entry on GitHub Edit its data