Scale Sublimism

From the AI Research Compass wiki

Scale Sublimism: Ring title for strong Monolith plus a strongly Mainstream paradigm: the recipe works, so the answer to every problem is the same model, a little bigger.

Beliefs

Scale Sublimism is a ring title awarded when two conditions hold together: a strong Monolith score (y above 0.7) and a strongly Mainstream paradigm score (p below -0.6). It belongs to those who find the scaling curve not just useful but sublime, an awe-inspiring line that keeps going up. Its creed is “just a little bigger”: no new architecture is needed, no new objective, no exotic harness, only the same pretraining-plus-post-training recipe with one more order of magnitude of compute, data and parameters.

It sits on the top edge of the chart, committed to the single end-to-end model and comfortable on either side of the Language-World axis. What distinguishes it from Compute Theocracy is the paradigm dimension: where the theocrat worships compute itself, the sublimist is specifically loyal to the current recipe and sceptical of anyone promising a revolution.

Arguments

  • Compute-optimal scaling studies showed that balancing model size and training tokens yields large gains from the existing recipe without architectural changes.
  • Many proposed alternatives to the Transformer have failed to beat a well-tuned, scaled-up Transformer when compared at equal compute.
  • Incremental scaling has delivered steady, predictable improvement, while paradigm bets have high variance and long timelines.

Critiques

  • “A little bigger” each time adds up to enormous cost, and returns per dollar may be falling even while absolute capability rises.
  • Clinging to the mainstream paradigm risks missing the next shift, just as earlier techniques were scaled past their usefulness before being replaced.
  • Some weaknesses, such as continual learning, reliable reasoning and grounding, show little sign of being solved by size alone.

Representative works

Relationships

How to get it

Earned in the full or pro test when: Architecture >0.7, Paradigm <−0.6.

A first draft. Corrections and better arguments are welcome through a pull request. Edit this entry on GitHub Edit its data