AI Doomerism

From the AI Research Compass wiki

AI Doomerism: Ring title for extreme caution scores: advanced AI poses risks severe enough that development should slow or pause until safety catches up.

Beliefs

AI Doomerism is a ring title earned by acceleration scores below -0.6, the opposite pole of Effective Accelerationism, on the right ring outside the main axes. Its holders think interpretability research deserves more resources than capability research, that independent bodies should safety-test models before deployment, and that more agent autonomy is not automatically better. The motto “Pause. Pause now.” captures the strongest form: capabilities are advancing faster than our ability to understand, evaluate and control them, and the prudent move is to stop racing.

Because the title concerns speed and risk rather than method, a doomer can believe in language monoliths or embodied swarms alike; what matters is the conviction that the downside is potentially catastrophic and irreversible. Its orthodox shade is a little unfair to a camp that sees itself as dissenting, but reflects that it proposes no new research paradigm: it accepts the current recipe as powerful and asks only that it slow down.

Arguments

  • Concrete technical problems such as reward hacking, unsafe exploration and distributional shift are documented in current systems and grow more consequential as systems gain autonomy.
  • Interpretability and evaluation methods lag far behind capability, so developers often cannot explain or reliably predict model behaviour.
  • When a failure could be irreversible, the usual “deploy and iterate” approach does not apply, which justifies precaution.

Critiques

  • Catastrophic scenarios rely on long chains of speculative assumptions that are hard to test empirically.
  • Pauses are hard to verify and enforce internationally, and may simply shift development to less careful actors.
  • Persistent warnings about future catastrophe can divert attention from concrete present harms such as bias, misuse and labour impacts.

Representative works

Relationships

How to get it

Earned in the full or pro test when: Speed <−0.6.

A first draft. Corrections and better arguments are welcome through a pull request. Edit this entry on GitHub Edit its data