Video Scalism

From the AI Research Compass wiki

Video Scalism: Video is the next internet-scale corpus, and scaling models on it will teach them about the world the way text taught them language.

Beliefs

Video Scalism looks at the slowing supply of fresh human text and points at a far larger reservoir: the world’s video. Hours of footage show physics, tool use, social interaction and cause and effect in a way no written description can. If text pretraining produced models that understand language, video pretraining at comparable scale should produce models that understand the visual and physical world. The recipe is the same; only the corpus changes.

It sits near the top, well toward World: perception is its substance, and time is treated as a first-class dimension. The Monolith lean comes from its scaling faith in a single large model trained on a giant corpus. Its paradigm level is Ultra-Orthodox: it keeps today’s scaling recipe intact and only swaps the corpus. Video at that scale strains storage, tokenization and compute, and the payoff is still a forecast rather than an accounting entry.

Arguments

  • Large vision models trained purely on sequences of images and video frames show in-context abilities reminiscent of language models.
  • Video contains temporal dynamics and physical interaction that static images and text largely lack.
  • The volume of available video dwarfs high-quality text, offering a path past the text data wall.

Critiques

  • Video is highly redundant; most frames add little information, so raw scale overstates the useful signal.
  • Compute and storage costs per unit of knowledge are far higher than for text.
  • Language-first rivals note that much of what matters for reasoning is abstract and rarely visible on camera.

Representative works

Relationships

How to get it

Land on this cell in the full or pro test: x ≈ +0.79, y ≈ +0.78, paradigm ≈ −0.89.

A first draft. Corrections and better arguments are welcome through a pull request. Edit this entry on GitHub Edit its data