Test-Time Centralism
From the AI Research Compass wiki
Test-Time Centralism: Spending more compute at inference, through sampling, search and longer thinking, is a scaling axis as important as pretraining.
Beliefs
Test-Time Centralism argues that the field spent years scaling only one side of the ledger. Pretraining compute grew by orders of magnitude, while each answer was still produced in a single forward pass per token with no second thoughts. The doctrine moves the budget to inference: sample many candidates and vote, search over reasoning trees, verify and revise, or simply let the model think longer. A smaller model that thinks harder can beat a bigger one that answers at once.
It sits in the upper half, well toward the Language side. The language lean reflects that most test-time methods operate over text reasoning traces. The Monolith lean is real but moderate: inference-time search sometimes uses verifiers and simple scaffolds, yet the core remains one model allotted more compute. Its paradigm level is Steady, one step from Mainstream: inference scaling extends today’s recipe, and became a mainstream axis only recently.
Arguments
- Sampling multiple reasoning paths and taking the majority answer consistently improves accuracy over a single greedy chain.
- Tree-structured search over intermediate thoughts solves planning puzzles that single-pass generation fails on.
- Compute-optimal analyses show that allocating test-time compute adaptively can rival scaling model parameters substantially.
Critiques
- Inference compute is paid on every query, so the approach shifts cost onto users and deployment.
- Gains depend on having a reliable verifier or a task where consistency signals correctness.
- Efficiency-minded rivals argue that on-device and latency-sensitive applications cannot afford to think for minutes.
Representative works
- Self-Consistency Improves Chain of Thought Reasoning in Language Models2022
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models2023
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters2024
Relationships
Friends
How to get it
Land on this cell in the full or pro test: x ≈ −0.70, y ≈ +0.42, paradigm ≈ −0.25.