Instruction Corporatism
From the AI Research Compass wiki
Instruction Corporatism: Holds that capability is organized and unlocked by instruction tuning, so every task should be cast as a natural-language instruction.
Beliefs
Instruction Corporatism believes in a well-ordered society of tasks. Translation, summarization, classification, code repair: each is a guild, and each guild is represented by instructions written in plain language. Pretraining supplies raw capacity, but it is instruction tuning that turns a text predictor into something that does what it is asked. Collect thousands of tasks, phrase them as instructions, mix them carefully, and a single model learns to generalize to instructions it has never seen.
It sits near the Language edge and near the top of the chart: one model, whose behavior is shaped by a large, curated system of task formats absorbed into its weights rather than handled by an engineered pipeline. Its paradigm level is orthodox. It was once the new idea that made zero-shot prompting practical; today it is core infrastructure in nearly every post-training recipe.
Arguments
- Fine-tuning on a broad mixture of tasks phrased as instructions improves zero-shot performance on held-out tasks, a result replicated across many model families.
- Instructions are a universal interface: any task a person can describe can be added without changing the architecture.
- Models can generate their own instruction data, letting the corporatist system expand cheaply.
Critiques
- Preference-learning camps argue instruction data teaches format more than judgment, and that feedback-based training matters more for real helpfulness.
- Instruction mixtures inherit the biases of whoever chose and wrote the tasks; benchmark contamination is a constant worry.
- Reasoning-focused rivals say that following instructions is not the same as solving hard problems, which needs search or long deliberation.
Representative works
- Finetuned Language Models Are Zero-Shot Learners2021
- Training language models to follow instructions with human feedback2022
- Self-Instruct: Aligning Language Models with Self-Generated Instructions2022
Relationships
Friends
Frenemies
How to get it
Land on this cell in the full or pro test: x ≈ −0.73, y ≈ +0.70, paradigm ≈ −0.76.