RAG Mutualism
From the AI Research Compass wiki
RAG Mutualism: A language model and an external index work best as partners, the index supplying facts and the model supplying reasoning and fluency.
Beliefs
RAG Mutualism believes in a cooperative arrangement between parametric and non-parametric memory. The model is good at understanding queries, synthesizing and writing; the index is good at holding large, current, citable facts. Neither should try to do the other’s job. Retrieve the relevant passages, put them in context, and let the model reason over them: knowledge stays updatable without retraining, answers can point to their sources, and the model can spend its parameters on skill rather than trivia.
It sits left of centre on the Language side, near the Harness end: retrieval is a system-level design with a retriever, an index and a generator. It is the practical, cooperative cousin of Retrieval Statism just to its left, which wants knowledge out of the weights as a matter of principle; mutualists are content as long as the two help each other. Its Mainstream shade marks an established school: retrieval augmentation is now standard practice in enterprise and search-backed assistants.
Arguments
- Retrieval lets a system answer with up-to-date or private information without retraining the model.
- Grounding answers in retrieved passages enables citations and tends to reduce unsupported claims on knowledge-intensive tasks.
- Retrieval-augmented models can match larger models on some knowledge benchmarks with fewer parameters, since facts live in the index.
Critiques
- Retrieval quality is a bottleneck: if the wrong passages come back, the model confidently answers from them.
- Long-context advocates argue that simply loading whole documents into context avoids retrieval errors and pipeline complexity.
- Chunking, embedding and ranking add many hand-tuned components that are hard to evaluate end to end.
Representative works
- REALM: Retrieval-Augmented Language Model Pre-Training2020
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks2020
- Improving language models by retrieving from trillions of tokens2021
Relationships
Friends
How to get it
Land on this cell in the full or pro test: x ≈ −0.59, y ≈ −0.96, paradigm ≈ −0.54.