RAG
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela
The paper in one sentence
RAG gives a generator access to an external, updateable document memory instead of relying only on facts stored in model weights.
What the paper does
RAG combines a pretrained sequence-to-sequence generator with a non-parametric document index. A learned retriever selects relevant passages, and the generator conditions on those passages when producing an answer. The paper studies variants that use one passage set for an entire output or retrieve separately for each generated token.
Why it matters to engineers
The paper established a practical pattern for grounding language-model outputs in retrievable evidence. Production RAG systems extend the pattern with ingestion, chunking, access control, reranking, citations, evaluation, and observability.
Key claims from the paper
- Parametric generation and non-parametric retrieval can be trained together for knowledge-intensive tasks.
- Retrieved passages can improve factual specificity and allow provenance to be inspected.
- The external knowledge index can be updated without retraining every fact into the generator.
Architecture, step by step
- 1
Knowledge index
Documents are encoded as dense vectors and stored in a searchable external index.
- 2
Retriever
A query encoder scores documents and returns a small set of likely relevant passages.
- 3
Generator
A sequence-to-sequence model conditions on the query and retrieved passages to produce the output.
- 4
Marginalization
The model combines generation probabilities across retrieved documents rather than treating one passage as certainly correct.
Production takeaways
- Retrieval quality sets an upper bound on grounded answer quality; evaluate recall before tuning generation prompts.
- Store document identity and permissions alongside chunks so citations and access control survive retrieval.
- Chunk size, overlap, embedding choice, metadata filters, and reranking should be evaluated as one retrieval system.
- Measure answer correctness and citation support separately: a fluent answer can still be unsupported.
Limitations and caveats
- Retrieval can return irrelevant or stale passages, and generation may ignore or misread good evidence.
- The paper’s Wikipedia-based setup does not address enterprise authorization, deletion, or multi-tenant isolation.
- Adding retrieval introduces indexing, latency, freshness, and observability concerns.
Interview questions to test understanding
- How would you measure whether a RAG failure came from retrieval or generation?
- When should a system use metadata filtering before vector similarity search?
- How would you keep an index fresh while preserving citation stability?
- What signals would you log for a production RAG request?