RAG (Retrieval-Augmented Generation)

RAG has two steps. First, the question is converted into a vector and the semantically closest passages in the document archive are located. Second, those passages are handed to the language model as context alongside the question, and the model produces its answer from that context alone.

The critical part is the second step: the model answers from the passages placed in front of it, not from its training data. That is why the source of an answer can be shown and verified. In a corporate knowledge base, this is the difference between a guess and a quotation.

Retraining a model on your own data is expensive and must be repeated on every document change. With RAG, an updated document only needs re-indexing; the model is left untouched.

The second difference is auditability. There is no way to show where a fine-tuned model got a piece of information; it is dissolved into the weights. With RAG every answer points at a concrete document passage. At SME scale, those two differences make RAG almost always the right choice.

RAG reduces hallucination but does not eliminate it. If the retrieval step selects the wrong passage, the model can produce a plausible-looking answer from the wrong context. That is why citation is not an optional feature but a required safety mechanism.

The second limit is the document itself. RAG cannot produce knowledge that is not written down in your archive, and a vague or contradictory document yields a vague answer. Answer quality directly reflects document quality.