subtopic

Where plain RAG fails and how we will judge

Run your LLM-module RAG on 20 new, harder questions: ones that need two sources ('Is product X in stock and what is its return policy?'), vague questions that need rewriting, and questions where the first search finds the wrong pages. Note the failures. Scores: answer correctness, faithfulness, and cost/time — compared with plain RAG on the old 50 and the new 20 questions.

Builds on: 4.1 Understand the problem, 4.2 Data and test set, 4.3 Prepare the documents, 4.4 Generate the answer, 4.5 Use a framework, 4.6 Evaluate