Module 4 · Project 2: Company Policy Q&A Assistant (RAG — Retrieval-Augmented Generation)

OBJECTIVE: build a chat assistant that answers questions from your organisation's own documents (policies, product terms, HR handbook) with the page it used as a source, and says 'I don't know' when the answer is not there — and learn RAG end to end. WHAT WE COVER (in project order): • Why RAG: giving the LLM knowledge it does not have, without retraining • Collecting documents and building a question–answer test set • Reading PDFs and splitting them into chunks • Embeddings and a vector database (ChromaDB) • Retrieval + an answer prompt with citations and an 'I don't know' rule • Chat memory and context limits for follow-up questions • Building the same pipeline with LangChain and LlamaIndex • Evaluation: retrieval hit rate, answer correctness, faithfulness, refusals • Improving: query rewriting, chunk size, re-ranking, metadata filters, a glimpse of advanced RAG • Safety: personal-data masking, harmful input and prompt injection • Deployment: Gradio chat with sources, plus logging AFTER THIS MODULE YOU CAN: • Build and evaluate a RAG chatbot on any set of documents • Find whether a bad answer came from retrieval or from generation, and fix it DATA: 10–20 PDF documents (your organisation's, or public ones such as a bank's product terms and policies) + 50 questions you write with their correct answers and source pages. TOOLS: PyMuPDF, sentence-transformers or an embedding API, ChromaDB, an LLM.