Cards/AI SystemsSystem Design · Day 7Aug 7, 2026

Your RAG Isn't Hallucinating

Your RAG Isn't Hallucinating — system design card, day 7, ai systems

The model gets blamed. The retriever is usually at fault.

RAG has two halves. Find the right passages, then write from them. If the right passage never arrives, no model on earth saves you — it will answer confidently from whatever you did send.

Where it actually breaks:

Chunking. Split at 500 characters and you cut a table in half. The answer exists, in two useless pieces.

Vocabulary. The user types "ARR." The document says "annual recurring revenue." Pure vector search can miss it. Keyword search wouldn't have.

Top-k. You retrieve 5. The answer ranked 9th.

Fix retrieval before you touch the prompt. Measure whether the correct chunk was in the context at all — that one number tells you which half is broken.

Before blaming the answer, check what the question was allowed to see.