Retrieval-Augmented Generation (RAG) Architecture

Metadata

• Category: AI Engineering / Architectures
• Subcategory: Information Retrieval & Context Augmentation
• Difficulty: Advanced (University Textbook Standard)
• Estimated Reading Time: 60 minutes
• Prerequisites: Linear algebra, Transformer architecture basics, Information Retrieval (IR) fundamentals, Big-O complexity analysis.
• Learning Outcomes: Formulate vector embeddings mathematically, implement multi-language chunking algorithms, analyze HNSW memory models and time/space complexity, trace query execution latencies, and evaluate outputs using strict RAGAS metrics.

1. First Principles: The Need for RAG

To understand Retrieval-Augmented Generation (RAG), we must first examine the memory model of Large Language Models (LLMs).

Parametric Memory vs. Working Memory

Mental Model: Think of Parametric Memory as the LLM's long-term frozen weights, and the Context Window as its short-term working memory.

An LLM stores knowledge in its Parametric Memory—the billions of floating-point weights (parameters) learned during pre-training.

1. Limitations:
   • Knowledge Cutoff: Once training stops, parametric memory is frozen.
   • Hallucination: When parametric memory lacks precise facts, the model interpolates probabilistically, generating plausible but incorrect answers.
   • Privacy: Fine-tuning an LLM on private data bakes sensitive information into weights, making access control impossible.

RAG introduces an explicit Non-Parametric Working Memory. It decouples knowledge storage from language generation. When a query arrives, the system retrieves external facts and injects them into the LLM's Context Window (its active working memory).

Formulating RAG Mathematically

Let $q$ be the user query, $D$ be a massive document corpus, and $M$ be the language model. Standard Generation targets: $P(y \mid q) = \prod_{i} P(y_i \mid q, y_{<i})$

RAG introduces a retriever $R(q, D)$ that returns a set of relevant document chunks $Z = \{z_1, z_2, ..., z_k\}$. The new generation probability is: $P(y \mid q, D) = \prod_{i} P(y_i \mid q, Z, y_{<i})$

By explicitly conditioning the output $y$ on the retrieved evidence $Z$, the model is grounded, significantly reducing entropy (uncertainty) in fact-based generation.

2. Zero to One: Assembling the RAG Pipeline

Before analyzing HNSW graph traversal memory constraints and Big-O latencies, you must know how to actually build and execute a basic RAG query.

Prompt Templating (The Missing Link)

The entire point of RAG is to fetch data and inject it into the LLM's prompt. You must format the raw string carefully to prevent the LLM from hallucinating.

def build_prompt(user_query, retrieved_documents):
    context = """
---
""".join(retrieved_documents)
    
    prompt = f"""
    You are an expert assistant. Answer the user's question using ONLY the provided Context. 
    If the answer is not in the context, say "I don't know".
    
    Context:
    {context}
    
    Question: {user_query}
    Answer:
    """
    return prompt

End-to-End Generation Execution

Once you have the prompt, you call the LLM API (e.g., OpenAI, Anthropic, or local Ollama).

import openai

client = openai.OpenAI()

def generate_answer(prompt):
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.0, # 0.0 prevents creative hallucination
        stream=True      # Essential for UX to mask latency
    )
    
    # Streaming the tokens back to the user instantly
    for chunk in response:
        print(chunk.choices[0].delta.content or "", end="", flush=True)

Pre-filtering vs Post-filtering

When fetching documents, users often apply metadata filters (e.g., "Find docs about Python where year > 2023").

• Post-filtering (Bad): You retrieve the top 10 closest vectors, then filter out the old ones. Result: If all 10 are old, you return 0 results to the user. (Recall Collapse).
• Pre-filtering (Good but Hard): You apply the filter during the graph traversal inside the vector database (like Pinecone or Weaviate).

Reciprocal Rank Fusion (RRF) Intuition

If you use Semantic Search (Vectors) AND Keyword Search (BM25), you get two different ranked lists. RRF merges them mathematically by penalizing items that are ranked low in either list. If Doc A is #1 in Vector and #10 in Keyword, but Doc B is #2 in both, RRF will often rank Doc B higher overall because it has strong consensus.

3. Mathematical Foundations of Retrieval

Information retrieval requires mapping discrete text into comparable mathematical structures. RAG uses two primary paradigms: Dense (Vector) and Sparse (Lexical) representations.

3.1 Dense Vector Embeddings

An embedding model (e.g., text-embedding-3-large) maps text chunks $z$ into a continuous high-dimensional vector space $\mathbb{R}^d$ (where $d$ is typically 384, 768, 1536, or 3072).

Let $E: \text{Text} \rightarrow \mathbb{R}^d$ be the embedding function. The query is $v_q = E(q)$ and a chunk is $v_z = E(z)$.

Similarity Metrics: The goal is to find $\arg\max_z S(v_q, v_z)$.

1. Cosine Similarity: Measures the angle between vectors. Robust to chunk length variations. $\text{Cosine}(v_q, v_z) = \frac{v_q \cdot v_z}{\|v_q\| \|v_z\|} = \frac{\sum_{i=1}^{d} v_{q,i} v_{z,i}}{\sqrt{\sum_{i=1}^{d} v_{q,i}^2} \sqrt{\sum_{i=1}^{d} v_{z,i}^2}}$ Time Complexity: $O(d)$.

2. L2 Distance (Euclidean): $L2(v_q, v_z) = \sqrt{\sum_{i=1}^{d} (v_{q,i} - v_{z,i})^2}$ Note: If vectors are normalized ($\|v\| = 1$), L2 distance is monotonically related to Cosine Similarity ($L2^2 = 2 - 2 \cdot \text{Cosine}$).

3.2 Sparse Retrieval (BM25)

Dense vectors capture semantics ("canine" $\approx$ "dog") but fail at exact keyword matching (e.g., UUIDs, specific error codes). Sparse representations maintain a vocabulary-sized sparse vector.

Okapi BM25 improves upon TF-IDF by saturating term frequency and normalizing for document length: $\text{score}(q, z) = \sum_{t \in q} \text{IDF}(t) \cdot \frac{f(t, z) \cdot (k_1 + 1)}{f(t, z) + k_1 \cdot (1 - b + b \cdot \frac{|z|}{\text{avgdl}})}$ Where:

• $f(t, z)$ is the term frequency of $t$ in chunk $z$.
• $|z|$ is the chunk length, $\text{avgdl}$ is average chunk length.
• $k_1$ (term frequency saturation, usually $1.2-2.0$) and $b$ (length normalization, usually $0.75$).

4. Ingestion Pipeline & Chunking Algorithms

The Offline Ingestion Pipeline converts raw corpus documents into searchable indexes.

graph TD
    A[Raw Documents PDF, Markdown, HTML] --> B[Text Extraction & Cleaning]
    B --> C[Chunking / Splitting]
    C --> D[Embedding Model]
    C --> E[BM25 Indexer]
    D --> F[(Vector Database HNSW Index)]
    E --> G[(Inverted Index Sparse)]

4.1 Recursive Character Chunking

This algorithm recursively splits text using a hierarchy of separators (e.g., \n\n, \n, ) to keep semantically related text together while respecting maximum token limits.

Python Implementation:

def recursive_chunking(text: str, chunk_size: int, overlap: int, separators: list[str]) -> list[str]:
    """
    Recursively splits text.
    Time Complexity: O(N) where N is string length.
    Space Complexity: O(N) to store chunks.
    """
    if len(text) <= chunk_size:
        return [text]
    
    # Find the highest priority separator that exists in the text
    for sep in separators:
        if sep in text:
            splits = text.split(sep)
            chunks = []
            current_chunk = ""
            
            for split in splits:
                if len(current_chunk) + len(split) + len(sep) > chunk_size:
                    if current_chunk:
                        chunks.append(current_chunk)
                    # Backtrack for overlap
                    current_chunk = split
                else:
                    current_chunk += (sep + split) if current_chunk else split
            
            if current_chunk:
                chunks.append(current_chunk)
            return chunks
            
    # Fallback to arbitrary fixed-size splitting
    return [text[i:i+chunk_size] for i in range(0, len(text), chunk_size - overlap)]

4.2 Semantic Chunking

Instead of syntactic characters, this method analyzes embedding distances between sequential sentences to find natural topic shifts.

Algorithm:

1. Split text into sentences $S_1, S_2, ..., S_n$.
2. Embed each sentence: $v_1, v_2, ..., v_n$.
3. Compute cosine distances between adjacent sentences: $D_i = 1 - \text{Cosine}(v_i, v_{i+1})$.
4. If $D_i > \tau$ (a predefined threshold or 95th percentile), split at $i$.

Python Implementation:

def cosine_distance(v1: list[float], v2: list[float]) -> float:
    """Computes the distance between two normalized vectors."""
    dot_product = sum(a * b for a, b in zip(v1, v2))
    return 1.0 - dot_product

def semantic_chunk(sentences: list[str], embeddings: list[list[float]], threshold: float) -> list[list[str]]:
    """
    Groups sentences based on topic drift.
    Time Complexity: O(N * d) where N is sentences, d is dimensions.
    """
    chunks = []
    current_chunk = [sentences[0]]

    for i in range(len(sentences) - 1):
        dist = cosine_distance(embeddings[i], embeddings[i+1])
        if dist > threshold:
            # Topic changed, commit current chunk
            chunks.append(current_chunk)
            current_chunk = []
        current_chunk.append(sentences[i+1])
        
    if current_chunk:
        chunks.append(current_chunk)
    return chunks

4.3 Guided Exercises: Chunking Structured Data

Standard text splitters fail on structured data. Try these approaches:

1. CSV by Row + Headers: Never chunk CSVs by arbitrary character counts. Instead, chunk row-by-row, explicitly prefixing each row with its column headers. Exercise: Write a script that reads a sales.csv and converts row[0] into "Date: 2023-01-01, Product: Widget, Revenue: $500" before embedding.
2. Nested JSON: Deeply nested JSON loses context if split arbitrarily. Flatten the JSON into dot-notation paths or summarize parent nodes. Exercise: Write a recursive function that converts {"user": {"id": 123, "settings": {"theme": "dark"}}} into "user.settings.theme: dark" as a distinct chunk.
3. Python Code by Function: Splitting code randomly breaks syntax. Use an Abstract Syntax Tree (AST) parser (like ast in Python or tree-sitter) to chunk code strictly at function or class boundaries. Exercise: Parse a Python file using the ast module and extract each ast.FunctionDef into a separate chunk, appending the class docstring as context.

4.4 Document Lifecycle Management

A production RAG system must handle documents changing over time. Treating the vector database as an append-only log leads to stale data and hallucinations.

• Upserts: When a document is modified, you must overwrite its existing chunks. Use deterministic IDs (e.g., hash(doc_url + chunk_index)) so that re-ingesting the same document automatically overwrites old vectors instead of duplicating them.
• Soft Deletes vs. Hard Deletes: Vector databases (like HNSW-based ones) struggle with hard deletes, as removing nodes can break graph connectivity. Instead, use soft deletes by adding a boolean metadata flag (is_deleted: true). Then, use metadata pre-filtering to exclude them from search results.
• Index Synchronization: Keep the Source of Truth (e.g., Postgres/S3) synced with the Vector DB. Use Change Data Capture (CDC) pipelines (like Debezium) or periodic reconciliation jobs to ensure the vector index perfectly reflects the current active corpus.

5. Vector Indexing and HNSW (Hierarchical Navigable Small World)

Performing brute-force KNN (K-Nearest Neighbors) across millions of $d$-dimensional vectors takes $O(N \cdot d)$ time, which is unacceptable for real-time systems. Vector databases (e.g., Pinecone, Milvus, Qdrant) use Approximate Nearest Neighbor (ANN) algorithms, the most prominent being HNSW.

HNSW Architecture & Memory Model

HNSW is a multi-layered proximity graph.

• Layer 0 contains all $N$ elements.
• Layer 1 contains a subset, Layer 2 an even smaller subset, etc. (Exponential decay, similar to a Skip List).
• Search Complexity: $O(\log N)$ time.
• Space Complexity: $O(N \cdot M + N \cdot d)$, where $M$ is the maximum number of connections per node.

Execution Trace of HNSW Search:

1. Start at the single entry point in the highest layer.
2. Greedily traverse neighbors to find the node closest to query $v_q$.
3. Drop down to the next layer, using the best node from the previous layer as the new entry point.
4. Repeat until reaching Layer 0. Return the local minimum.

Edge Case: "Hub nodes" in Layer 0 can cause massive memory bloat if $M$ is not strictly capped. HNSW enforces a heuristic edge-selection algorithm to maintain diversity and limit graph degree.

6. The Query Pipeline: Hybrid Search & Reranking

The online pipeline must balance speed, recall, and precision.

sequenceDiagram
    participant User
    participant App
    participant VectorDB
    participant BM25
    participant CrossEncoder
    participant LLM

    User->>App: "How do I reset error code E-499?"
    App->>VectorDB: Query Embedding (Dense)
    App->>BM25: Query String (Sparse)
    
    VectorDB-->>App: Top 100 Semantic Matches
    BM25-->>App: Top 100 Keyword Matches
    
    App->>App: Reciprocal Rank Fusion (RRF)
    Note right of App: O(K log K) Sort
    App->>CrossEncoder: Top 50 Fused Candidates
    Note right of CrossEncoder: O(K * N^2) Transformer Attention
    CrossEncoder-->>App: Top 5 Ranked Chunks
    
    App->>LLM: Prompt + Top 5 Chunks + Query
    LLM-->>User: Grounded Response

6.1 Reciprocal Rank Fusion (RRF)

How do we combine scores from BM25 (unbounded positive floats) and Cosine Similarity ([-1, 1])? We cannot sum them directly. RRF ignores the absolute scores and fuses based on rank position.

$\text{RRF\_Score}(z) = \sum_{r \in \text{Retrievers}} \frac{1}{k_{\text{rrf}} + \text{rank}_r(z)}$ Where $k_{\text{rrf}}$ is a smoothing constant (typically 60).

6.2 Bi-Encoders vs. Cross-Encoders

• Bi-Encoder (Vector Retrieval): The query and chunk are embedded independently. $E(q)$ and $E(z)$ are pre-computed (or computed once at runtime for $q$). The interaction is a simple dot product. Fast, but lacks deep contextual understanding of how query terms interact with chunk terms.
• Cross-Encoder (Reranker): The query and chunk are concatenated and passed through a Transformer model together: [CLS] Query [SEP] Chunk [EOS]. Self-attention mechanisms calculate the attention between every word in the query and every word in the chunk.
  • Time Complexity: $O((L_q + L_z)^2)$ per chunk. This is why we only rerank the top-K (e.g., $K=50$) candidates, not the whole database.

7. Execution Trace & Latency Budgeting

A production RAG request must be strictly budgeted for latency (Target: $<1000$ms to First Token).

| Step | Operation | Compute Location | Time Complexity | Typical Latency | | :--- | :--- | :--- | :--- | :--- | | 1 | Query Embedding | Embedding API / GPU | $O(L_q^2)$ | 30-50 ms | | 2 | Sparse Retrieval | ElasticSearch / DB | $O(\|q\| \cdot \log N)$ | 10-30 ms | | 3 | Dense Retrieval | Vector DB (HNSW) | $O(\log N \cdot d)$ | 10-30 ms | | 4 | RRF Fusion | CPU (App Server) | $O(K)$ | < 1 ms | | 5 | Cross-Encoder Rerank | GPU Node | $O(K \cdot (L_q + L_z)^2)$ | 100-250 ms | | 6 | LLM Pre-fill (Prompting)| LLM GPU (vLLM) | $O((L_{prompt} + \sum L_{chunks})^2)$ | 150-400 ms | | 7 | Time-to-First-Token | - | - | 300 - 760 ms |

Edge Case: What if a user submits a 5000-token query? The Cross-Encoder reranking step will OOM (Out Of Memory) or timeout due to quadratic scaling. Solution: Cap query length before routing to Cross-Encoder.

8. Advanced RAG Architectures

1. HyDE (Hypothetical Document Embeddings):
   • Problem: User queries are often short questions ("what is SLA?"), whereas documents are long statements. They occupy different areas of the vector space.
   • Solution: Pass the query to a fast, cheap LLM and ask it to "answer" the question. Even if it hallucinates, the structure of the generated text resembles the target document. Embed this Hypothetical Document instead of the query for dense retrieval.
2. GraphRAG:
   • Entities and relationships are extracted via LLMs during ingestion into a Knowledge Graph. Retrieval traverses graph edges to pull multi-hop context (e.g., "Company A -> owned by -> Company B -> acquired -> Product X").
3. Multi-Tenant Vector Space Isolation (RBAC Case Study):
   • Problem: In enterprise RAG, users must only retrieve documents they have access to (Role-Based Access Control). Using a single massive vector space without boundaries leaks confidential data.
   • Solution: Embed an access_level or tenant_id in the metadata of every chunk. Use Metadata Pre-filtering in the vector database to restrict the HNSW graph traversal to only nodes matching the user's tenant_id. This guarantees strict isolation at the database layer without sacrificing recall, unlike post-filtering which causes "Recall Collapse."

9. Evaluation Criteria (RAGAS Framework)

Evaluating RAG requires treating the pipeline as two distinct segments: Retrieval ($R$) and Generation ($G$).

1. Context Precision (Evaluating $R$): Was the highly relevant context ranked at the top?
2. Context Recall (Evaluating $R$): Did the retriever find all the necessary facts to answer the question?
3. Faithfulness (Evaluating $G$): Measures hallucinations. Given the generated answer, can every stated claim be logically deduced via Natural Language Inference (NLI) exclusively from the retrieved context? $\text{Faithfulness} = \frac{\text{Number of Claims Supported by Context}}{\text{Total Number of Claims in Answer}}$
4. Answer Relevance (Evaluating $G$): Uses embedding distance or an LLM judge to determine if the generated answer directly addresses the user query, penalizing evasive answers.

10. Interview Questions

Question 1: Explain the mathematical distinction between Cosine Similarity and L2 distance, and why normalization matters. Answer: L2 distance measures straight-line magnitude in Euclidean space, while Cosine Similarity measures the angle between vectors. If vector magnitude encapsulates information (e.g., term frequency density), L2 differs from Cosine. However, if vectors are normalized to a magnitude of 1 (projected onto a unit hypersphere), $L2^2 = 2(1 - \cos(\theta))$. Vector databases perform faster inner-product computations when vectors are pre-normalized, acting mathematically equivalent to Cosine similarity.

Question 2: What is the Big-O Time Complexity of processing a user query through a Cross-Encoder versus a Bi-Encoder? Answer: For a dataset of $N$ documents, a Bi-Encoder takes $O(d)$ per chunk, totaling $O(N \cdot d)$ (or $O(\log N \cdot d)$ with ANN). A Cross-Encoder concatenates query $L_q$ and chunk $L_z$, passing them through a Transformer with self-attention complexity $O((L_q + L_z)^2)$. Running this across $N$ documents takes $O(N \cdot (L_q + L_z)^2)$, making it computationally intractable for full retrieval, thereby necessitating a two-stage retrieve-and-rerank pipeline.

Question 3: How does the Reciprocal Rank Fusion (RRF) algorithm handle the "scale disparity" problem in Hybrid Search? Answer: BM25 produces unbounded non-normalized scores based on term frequencies, whereas dense Cosine similarities are tightly bounded between [-1, 1]. RRF eliminates score scale entirely by operating strictly on the ordinal rank position of the items. By summing the inverse of their ranks ($1 / (k + rank)$), it probabilistically favors documents that perform well in both retrieval methods without requiring complex distribution normalization.

Question 4: Describe an edge case where Semantic Chunking produces suboptimal results and how to mitigate it. Answer: Semantic chunking computes distances between adjacent sentences. If a document interleaves code snippets and explanatory text, adjacent sentence embeddings might swing wildly (e.g., English -> Python -> English), causing fragmentation into tiny chunks. Mitigation: Apply a sliding window smoothing function to the distances, or use a multi-modal embedding model that inherently aligns code and text semantic spaces.

Question 5: A user complains that the RAG system provides outdated answers, despite the vector DB being updated daily. Walk through the debugging execution trace. Answer:

1. Check parametric memory leakage: Ensure the generation prompt strictly commands Answer ONLY using the provided context.
2. Evaluate Context Recall: Is the retriever fetching the new document? If no, inspect BM25 indexing (was the inverted index rebuilt?) and Vector Graph (did HNSW insertion drop the node to a low layer without updating the entry point?).
3. Evaluate Metadata Filtering: Are we enforcing temporal decay or filtering? A time-weighted retrieval score (e.g., $S_{final} = S_{vec} \cdot e^{-\lambda t}$) must be properly tuned to prioritize recent chunks.

11. FAQs

Q: Is RAG better than fine-tuning? A: They solve different problems. Fine-tuning teaches the model new skills (tone, format, domain language). RAG injects facts at runtime. For knowledge that changes frequently (daily price feeds, support docs), RAG is always preferred because you only need to update the vector index, not retrain the model. For stable behavioral patterns (writing in a specific style, solving domain-specific code), fine-tuning wins. In production, they are often combined.

Q: What is the optimal chunk size? A: There is no universal answer. Chunk size is a hyperparameter tuned by evaluating Context Recall on your dataset. Small chunks (128–256 tokens) give precise retrieval but may lack context. Large chunks (512–1024 tokens) provide rich context but may retrieve irrelevant sentences that dilute the answer. Recommended starting point: 512 tokens with 10% overlap, evaluated with RAGAS Context Precision.

Q: Can RAG hallucinate even with perfect retrieval? A: Yes. The LLM generation step can still hallucinate if the system prompt is weak, the model is small, or the retrieved context is partially contradictory. The Faithfulness metric in RAGAS directly measures this. Always include an instruction like Answer ONLY using the provided context. If the answer is not in the context, say "I don't know".

Q: What vector database should I use in production? A: Key selection criteria: (1) Managed vs. self-hosted, (2) Filtering support (metadata filters during ANN search), (3) Hybrid search support (built-in BM25 + dense). For managed cloud: Pinecone. For self-hosted with full hybrid search: Qdrant or Weaviate. For large-scale enterprise: Milvus.

12. Projects

Project 1: Build a Document QA RAG System

Goal: Create an end-to-end RAG pipeline that ingests a PDF and answers questions.

Steps:

1. Parse a PDF using pypdf2 or pdfplumber.
2. Chunk text using recursive character splitting (chunk size 512, overlap 50).
3. Embed chunks with sentence-transformers (all-MiniLM-L6-v2).
4. Store embeddings in ChromaDB (local vector store).
5. At query time: embed the query, retrieve top-5 chunks, inject into an LLM prompt.
6. Use ollama (llama3) or OpenAI's API for generation.

import chromadb
from sentence_transformers import SentenceTransformer

client = chromadb.Client()
collection = client.create_collection("docs")
model = SentenceTransformer('all-MiniLM-L6-v2')

def ingest(chunks: list[str]):
    embeddings = model.encode(chunks).tolist()
    ids = [str(i) for i in range(len(chunks))]
    collection.add(embeddings=embeddings, documents=chunks, ids=ids)

def retrieve(query: str, k: int = 5) -> list[str]:
    q_emb = model.encode([query]).tolist()
    results = collection.query(query_embeddings=q_emb, n_results=k)
    return results['documents'][0]

Project 2: Hybrid Search Evaluation Harness

Goal: Compare BM25-only, Dense-only, and Hybrid RAG pipelines on a custom dataset using RAGAS Context Recall.

Steps:

1. Create a 50-question Q&A evaluation dataset from a domain document.
2. Implement three retrieval pipelines: BM25 (rank_bm25 library), Dense (ChromaDB), and Hybrid (RRF fusion).
3. Evaluate each using ragas library: from ragas.metrics import context_recall.
4. Plot a comparison bar chart showing recall scores for each strategy.

Expected outcome: Hybrid search beats both individual strategies by 15–30% on recall for domain-specific questions with mixed keyword and semantic content.

13. Revision Notes

| Concept | Key Fact | | :--- | :--- | | RAG Purpose | Decouples knowledge from model weights; enables updatable, private, grounded generation | | Cosine Similarity | Measures angle between vectors; range [-1, 1]; robust to vector magnitude | | BM25 | Sparse keyword retrieval; handles exact terms (UUIDs, codes) that dense models miss | | HNSW | ANN graph index; $O(\log N)$ search; multi-layered skip-list-like structure | | RRF | Rank fusion; solves score-scale disparity between BM25 and dense scores | | Cross-Encoder | $O((L_q + L_z)^2)$ reranker; deep contextual attention; use only on top-K candidates | | Faithfulness | RAGAS metric; claims in answer verifiable against retrieved context | | Chunking Tradeoff | Small chunks = high precision, low context. Large = rich context, noisy retrieval |

Debugging Guide

Debugging a Retrieval-Augmented Generation (RAG) system can be complex because errors can stem from the retriever, the generator, or the glue between them.

Common Bugs and Fixes:

1. Model Hallucinates Despite Good Context Fix: This usually means the Prompt Template is not strict enough. Enforce strict bounding phrases such as, "You are a factual answering assistant. You must ONLY use the provided context to answer the user query. If the context does not contain the answer, you must output 'I do not know' and nothing else." Additionally, lower the LLM's temperature parameter to 0.0 to reduce probabilistic creativity.
2. Poor Retrieval of Domain-Specific Keywords Fix: Pure dense vector retrieval (Cosine/L2) struggles with exact acronyms (e.g., 'UUID-904', 'AWS-EC2'). The fix is to implement Hybrid Search. Add a sparse retrieval mechanism like BM25 to capture exact token matches, and combine the scores with the vector retrieval using Reciprocal Rank Fusion (RRF).
3. Truncated or Incomplete Answers Fix: The LLM might be running out of context window tokens or hitting a max token generation limit. Check the token count of your retrieved chunks. If you are retrieving 10 chunks of 1024 tokens each, you might be exceeding the context window of smaller models. Reduce the chunk size to 512, retrieve fewer chunks, or increase the max generation length.
4. Slow Query Response Time (High Latency) Fix: If your pipeline takes several seconds to generate the first token, investigate the cross-encoder reranking step. Rerankers scale quadratically and will cause massive latency if you pass too many chunks to them. Cap the initial retrieval pool to the top 20-50 chunks before passing them to the reranker.

Assignments

Assignment 1: Build a Basic Document QA System

Objective: Create a fundamental Retrieval-Augmented Generation pipeline using LangChain or LlamaIndex to understand the core mechanics of ingestion and retrieval. Deliverables:

1. Ingest a small dataset of 5-10 text documents (e.g., historical articles or technical documentation).
2. Chunk the documents using a recursive character text splitter. Experiment with chunk sizes of 512 and 1024 tokens.
3. Generate embeddings using an open-source embedding model (like sentence-transformers/all-MiniLM-L6-v2).
4. Store these embeddings in a local vector database such as ChromaDB or FAISS.
5. Connect a large language model to the retriever and create a prompt template that strictly limits the model to only answer using the retrieved context. Write a short markdown report comparing the response quality when using 512-token chunks versus 1024-token chunks, and include at least 5 sample queries and their generated answers.

Assignment 2: Implement Advanced Retrieval Techniques

Objective: Upgrade your basic RAG pipeline to handle complex queries using advanced retrieval methods and hybrid search. Deliverables:

1. Implement Multi-Query Retrieval: Modify your retrieval step to generate multiple variations of the user's query using an LLM, retrieve documents for all variations, and take the unique union of the results.
2. Implement Re-ranking: Add a cross-encoder model (like BAAI/bge-reranker-base) to re-score and re-order the retrieved documents before passing them to the generator model. Submit a fully functioning Python application or Jupyter Notebook. Provide a benchmark script that runs 20 difficult queries and logs the relevance of the retrieved documents before and after the re-ranking step is applied. Add a README file explaining how to execute the evaluation.

Testing Strategy

Testing a Retrieval-Augmented Generation (RAG) pipeline requires evaluating both the retrieval component and the generation component independently, as well as assessing the system as a whole. A robust testing strategy ensures that your application is accurate, grounded, and resistant to hallucinations.

1. Evaluating the Retriever: The retrieval phase must be tested to ensure it fetches the most relevant context for a given query. Use standard information retrieval metrics such as Mean Reciprocal Rank (MRR), Normalized Discounted Cumulative Gain (NDCG), and Precision@K. Create a golden dataset containing pairs of queries and the exact document IDs that contain the answers. Regularly run automated tests against this golden dataset every time you update your embedding model, chunking strategy, or vector database parameters to ensure retrieval performance does not degrade.

2. Evaluating the Generator: The generation phase needs to be tested for faithfulness (groundedness) and answer relevance. Use LLM-as-a-judge frameworks like RAGAS or TruLens to evaluate the outputs. Faithfulness checks if the generated answer can be entirely deduced from the retrieved context, penalizing any hallucinated information. Answer relevance checks if the generated response directly addresses the user's initial prompt without unnecessary verbosity.

3. End-to-End System Testing: Conduct integration testing by simulating real-world user interactions. Establish a diverse set of test queries, including edge cases, adversarial prompts, and multi-turn conversational follow-ups. Monitor the end-to-end latency to ensure the pipeline responds within acceptable time limits (e.g., time to first token). Incorporate human-in-the-loop evaluation for continuous feedback, capturing user ratings (thumbs up/down) to iteratively fine-tune both the prompt templates and the retrieval configuration over time.