Rag Implementation Cookbook

---
id: PROMPT-006
version: 1.0
author: AI Agent Coding Framework
last_updated: 2026-05-13
applicable_stack: [Python, LangChain, ChromaDB/FAISS, OpenAI]
category: RAG_Pipeline
difficulty: Intermediate
domain: Healthcare
---

# Prompt: RAG Implementation — Safe Medical Data Retrieval & Generation

**Purpose:** Build a Retrieval-Augmented Generation (RAG) pipeline for a Health Assistant that retrieves medical information from a vector database and generates safe, source-cited responses. Hallucination prevention is critical.

---

## [CONTEXT]

- **Tech stack:** Python 3.11+, LangChain, ChromaDB (or FAISS), OpenAI Embeddings, GPT-4o
- **Current state:** Project has existing modules:
  - `src/embeddings/` — document embedding pipeline
  - `src/vectorstore/` — ChromaDB wrapper with `search()` method
  - `src/llm/` — LLM client configuration
- **Data source:** Medical knowledge base (PDFs, clinical guidelines) pre-chunked and embedded
- **Document schema:**
  ```python
  Document {
      page_content: str        # Text chunk
      metadata: {
          source_id: str       # Document identifier
          source_name: str     # Document title
          page_number: int     # Page reference
          last_updated: str    # ISO date
          category: str        # e.g. "cardiology", "nutrition"
      }
  }
  ```
- **Existing code:**
  - `VectorStore.search(query, k=5, threshold=0.7)` returns `List[Document]`
  - `LLMClient.generate(prompt, temperature=0.1)` returns `str`
  - Logging configured via `structlog`

---

## [TASK]

**Objective:** Create a RAG chain that:
1. Takes a user health query
2. Retrieves relevant documents from the vector store
3. Generates a response grounded in retrieved sources
4. Cites sources in every response
5. Handles empty retrieval results safely

**Acceptance Criteria:**
- [ ] Query embedding and vector search implemented
- [ ] Similarity threshold filtering (configurable, default 0.7)
- [ ] Response includes source citations (document name, page number)
- [ ] Confidence score calculated (based on retrieval similarity)
- [ ] **Fallback strategy:** When vector DB returns 0 results:
  - Return safe message: "I couldn't find relevant medical information for your query. Please consult a healthcare professional for personalized advice."
  - Log the failed query with `level=WARNING` for analysis
  - Never fabricate or guess medical information
- [ ] Low-confidence responses (score < 0.5) include disclaimer
- [ ] Temperature set to 0.1 (minimize creativity for medical domain)
- [ ] All retrieval and generation steps wrapped in try-catch
- [ ] Unit tests for: normal retrieval, empty results, low confidence, error handling

---

## [CONSTRAINTS]

### Karpathy Principles Enforcement

**Principle 1 — Think Before Coding:**
- State assumptions about the embedding model and chunk size before implementing
- If the query is ambiguous, the chain should ask for clarification rather than guess

**Principle 2 — Simplicity First:**
- Single-chain architecture only. No multi-agent routing or complex orchestration
- No LangChain Agent or Tool abstractions — use a simple `RunnableSequence` or function chain
- If it can be a function, don't make it a class

**Principle 3 — Surgical Changes:**
- Use existing `VectorStore.search()` and `LLMClient.generate()` — do not rewrite them
- Add the RAG chain as a new module `src/rag/chain.py` without modifying existing code

**Principle 4 — Goal-Driven Execution:**
- Define success as: "Given a health query, return a source-cited answer or a safe fallback"
- Write tests that verify both the happy path and the fallback path

### FORBIDDEN
- ❌ Do not hardcode medical advice or treatment recommendations
- ❌ Do not return raw LLM output without source grounding
- ❌ Do not ignore empty retrieval results (must trigger fallback)
- ❌ Do not use `temperature > 0.3` for medical content generation
- ❌ Do not log patient queries containing PII without redaction
- ❌ Do not use LangChain Agents, Tools, or Router chains (keep it simple)

### REQUIRED
- ✓ Every response must cite at least one source document (or trigger fallback)
- ✓ Confidence score exposed in response metadata
- ✓ Medical disclaimer on low-confidence responses
- ✓ Try-catch for all vector DB and LLM operations
- ✓ Structured logging with `query_id`, `num_results`, `confidence_score`
- ✓ Fallback response is a constant (not generated by LLM)
- ✓ Unit tests with >= 80% coverage for the RAG chain module

### Process
- ✓ Run Self-Check before output
- ✓ Include Self-Check report + test examples

---

## [OUTPUT FORMAT]

- **Format:** Python files:
  - `src/rag/chain.py` — RAG chain implementation
  - `src/rag/prompts.py` — Prompt templates (system + user)
  - `src/rag/models.py` — Response data models (Pydantic)
  - `tests/test_rag_chain.py` — Unit tests
- **Style:** PEP 8, type hints mandatory, docstrings on public functions
- **Length:** Code only (no lengthy explanations)
- **Include:** Self-Check report

### Expected Response Structure