In this capstone you build a Document Intelligence System: load a PDF, chunk and embed its contents, store everything in ChromaDB, and expose a natural language Q&A interface over the document. Every component comes from this course — no new concepts, only integration.
A command-line document Q&A tool that takes any PDF and a natural language question, retrieves the most relevant passages using semantic search, and streams an LLM-generated answer grounded in the document. This is a minimal but complete RAG system.
doc_intelligence/ ├── ingest.py # Load PDF → chunk → embed → store ├── search.py # Query the vector store ├── qa.py # Combine retrieval + LLM generation ├── main.py # CLI entry point └── chroma_db/ # Persisted vector store (auto-created)
Install dependencies
pip install openai chromadb pypdf langchain
Load the PDF
Use pypdf.PdfReader to extract text from each page
Chunk the text
Use RecursiveCharacterTextSplitter(chunk_size=512, chunk_overlap=64)
Embed all chunks in batches
Call openai.embeddings.create with input=batch, model="text-embedding-3-small"
Store in ChromaDB
collection.upsert(ids, embeddings, documents, metadatas={page_number, source})
Print ingestion summary
Total pages, total chunks, collection.count()
# search.py — implement this function
def retrieve(query: str, collection, n_results: int = 4) -> list[dict]:
"""
Embed the query and return the top-n most relevant chunks.
Returns a list of dicts with keys:
- text: str (the chunk content)
- page: int (page number from PDF)
- distance: float (cosine distance, lower = more similar)
"""
# TODO: embed query, call collection.query, format results
raise NotImplementedErrorEmbed the query string
Same model as ingestion — text-embedding-3-small
Query the collection
n_results=4, include=["documents", "distances", "metadatas"]
Return formatted results
List of {text, page, distance} dicts
Test with 3 different queries
Print the top result for each and verify it makes sense
import anthropic
from search import retrieve
client = anthropic.Anthropic()
def answer(question: str, collection) -> str:
# 1. Retrieve relevant chunks
chunks = retrieve(question, collection, n_results=4)
# 2. Build context string
context = "
---
".join(
f"[Page {c['page']}] {c['text']}" for c in chunks
)
# 3. Generate answer with grounded prompt
response = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=1024,
messages=[{
"role": "user",
"content": f"""Answer the question using ONLY the context provided below.
If the answer is not in the context, say "I cannot find this in the document."
CONTEXT:
{context}
QUESTION: {question}""",
}],
)
return response.content[0].text
if __name__ == "__main__":
import chromadb
chroma = chromadb.PersistentClient(path="./chroma_db")
collection = chroma.get_collection("documents")
question = input("Ask a question about the document: ")
print("
" + answer(question, collection))Write 5 test questions with known correct answers
Based on the PDF content, create a small golden set
Score retrieval separately from generation
Is the correct passage in the top-4 retrieved chunks? (Recall@4)
Measure answer quality
Does the generated answer correctly address the question? Score 1–5 manually
Find one failure case
What question does the system fail on? Why? (wrong chunk? hallucination?)
Try one improvement
Change chunk size, n_results, or system prompt — does the failure case improve?
Tip: start with a shorter document (5–20 pages) so you can manually verify whether retrievals are correct.
Congratulations — you have completed the Embeddings & Vector Databases course. You are now ready for Advanced RAG Patterns, where you will learn chunking strategies, hybrid search, re-ranking, and how to evaluate your retrieval quality systematically with RAGAS.