The vocabulary gap between how users ask questions and how documents are written is a leading cause of RAG failure. Query transformation techniques address this before the vector search even runs — no index changes required.
A user asks: “why is my model not converging?”Your corpus contains a section titled: “Common Causes of Training Instability in Neural Networks.”
The semantic gap between “not converging” and “training instability” is real — embedding models partially bridge it, but not perfectly. Query transformation closes the remaining gap.
Problem
User queries are short and conversational. Documents are long and formal. The vocabulary gap causes retrieval misses.
Solution
Ask the LLM to generate a hypothetical document that would answer the query. Embed and search with that document instead of the raw query.
Original query
Transformed
import anthropic
client = anthropic.Anthropic()
def generate_hypothetical_doc(query: str) -> str:
"""Generate a hypothetical document that would answer the query."""
response = client.messages.create(
model="claude-haiku-4-5-20251001",
max_tokens=300,
messages=[{
"role": "user",
"content": f"Write a 2-paragraph passage from a technical document that directly answers: '{query}'. Be concise and factual.",
}],
)
return response.content[0].text
def hyde_search(query: str, collection, n_results: int = 5) -> list[dict]:
# Generate hypothetical answer document
hypo_doc = generate_hypothetical_doc(query)
# Embed the hypothetical document, not the raw query
hypo_embedding = embed(hypo_doc)
results = collection.query(
query_embeddings=[hypo_embedding],
n_results=n_results,
)
return results| Technique | Best For | Added Latency |
|---|---|---|
| HyDE | Short user queries vs. long formal documents | ~300ms (LLM call) |
| Step-Back | Narrow technical questions needing broader context | ~200ms |
| Multi-Query | General purpose — best default for production | ~200ms + extra retrievals |
Next: multi-query retrieval in depth and step-back prompting — then contextual compression, which filters retrieved chunks down to only the sentences that actually answer the question.