HyDE RAG

RAG pipeline using Hypothetical Document Embeddings for improved retrieval

Overview

The HyDE (Hypothetical Document Embeddings) RAG module implements a two-stage RAG pipeline. It first generates a hypothetical document that would perfectly answer the user’s query, then uses that document for semantic search. This approach can improve retrieval accuracy by searching for detailed content rather than a short query.

Module: src.rag.hyde

Source: src/rag/hyde.py

Configuration

Default Models

# Use a more creative model for HyDE document generation
llm_hyde = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0.7)

# Use a powerful model for final answer generation
llm_answer = ChatOpenAI(model_name="gpt-4o", temperature=0)

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

Vector Store

vectorstore = Chroma(
    persist_directory=str(chroma_db_dir),
    embedding_function=embeddings,
    collection_name="guia_embarazo_parto"
)

retriever = vectorstore.as_retriever(search_kwargs={"k": 5})

Prompt Templates

HyDE Document Generation Prompt

hyde_prompt_template = """
You are a medical expert writing a detailed section for a medical guide on pregnancy and childbirth.

Based on this question: {question}

Write a detailed and comprehensive medical document that would perfectly answer this question.
The document should include:
- Accurate medical information on the topic
- Relevant clinical details
- Appropriate medical recommendations
- Important considerations for maternal health
- Practical information and advice

Write the document as if it were part of an official medical guide on pregnancy and childbirth.
Be specific, detailed, and use appropriate medical terminology.

HYPOTHETICAL DOCUMENT:
"""

Answer Generation Prompt

Uses the standard medical expert prompt (same as Simple RAG).

Functions

generate_hypothetical_document

def generate_hypothetical_document(query: str) -> Dict[str, Any]

Generates a hypothetical document based on the user’s query.

query · str · required

The user’s question

return · Dict[str, Any]

Dictionary containing:

• document (str): The generated hypothetical document
• input_tokens (int): Input tokens used
• output_tokens (int): Output tokens generated
• total_tokens (int): Total tokens
• usage_source (str): Source of usage data
• cost (float): Cost in USD
• cost_source (str): Source of cost calculation

format_docs

def format_docs(docs: List[Any]) -> str

Formats the retrieved documents to be included in the final prompt.

docs · List[Any] · required

A list of retrieved LangChain Document objects

return · str

A formatted string containing the content of the documents

process_hyde_query

def process_hyde_query(
    query: str, 
    custom_hyde_llm: ChatOpenAI = None, 
    custom_answer_llm: ChatOpenAI = None
) -> Dict[str, Any]

Processes a query using the full HyDE RAG pipeline.

query · str · required

The user’s question

custom_hyde_llm · ChatOpenAI · default: "None"

Custom model for hypothetical document generation

custom_answer_llm · ChatOpenAI · default: "None"

Custom model for answer generation

return · Dict[str, Any]

Dictionary containing:

• answer (str): The final generated answer
• contexts (List[str]): Retrieved document contents
• hypothetical_document (str): The generated hypothetical document
• hyde_metrics (dict): Metrics for HyDE generation
  • input_tokens (int)
  • output_tokens (int)
  • cost (float)
  • usage_source (str)
  • cost_source (str)
• answer_metrics (dict): Metrics for answer generation
  • input_tokens (int)
  • output_tokens (int)
  • cost (float)
  • usage_source (str)
  • cost_source (str)
• total_cost (float): Combined cost
• total_input_tokens (int): Combined input tokens
• total_output_tokens (int): Combined output tokens
• usage_sources (List[str]): Sources of usage data
• cost_sources (List[str]): Sources of cost calculations

query_for_evaluation

def query_for_evaluation(
    question: str, 
    hyde_model: str = None, 
    answer_model: str = None,
    custom_hyde_llm: Optional[BaseChatModel] = None,
    custom_answer_llm: Optional[BaseChatModel] = None
) -> dict

A wrapper function for RAG evaluation frameworks like Ragas.

question · str · required

The question to process

hyde_model · str · default: "None"

The name of the LLM model to use for HyDE generation. Defaults to “gpt-3.5-turbo”

answer_model · str · default: "None"

The name of the LLM model to use for answer generation. Defaults to “gpt-4o”

custom_hyde_llm · BaseChatModel · default: "None"

Pre-configured LLM for HyDE. Takes precedence over hyde_model

custom_answer_llm · BaseChatModel · default: "None"

Pre-configured LLM for answer. Takes precedence over answer_model

return · dict

Dictionary containing:

• question (str): Original question
• answer (str): Generated answer
• contexts (List[str]): Retrieved document contents
• metadata (dict): Comprehensive metadata including:
  • execution_time (float): Total execution time in seconds
  • input_tokens (int): Total input tokens (HyDE + Answer)
  • output_tokens (int): Total output tokens (HyDE + Answer)
  • total_cost (float): Total cost in USD
  • retrieval_method (str): “hyde”
  • llm_hyde_model (str): Model used for HyDE generation
  • llm_answer_model (str): Model used for answer generation
  • hyde_provider (str): Provider for HyDE model
  • answer_provider (str): Provider for answer model
  • hyde_model_id (str): Full HyDE model ID
  • answer_model_id (str): Full answer model ID
  • hyde_cost (float): Cost for HyDE generation
  • answer_cost (float): Cost for answer generation
  • usage_source (str): Combined usage sources
  • cost_source (str): Cost calculation source

Usage Example

from src.rag.hyde import query_for_evaluation

# Basic usage with default models
result = query_for_evaluation(
    question="¿Cuáles son las etapas del parto?"
)

print(result["answer"])
print(f"HyDE model: {result['metadata']['llm_hyde_model']}")
print(f"Answer model: {result['metadata']['llm_answer_model']}")
print(f"Total cost: ${result['metadata']['total_cost']:.6f}")
print(f"HyDE cost: ${result['metadata']['hyde_cost']:.6f}")
print(f"Answer cost: ${result['metadata']['answer_cost']:.6f}")

# Using custom models
result = query_for_evaluation(
    question="¿Qué es la episiotomía?",
    hyde_model="gpt-4o-mini",
    answer_model="gpt-4o"
)

# Using pre-configured LLMs
from langchain_openai import ChatOpenAI

hyde_llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0.8)
answer_llm = ChatOpenAI(model_name="gpt-4o", temperature=0)

result = query_for_evaluation(
    question="¿Cuándo es necesaria una cesárea?",
    custom_hyde_llm=hyde_llm,
    custom_answer_llm=answer_llm
)

Pipeline Flow

1. Generate HyDE: Uses gpt-3.5-turbo (temperature=0.7) to generate a detailed hypothetical document that would answer the question
2. Retrieve: Uses the hypothetical document (not the original query) to perform semantic search and retrieve the top 5 most relevant actual documents
3. Format: Formats retrieved documents with metadata
4. Generate Answer: Uses gpt-4o (temperature=0) to generate the final answer based on retrieved context
5. Track: Captures separate metrics for both HyDE and answer generation

Key Features

• Two-stage retrieval: Generates hypothetical content first, then searches
• Improved semantic matching: Searches with detailed content vs. short query
• Dual model tracking: Separate metrics for HyDE and answer generation
• Creative HyDE generation: Uses higher temperature (0.7) for document generation
• Precise answer generation: Uses temperature 0 for final answer
• Comprehensive cost tracking: Tracks costs for both stages

When to Use HyDE

HyDE works best when:

• User queries are short or ambiguous
• You need to bridge vocabulary gaps between query and documents
• Documents use different terminology than typical user queries
• You want to improve recall for conceptual questions