Installation

Complete installation guide for the Obstetrics RAG Benchmark

This guide provides detailed installation instructions for setting up the Obstetrics RAG Benchmark on your local machine.

Prerequisites

Before installing, ensure your system meets these requirements:

System Requirements

• Operating System: Linux, macOS, or Windows with WSL2
• Python: Version 3.8 or higher
• Git: For repository cloning
• Internet Connection: Required for API calls and package installation

API Requirements

You’ll need an OpenAI API key to run evaluations. The benchmark uses:

• text-embedding-3-small for creating document embeddings
• GPT-4o (default) or other OpenAI models for answer generation

Get your API key from OpenAI Platform.

Installation steps

Troubleshooting

Common issues and solutions

ModuleNotFoundError: No module named 'langchain'

Problem: Dependencies not installed or wrong Python environment.

Solution:

# Ensure virtual environment is activated
source venv/bin/activate  # Linux/macOS
venv\Scripts\activate     # Windows

# Reinstall dependencies
pip install -r requirements.txt

OpenAI API authentication error

Problem: Invalid or missing OpenAI API key.

Solution:

1. Verify your API key is correct in .env:
   cat .env | grep OPENAI_API_KEY
   
2. Ensure no extra spaces or quotes around the key
3. Verify your OpenAI account has credits: platform.openai.com/account/billing

ChromaDB connection error

Problem: Vector database not initialized or corrupted.

Solution:

1. Delete existing database:
   rm -rf data/embeddings/chroma_db
   
2. Recreate embeddings:
   python scripts/create_embeddings.py
   

FileNotFoundError: chunks_final.json

Problem: Missing source data files.

Solution: Ensure you cloned the complete repository:

ls data/chunks/chunks_final.json

If missing, re-clone the repository or check that Git LFS files downloaded correctly.

Python version compatibility issues

Problem: Using Python version below 3.8.

Solution:

1. Install Python 3.8+ from python.org
2. Create virtual environment with correct version:
   python3.8 -m venv venv
   source venv/bin/activate
   pip install -r requirements.txt
   

Rate limit errors from OpenAI API

Problem: Exceeding OpenAI API rate limits.

Solution:

• Tier 1 accounts have lower rate limits
• Wait a few minutes between evaluation runs
• Consider upgrading your OpenAI account tier
• For comprehensive evaluations, use --debug flag to see detailed progress

Upgrading

To upgrade to the latest version:

# Pull latest changes
git pull origin main

# Update dependencies
pip install -r requirements.txt --upgrade

# Recreate embeddings if data changed
python scripts/create_embeddings.py

Uninstallation

To completely remove the project:

# Deactivate virtual environment
deactivate

# Remove project directory
cd ..
rm -rf RAG-Benchmark

Next steps

Quickstart Guide

Run your first evaluation in 5 minutes

RAG Architectures

Learn about the different RAG strategies

Evaluation Guide

Understanding RAGAS metrics and results

API Reference

Explore the Python API for custom workflows

Getting help

If you encounter issues not covered in this guide:

• Check the GitHub Issues
• Review the README
• Open a new issue with:
  • Your Python version (python --version)
  • Full error message
  • Steps to reproduce
  • Your operating system

System requirements summary

Component	Requirement	Notes
Python	3.8+	Tested on 3.8, 3.9, 3.10, 3.11
RAM	4GB minimum	8GB+ recommended for large evaluations
Storage	500MB	For embeddings and results
OpenAI API	Active account	With available credits
Internet	Required	For API calls

Estimated costs: Running a single RAG evaluation with 10 questions typically costs $0.05-0.15 USD depending on the model used.