Implementing Hybrid RAG: Combining Vector and Keyword Search

The Limitations of Pure Vector Search

Vector search relies on embeddings—mathematical representations of text in high-dimensional space. While this allows a system to understand that 'warm' is related to 'hot,' it treats words as proximity points rather than literal strings. If your knowledge base contains a specific product code like 'XJ-9000,' a vector search might return results for similar-sounding vacuum cleaners but miss the exact document for that specific model.

This is where traditional search algorithms like BM25 (Best Matching 25) excel. BM25 is a ranking function used by search engines to estimate the relevance of documents to a given search query based on the appearance of specific terms. By combining these two, we create a 'hybrid' system that covers both semantic breadth and keyword specificity.

Architecting the Hybrid Retrieval Pipeline

To implement Hybrid RAG, you need two parallel processes for every query. First, the query is converted into an embedding and sent to a vector database. Second, the raw text query is sent to a full-text search index. The results from both must then be normalized and merged.

• Vector Index: Handles semantic similarity and natural language nuances.
• Keyword Index: Handles exact matches, technical jargon, and rare tokens.
• Fusion Algorithm: Re-ranks documents from both sources into a single prioritized list.
• LLM Generation: Uses the merged context to generate the final response.

Reciprocal Rank Fusion (RRF) Explained

The challenge with hybrid search is that vector scores (Cosine Similarity or Euclidean Distance) and BM25 scores are on completely different scales. You cannot simply add them together. Reciprocal Rank Fusion (RRF) solves this by looking at the rank position of a document rather than its raw score.

RRF works on a simple principle: a document that appears at the top of both search lists is significantly more relevant than a document that appears at the top of only one.

Step-by-Step Implementation in Python

Using a framework like LangChain or LlamaIndex simplifies this, but understanding the underlying logic is crucial. Below is a conceptual example of how to merge results from two different search providers using a weighted approach.

def hybrid_rerank(vector_results, keyword_results, alpha=0.5):
    # alpha 1.0 = pure vector search, 0.0 = pure keyword search
    scores = {}
    
    for rank, doc in enumerate(vector_results):
        scores[doc.id] = scores.get(doc.id, 0) + alpha * (1 / (rank + 60))
        
    for rank, doc in enumerate(keyword_results):
        scores[doc.id] = scores.get(doc.id, 0) + (1 - alpha) * (1 / (rank + 60))
        
    sorted_docs = sorted(scores.items(), key=lambda x: x[1], reverse=True)
    return sorted_docs

Tuning Your Hybrid Parameters

The 'alpha' parameter in your fusion logic is the most important knob to turn. If you are building a tool for searching medical journals or legal documents, you might want to favor keyword search (alpha < 0.5) to ensure specific citations are captured. For a conversational chatbot designed to answer vague questions, a higher alpha (favoring vectors) is usually better.

It is also essential to consider the 'k' value in RRF, which handles the penalty for documents further down the list. A standard starting value is 60, but this can be adjusted based on your average document length and the precision of your embedding model.

Evaluating Performance: Is Hybrid Actually Better?

Building the pipeline is only half the battle. You must evaluate whether the hybrid approach actually improves your retrieval. You can use metrics like Mean Reciprocal Rank (MRR) or Hit Rate to compare your hybrid system against the baseline vector-only system.

1. Create a 'golden dataset' of question-answer pairs with known document sources.
2. Run your retrieval engine using only vector search and record the hit rate.
3. Enable hybrid search and measure the improvement on keyword-heavy queries.
4. Iterate on the alpha weighting until you find the sweet spot for your specific domain.

Best Practices for Production RAG

When moving to production, consider the latency impact of running two separate searches. Modern vector databases like Pinecone, Weaviate, or Qdrant often offer built-in hybrid search features that handle the fusion and normalization at the database level, which is much faster than doing it in your application code.

Additionally, always sanitize your keyword search inputs. While embeddings are robust against typos, BM25 is not. Applying basic lemmatization or spell-checking to the keyword branch can prevent the retrieval from failing due to a minor orthographic error.