What is hybrid search?
Hybrid search combines keyword search and vector search into a single retrieval system, running both approaches against the same query and merging their results into one final, unified ranking. This article focuses specifically on the mechanics of that combination, the different fusion techniques used to merge two fundamentally different kinds of relevance scores fairly, why naive combination approaches tend to produce poorly balanced results, and the practical engineering decisions involved in building a hybrid system that actually delivers on the promise of combining both approaches’ strengths rather than diluting them.
Why combining two different scoring systems is harder than it first appears
Keyword search, typically using the BM25 scoring covered in this collection’s discussion of sparse retrieval, produces relevance scores on one numerical scale, while vector search produces similarity scores, covered in this collection’s discussion of embedding similarity, on an entirely different scale, and these two kinds of scores aren’t directly comparable to each other in any meaningful way. A BM25 score of fifteen and a cosine similarity score of zero point eight don’t represent the same underlying notion of relevance, and simply averaging or directly comparing these two numbers produces a combined ranking that’s arbitrarily skewed toward whichever scoring system happens to produce numbers on a larger scale, rather than reflecting genuine relevance from either approach.
This is the core technical challenge hybrid search has to solve, and it’s why hybrid search implementations rely on more sophisticated fusion techniques specifically designed to combine rankings, rather than raw scores, from two fundamentally incomparable scoring systems into one fair, well-balanced final result.
Reciprocal rank fusion and why it sidesteps the scale-comparison problem entirely
The most widely used technique for combining hybrid search results, reciprocal rank fusion, sidesteps the incomparable-scores problem by ignoring the actual scores altogether and working purely with each result’s rank position, its position within the ordered list of results, from each individual search method. A document’s combined score comes from a formula based on its rank in the keyword search results and its rank in the vector search results, giving a meaningful boost to documents that rank highly in either or both individual result lists, without ever needing to compare a BM25 score directly against a cosine similarity score.
This approach’s real strength is its simplicity and robustness, it doesn’t require any application-specific tuning of how to weigh two different scoring scales against each other, and it tends to produce reasonable, well-balanced combined rankings across a wide range of applications without needing extensive customization. This is exactly why reciprocal rank fusion has become something close to a default choice for combining hybrid search results in practice, even though more sophisticated, application-tuned combination approaches can sometimes outperform it for a specific application’s particular needs.
Weighted combination approaches and when they’re worth the added tuning effort
An alternative to rank-based fusion applies explicit weights to each search method’s contribution, deciding upfront how much influence keyword search should have relative to vector search in the final combined ranking, then normalizing each method’s scores onto a comparable scale before combining them according to those weights. This approach offers more direct, deliberate control over the balance between the two methods than pure rank-based fusion provides, letting a team lean more heavily toward keyword search’s precision for an application where exact terminology matters considerably, or toward vector search’s semantic flexibility for an application where conceptual matching matters more.
This added control comes with real added complexity, deciding on the right weighting requires the same empirical tuning covered throughout this collection’s broader discussion of threshold and parameter calibration, testing different weight combinations against a representative sample of an application’s actual queries and measuring which balance actually produces the best combined ranking, rather than guessing at reasonable-sounding weights without validating them against real data.
Why the right fusion approach depends on an application’s actual query mix
An application handling mostly precise, exact-terminology queries benefits from a fusion approach that leans more heavily toward keyword search’s contribution, while an application handling mostly conceptual, exploratory queries benefits from leaning more toward vector search, and an application with a genuinely mixed distribution of both kinds of queries needs a fusion approach that performs reasonably well across that entire mix rather than being over-tuned toward either extreme. Understanding an application’s actual query distribution, connecting to the broader discussion of AI native testing covered elsewhere in this collection, is essential for choosing and tuning a fusion approach that actually fits real usage rather than an assumed, idealized query pattern.
This is why hybrid search tuning benefits from ongoing measurement rather than a one-time setup decision, an application’s actual query mix can shift over time as its user base or use cases evolve, and a fusion configuration tuned well for an earlier query distribution may no longer be optimally balanced once that distribution has shifted meaningfully.
How re-ranking fits alongside hybrid search’s fusion step
Hybrid search’s fusion step produces one combined candidate list, but this combined list often still benefits from the re-ranking step covered in this collection’s dedicated discussion of that topic, applying a more precise, computationally expensive relevance judgment to the fused candidates before finalizing what actually gets used downstream. This is a genuinely distinct step from fusion itself, fusion combines two different retrieval methods’ candidate lists into one, while re-ranking applies a separate, more careful relevance judgment to whatever candidates made it into that combined list, regardless of which original method surfaced them.
Layering re-ranking on top of hybrid search’s fusion step is a common and often valuable pattern in production retrieval systems, capturing the combined recall benefit of running two complementary search methods together, while still applying a final, more precise relevance pass before committing to a final ranking, rather than trusting either the fusion formula or either individual method’s raw ranking as the final word on relevance.
Why running both search methods in parallel matters for hybrid search’s practical latency
Because hybrid search runs two separate retrieval operations for every single query, its practical latency depends heavily on whether those two operations run sequentially or in parallel, running keyword search and vector search one after another roughly doubles the latency compared to running them simultaneously and combining their results once both complete. This connects directly to the broader infrastructure discussion covered throughout this collection, a well-built hybrid search system architects its two underlying searches to run concurrently, keeping overall latency close to whichever individual method is slower rather than the sum of both methods’ individual latencies.
This is a straightforward but consequential architectural detail that’s easy to overlook when building a hybrid search system for the first time, a naive, sequential implementation can end up with meaningfully worse latency than either search method alone would have, entirely avoidable simply by running the two searches concurrently rather than one after the other.
Common mistakes teams make around hybrid search
1. Directly comparing or averaging raw keyword and vector search scores without accounting for their fundamentally incomparable scales, producing an arbitrarily skewed combined ranking.
2. Choosing a weighted fusion approach without empirically tuning those weights against a representative sample of the application’s own actual queries.
3. Treating a hybrid search fusion configuration as a permanent, one-time setup decision rather than revisiting it as an application’s actual query mix evolves over time.
4. Skipping re-ranking after fusion, missing the additional precision improvement available from applying a more careful relevance judgment to the already-combined candidate set.
5. Running keyword and vector search sequentially rather than in parallel, unnecessarily doubling hybrid search’s practical latency compared to a properly concurrent implementation.
What connects these mistakes is underestimating the real engineering care required to combine two fundamentally different retrieval methods well, hybrid search’s value comes specifically from thoughtful, well-tuned combination, not from simply running both methods and hoping their results merge sensibly on their own without deliberate fusion logic behind them.
The deeper point about hybrid search is that keyword and vector search solve genuinely different aspects of the same underlying relevance problem, and the real engineering challenge in hybrid search isn’t running both methods, that part is comparatively straightforward, it’s combining their outputs fairly and effectively, a problem that requires its own dedicated technique and tuning rather than assuming two good retrieval methods automatically produce a good combined result simply by being run together.