 (RAG) applications that handle large volumes of multimodal, unstructured data require both the precision of text search and the semantic similarity of vector search. Using either keyword search or vector search alone would not deliver optimally relevant results, resulting in unsatisfactory user experiences. Hybrid search works to provide a solution to this challenge.

 to deliver the most relevant answers to end users.

While text (or keyword) search is effective in finding exact matches for query terms, 

 provides the added benefit of identifying results with similar meanings, even if they don't contain the exact query term. This ensures that contextually similar matches are also included in the combined results of both methods of search.

For example, a hybrid query like "healthy recipes with feta cheese" can benefit from both: keyword search ensures that the results contain feta cheese while semantic search looks beyond the actual "healthy" keyword and finds recipes that generally fit in the same healthy category, such as low-carb, low-sugar, or vegetable-focused.

Conversely, if there are specific terms or phrases within the dataset that should be explicitly omitted from the search results, hybrid search offers a valuable solution. By combining text search with semantic search, hybrid search ensures that these excluded keywords are effectively filtered out of the final results, delivering the most optimal experience to end users.

Hybrid search applications can benefit organizations in multiple ways.

Hybrid search enhances user experience by understanding query intent, delivering context-focused search results, and reducing search effort, allowing users to quickly find the information they need.

Analyzing search queries provides insights into user needs and preferences, helping organizations create higher value by identifying knowledge gaps and informing content creation and optimization strategies.

As hybrid search helps improve insight discovery, it also supports better decision-making and productivity by surfacing the right information at the right time.

Hybrid search ranks and delivers the "best" results in terms of relevance from a keyword search result set and a vector search result set. Some popular and effective hybrid search techniques are discussed below.

A popular approach that many hybrid search systems rely on is combining 

 (Best Match 25) and vector similarity (measured using distance metrics like dot product, Euclidean, cosine, Hamming, etc.) scores to deliver more accurate and relevant search results.

BM25 is the default scoring algorithm in 

. It focuses on keyword relevance and ranks documents based on the frequency in which the queried keywords appear, considering factors like document length and overall term frequency.

On the other hand, with vector search, documents are ranked based on their semantic relevance to the query rather than keyword matching. The user query is first converted into 

 — numerical representations of the query text, which is then compared to the vector embeddings generated from all data within the database in order to find the closest matches. In 

, semantic similarity can be determined using Euclidean, cosine, or dot product metrics.

Finally, the BM25 and vector search scores are combined to create a unified ranking, delivering the highest-ranked results to end users.

There are multiple ways to combine the BM25 and vector search scores, among them are reciprocal rank fusion (RRF) and relative score fusion (RSF) — both of which are recommended techniques for hybrid search in MongoDB Atlas.

RRF works by calculating the reciprocal rank of each document across different search methods and then combining these ranks into a unified score for each document. It's particularly useful when there are various contextual meanings and data fields that need to be taken into consideration, because it allows for a more balanced and well-rounded overall ranking.

RSF normalizes the scores from each search method, scaling them to a common range (typically 0 to 1), before combining them. This normalization ensures that the relative importance of each search method is preserved, even if their original score distributions differ. The normalized values can provide a more nuanced and accurate ranking of results compared to methods like RRF, which primarily focus on the order of results.

Combining dense and sparse vector search results

Another approach often leveraged by vector-only hybrid search solutions is using sparse and dense vectors to combine keyword and semantic relevance to improve search results.

Sparse vectors are primarily used for keyword-based search. They represent documents as a set of words or tokens, where each dimension corresponds to a specific word in the vocabulary. Sparse vectors excel at finding exact matches for query terms, making them suitable for tasks like filtering and precise information retrieval.

Dense vectors, generated by machine learning models trained on large amounts of data, capture the semantic meaning and context of documents. Dense vector search can handle complex queries, including those with natural language, ambiguous, or implicit terms.

Hybrid search then combines the scores from both sparse and dense vectors to deliver the final search results to the user, prioritizing the highest ones.

Enhance search and generative AI applications with hybrid search

As explained above, hybrid search shines in scenarios where there's a need for both precision (where keyword search excels) and semantic search capabilities (where vector search excels), and where user queries can vary from simple to complex, including both keyword and natural language queries.

Organizations can implement hybrid search to deliver better results to end users via practical applications. Below are some common use cases where hybrid search can help improve search quality, and ultimately the user experience in a range of intelligent search and generative AI use cases.

Understanding users’ interests and intent is a strong competitive advantage–like how Netflix provides a personalized selection of shows and movies based on your watch history, or how Amazon recommends products based on your purchase history. This is particularly powerful in e-commerce, media & entertainment, financial services, and product/service-oriented industries where the customer experience tightly influences the bottom line.

Retrieval augmented generation applications

 (LLMs), search is increasingly becoming essential in 

 application development. It augments LLMs by providing domain-specific context outside of what the LLMs “know,” ensuring relevance and accuracy of the gen AI output. In this case, hybrid search combining semantic search and text search can help the RAG model generate responses that are more contextually relevant, helping reduce hallucinations and improve the accuracy of the response.

By combining keyword search and semantic search, hybrid search can better understand user intent and deliver tailored results. For instance, a customer searching for "women summer dress from XYZ brand" can receive results for various style and color combinations of dresses from that specific brand, even if they didn't explicitly mention those preferences.

As organizations' data grows, so is the need for more sophisticated, enhanced search, especially when there's a mix of structured (dates, locations, product IDs) and unstructured data (images, PDFs, audios, videos). Hybrid search makes it easier for users to locate relevant resources and obtain insights beyond a basic list of search results, particularly for queries that require advanced capabilities like semantic retrieval or natural language answers.

Considerations when implementing hybrid search

There are several key factors to consider when planning and deploying a hybrid search system.

First, data quality is essential. High-quality data is the cornerstone of effective hybrid search. This involves ensuring data accuracy and consistency. Data cleaning and preprocessing are essential steps to remove errors, inconsistencies, and noise. Additionally, data enrichment can enhance search results by adding context and metadata to unstructured data like text, images, videos, and audios.

Second, architecture and techniques are also key. Since hybrid search relies on both text data and vector embeddings, it's important to select a 

 that is versatile, secure, and scalable with the business needs. Choosing the right vector database means thoroughly evaluating key factors including the database's ability to provide performant indexing and querying capabilities, especially for real-time applications. The database should also seamlessly integrate with the organization's existing technology stack and be capable of handling large, growing datasets in various formats, all while maintaining a high level of security and database management support.

Further, it's worth understanding the techniques that will be applied to the hybrid search system, so that their strengths and weaknesses can be fully considered. A ranking technique that works well for certain datasets and use cases may need adjustments in order to work well for others.

And finally, as with any technology implementation, it's recommended that purposes and expected outcomes are defined prior to deployment, so that they can be measured and improved. For instance, how search quality is measured, how user feedback will be incorporated to improve results, whether the system delivers the metrics that it should, and more.

A well-implemented hybrid search system can deliver significant benefits, including improved user experiences thanks to more relevant results and better insights. And as the full potential of data is unlocked, this can also help enhance the overall organizational efficiency and decision making.

Get started with hybrid search in MongoDB Atlas

As organizations continue to innovate in the rapidly evolving technology ecosystem, building robust gen AI and search applications supporting customer, employee, and stakeholder experiences can deliver powerful competitive advantages.

With MongoDB, you can efficiently perform hybrid search, full-text search, and vector search, all natively within the database. Hybrid search combines the power of Atlas Search and Atlas Vector Search to deliver high-quality search results to support a variety of search and gen AI applications. There’s no need to sync data for indexing and retrieval between your operational and vector databases—saving time, reducing complexity, and preventing errors. Your operational and vector data stay in one place.

Start building today—simplify your developer experience and create more value using the fully-managed, secure MongoDB developer data platform, integrated with a vast AI partner ecosystem, including all major cloud providers, LLM model providers, and system integrators.

Explore MongoDB Atlas - the vector database with built-in search, vector, and more capabilities, 

Build performant applications with hybrid search in MongoDB Atlas, see our 

Get strategic advice and implementation support for search and 

Hybrid Search Explained - Rich text

Get Started With MongoDB Atlas

Software Development Life Cycle - Get Started - Reuse

Learn MongoDB with expert tutorials and tips on our new Developer YouTube channel. <mark>Subscribe ></mark>