Go from 0 to 1 to Enterprise-Ready with MongoDB Atlas and LLMs

David Macias
June 22, 2023 | Updated: September 5, 2024
#ArtificialIntelligence

This post is also available in: Deutsch, Français, Español, Português, Italiano, 한국어, 简体中文.

Update 9/5/2024: Since this post was originally published in June, 2023, Atlas Vector Search has since gone generally available! The fastest and easiest way to go from 0 to 1 with MongoDB Atlas and LLMs is to try the Atlas Vector Search Quick Start. See how many companies, such as Okta, Novo Nordisk, and Anywhere Real Estate have since gone from 1 to enterprise-ready production applications.

Creating compelling, truly differentiated experiences for your customers from generative AI-enriched applications means grounding artificial intelligence in truth. That truth comes from your data, more specifically, your most up-to-date operational data. Whether you’re providing hyper-personalized experiences with advanced semantic search or producing user-prompted content and conversations, MongoDB Atlas unifies operational, analytical and vector search data services to streamline embedding the power of LLMs and transformer models into your apps.

Everyday, developers are building the next groundbreaking, transformative generative AI powered application. Commercial and open source LLMs are advancing at breakneck speed. The frameworks and tools to build around them are plentiful and democratize innovation. And yet taking these applications from prototype to enterprise-ready is the chasm development teams must cross. First, these large models can provide incorrect or uninformed answers, because the data they have access to is dated. There are two options to solve uninformed answers - fine-tuning a large model or providing it with long-term memory. However, doing so begets a second barrier - deploying an application around an informed LLM with the right security controls in place, and at the scale and performance users expect.

Developers need a data platform that has the data model flexibility to adapt to the constantly changing unstructured and structured data that informs large models without the hindrance of rigid schemas. While fine-tuning a model is an option, it’s a cost-prohibitive one in terms of time and computational resources. This means developers need to be able present data as context to large models as part of prompts. They need to give these generative models long-term memory. We discuss a few examples of how to do so with various LLMs and generative AI frameworks below.

Check out our AI resource page to learn more about building AI-powered apps with MongoDB.

Five resources to get started with MongoDB Atlas and Large Language Models

MongoDB Atlas makes it seamless to integrate leading generative AI services and systems such as the hyperscalers and open source LLMs and frameworks. By combining document and vector embedding data stores in one place via Atlas Database and Atlas Vector Search (preview), developers can accelerate building their generative AI-enriched applications that are grounded in the truth of operational data. Below are examples with how to work with popular LLM frameworks and MongoDB:

1. Get started with Atlas Vector Search (preview) and OpenAI for semantic search

This tutorial walks you through the steps of performing semantic search on a sample movie dataset with MongoDB Atlas. First, you’ll set up an Atlas Trigger to make a call to an OpenAI API whenever a new document is inserted into your cluster, so as to convert it into a vector embedding. Then, you’ll perform a vector search query using Atlas Vector Search. There’s even a special bonus section for leveraging HuggingFace models. Read the tutorial.

2. Build a Gen AI-enriched chat app with your proprietary data using Llamalndex and MongoDB

LlamaIndex provides a simple, flexible interface to connect LLMs with external data. This joint blog from LlamaIndex and MongoDB goes into more detail about why and how you might want to build your own chat app. The attached notebook in the blog provides a code walkthrough on how to query any PDF document using English language queries. Read the blog.

3. See the docs for how to use Atlas Vector Search (preview) as a vector store with LangChain

As stated in the partnership announcement blog post, LangChain and MongoDB Atlas are a natural fit, and it’s been demonstrated by the organic community enthusiasm which has led to several integrations in LangChain for MongoDB. In addition to now supporting Atlas Vector Search as a Vector Store there is already support to utilize MongoDB as a chat log history. Read the docs: python, javascript.

4. Generate predictions directly in MongoDB Atlas with MindsDB AI Collections

MindsDB is an open-source machine learning platform that brings automated machine learning to the database. In this blog you’ll generate predictions directly in Atlas with MindsDB AI Collections, giving you the ability to consume predictions as regular data, query these predictions, and accelerate development speed by simplifying deployment work-flows. Read the blog.

5. Integrate HuggingFace transformer models into MongoDB Atlas with Atlas Triggers

HuggingFace is an AI community that makes it easy to build, train and deploy machine learning models. Leveraging Atlas Triggers alongside HuggingFace allows you to easily react to changes in operational data that provides long-term memory to your models. Learn how to set up Triggers to automatically predict the sentiment of new documents in your MongoDB database and add them as additional fields to your documents. See the GitHub Repo.

Figure 1: The sample app architecture shows how external, or proprietary, data provides long-term memory to an LLM and how the data flows from a user's input to an LLM-powered response.

From prototype to production with MongoDB for generative AI-enriched apps

MongoDB’s developer data platform built on Atlas provides a modern, optimized developer experience while also being battle tested by thousands of enterprises globally to perform at scale and securely.

Whether you are building the next big thing at a startup or enterprise, Atlas enables you to:

  • Accelerate building your generative AI-enriched applications that are grounded in the truth of operational data.

  • Simplify your app architecture by leveraging a single platform that allows them to store app and vector data in the same place, react to changes in source data with serverless functions, and search across multiple data modalities for improving relevance and accuracy in responses that their apps generate.

  • Easily evolve your gen AI-enriched apps with the flexibility of the document model while maintaining a simple, elegant developer experience.

  • Seamlessly integrate leading AI services and systems such as the hyperscalers and open source LLMs and frameworks to stay competitive in dynamic markets.

  • Build gen AI-enriched applications on a high performance, highly scalable operational database that's had a decade of validation over a wide variety of AI use cases.

While these examples are the building blocks for something more innovative, MongoDB can help you go from concept to production to scale. Get started today by signing up for MongoDB Atlas free tier and integrating with your preferred frameworks and LLMs. If you’re interested in working with us more closely, check out our MongoDB AI Innovators program, which enables Artificial Intelligence innovation and showcases cutting-edge solutions from startups, customers, and partners.

← Previous

頁面標題：Atlas Vector Search 上市：借助語意搜尋和 AI 針對任何類型的資料構建智慧型應用程式

我們很高興地宣布 Atlas Vector Search 現已正式上市。Vector Search 現在支持生產工作負載，開發者能夠構建由語意搜尋和生成式人工智慧提供支持的智慧型應用程式，同時透過 Search Nodes （搜尋節點）來最佳化資源消耗並提高效能。 請閱讀下面的部落格，了解完整的公告以及該功能的一系列優勢。 這一刻終於到來。人工智慧已經觸手可及。原本僅限於企業內部資料科學和機器學習團隊中的工具，現在已隨時為世界各地的構建者敞開大門。但是，要充分運用這些新工具的強大功能，需要一個可靠、可靈活組合且簡練的資料平台來進行構建。同時，正如我們所知，這些新功能的好壞取決於它們能夠存取的資料或「基準真相」的品質。因此，我們很高興能在 MongoDB Atlas 開發者資料平台添加另一項新功能，釋放出資料的全部潛能並支持 AI 應用程式的發展。今天， MongoDB 隆重宣布全新的 Vector Search 功能，它旨在滿足各種形式的資料需求，使我們的合作夥伴得以享受這些令人驚豔的新功能所帶來的優勢。 歡迎各位查看我們的 AI 資源頁面 ，以瞭解更多使用 MongoDB 來建構 AI 驅動應用程式的相關資訊。 這項新功能是什麼？ Vector Search 是一種根據語意或資料含義，而非根據資料本身來查詢資料的功能。這種功能的實現原理在於，能將任何形式的資料轉為數位向量，並可透過複雜的演算法進行比較。第一步是獲取原始資料，例如文本、音訊、圖片或影片，並使用「編碼模型」將它們轉換為「向量」或「嵌入」。受益於人工智慧的最新進展，這些向量現在能透過將低構面資料投影到包含更多資料上下文的高構面空間中，從而更準確地理解資料的含義。一旦將資料轉換為數值表示，就可以使用近似最鄰近（Approximate Nearest Neighbor）演算法尋找相似值，該演算法能使查詢非常快速地找到具有相似向量的資料。這能讓使用者在用自然語言查詢如「請推薦一些悲傷的電影」或「請找一些類似……的圖片」等問題時更加滿意。這項功能解鎖了全新類型的可能性。 它與我們的平台有何關係？ MongoDB Atlas 已原生內建此功能，開發者無需複製和轉換資料、學習一些新的堆棧和語法，也無須管理一整套新設立的基礎架構。借助 MongoDB 的 Atlas Vector Search，這一切都不再是必需，開發者可以在一個經過實戰考驗的世界級平台中利用這些強大的新功能，以前所未有的速度構建應用程式。有效使用 AI 和 Vector Search 所面臨的許多挑戰，都源自於安全地公開應用程式資料所涉及的複雜性。這些任務會給開發者的體驗帶來多層的摩擦，並使應用程式更難構建、除錯和維護。MongoDB 消除了所有這些挑戰，同時將 Vector Search 的強大功能帶入一個可靈活垂直和水平擴展的平台，以支持任何的工作負載。最後，如果沒有對安全性和可用性的保證，這一切將毫無意義。MongoDB 致力於提供安全資料管理的解決方案，透過冗餘和自動故障轉移實現的高可用性，確保應用程式始終維持穩定運行。 MongoDB.local 倫敦新功能發表會 在 .Local 的倫敦發表會上，我們很高興地宣布推出專用的 Vector Search 聚合階段，該階段可以透過 $vectorSearch 調用。新的聚合階段引入了一些增加了新能力的新概念，並使 Vector Search 比以往都更容易使用。借助 $vectorSearch，開發者還可以使用 MQL 語法的預先篩選器（例如 $gte, $eq 等) ，以在遍歷索引時過濾文件，從而獲得一致的結果和高效能。任何了解 MongoDB 的開發者都能夠輕鬆利用這項過濾功能！最後，我們還介紹了兩種在聚合階段內部調整結果的方法，也就是「numCandidates」和「limit」參數。透過這些參數，開發者可以調整應為近似最鄰近搜索候選的文件數量，接著使用「limit」來限制想要的結果數量。 它如何與生態系統互動？ 人工智慧的發展及大量創新十分驚人，而開源社群的突飛猛進也令人嘆為觀止。開源語言模型以及將它們整合到應用程式中的各種方法取得了巨大的進步。隨著人工智慧展現出強大力量，建立能讓開發者對功能靈活發揮的穩固抽象也因此變得前所未見的重要。考量到這一點，我們非常興奮地向大家分享 LangChain 和 LlaMAindex，它們支持了多種功能，包括 Vector Search、聊天記錄和文件索引等。我們正在快速推進，並將繼續為主要提供商發布新功能。 總結 一切才剛開始，MongoDB 致力於提供市場上最好的開發者資料平台，協助開發者打造下一代結合 AI 的應用程式。我們還會持續研究更多可以支持的框架和外掛程式架構。然而始終不變的是，最重要的核心仍是開發者。我們將與社群交流，找到並提供最佳服務的方式，確保每一步都能滿足開發者的需求。繼續向前構建吧！ 內容區塊： 要了解有關 Atlas Vector Search 的更多資訊以及它是否適合您，請查看我們的 文件 、 白皮書 和 教學 ， 或立即開始使用 。 前往 MongoDB.local 中心查看我們接下來會在哪裡出現。

June 22, 2023
Next →

AI-Powered Call Centers: A New Era of Customer Service

Customer satisfaction is critical for insurance companies. Studies have shown that companies with superior customer experiences consistently outperform their peers. In fact, McKinsey found that life and property/casualty insurers with superior customer experiences saw a significant 20% and 65% increase in Total Shareholder Return , respectively, over five years. A satisfied customer is a loyal customer. They are 80% more likely to renew their policies, directly contributing to sustainable growth. However, one major challenge faced by many insurance companies is the inefficiency of their call centers. Agents often struggle to quickly locate and deliver accurate information to customers, leading to frustration and dissatisfaction. This article explores how Dataworkz and MongoDB can transform call center operations. By converting call recordings into searchable vectors (numerical representations of data points in a multi-dimensional space), businesses can quickly access relevant information and improve customer service. We'll dig into how the integration of Amazon Transcribe, Cohere, and MongoDB Atlas Vector Search—as well as Dataworkz's RAG-as-a-service platform— is achieving this transformation. From call recordings to vectors: A data-driven approach Customer service interactions are goldmines of valuable insights. By analyzing call recordings, we can identify successful resolution strategies and uncover frequently asked questions. In turn, by making this information—which is often buried in audio files— accessible to agents, they can give customers faster and more accurate assistance. However, the vast volume and unstructured nature of these audio files make it challenging to extract actionable information efficiently. To address this challenge, we propose a pipeline that leverages AI and analytics to transform raw audio recordings into vectors as shown in Figure 1: Storage of raw audio files: Past call recordings are stored in their original audio format Processing of the audio files with AI and analytics services (such as Amazon Transcribe Call Analytics ): speech-to-text conversion, summarization of content, and vectorization Storage of vectors and metadata: The generated vectors and associated metadata (e.g., call timestamps, agent information) are stored in an operational data store Figure 1: Customer service call insight extraction and vectorization flow Once the data is stored in vector format within the operational data store, it becomes accessible for real-time applications. This data can be consumed directly through vector search or integrated into a retrieval-augmented generation (RAG) architecture, a technique that combines the capabilities of large language models (LLMs) with external knowledge sources to generate more accurate and informative outputs. Introducing Dataworkz: Simplifying RAG implementation Building RAG pipelines can be cumbersome and time-consuming for developers who must learn yet another stack of technologies. Especially in this initial phase, where companies want to experiment and move fast, it is essential to leverage tools that allow us to abstract complexity and don’t require deep knowledge of each component in order to experiment with and realize the benefits of RAG quickly. Dataworkz offers a powerful and composable RAG-as-a-service platform that streamlines the process of building RAG applications for enterprises. To operationalize RAG effectively, organizations need to master five key capabilities: ETL for LLMs: Dataworkz connects with diverse data sources and formats, transforming the data to make it ready for consumption by generative AI applications. Indexing: The platform breaks down data into smaller chunks and creates embeddings that capture semantics, storing them in a vector database. Retrieval: Dataworkz ensures the retrieval of accurate information in response to user queries, a critical part of the RAG process. Synthesis: The retrieved information is then used to build the context for a foundational model, generating responses grounded in reality. Monitoring: With many moving parts in the RAG system, Dataworkz provides robust monitoring capabilities essential for production use cases. Dataworkz's intuitive point-and-click interface (as seen in Video 1) simplifies RAG implementation, allowing enterprises to quickly operationalize AI applications. The platform offers flexibility and choice in data connectors, embedding models, vector stores, and language models. Additionally, tools like A/B testing ensure the quality and reliability of generated responses. This combination of ease of use, optionality, and quality assurance is a key tenet of Dataworkz's "RAG as a Service" offering. Diving deeper: System architecture and functionalities Now that we’ve looked at the components of the pre-processing pipeline, let’s explore the proposed real-time system architecture in detail. It comprises the following modules and functions (see Figure 2): Amazon Transcribe , which receives the audio coming from the customer’s phone and converts it into text. Cohere ’s embedding model, served through Amazon Bedrock , vectorizes the text coming from Transcribe. MongoDB Atlas Vector Search receives the query vector and returns a document that contains the most semantically similar FAQ in the database. Figure 2: System architecture and modules Here are a couple of FAQs we used for the demo: Q: “Can you explain the different types of coverage available for my home insurance?” A: “Home insurance typically includes coverage for the structure of your home, your personal belongings, liability protection, and additional living expenses in case you need to temporarily relocate. I can provide more detailed information on each type if you'd like.” Q: “What is the process for adding a new driver to my auto insurance policy?" A: “To add a new driver to your auto insurance policy, I'll need some details about the driver, such as their name, date of birth, and driver's license number. We can add them to your policy over the phone, or you can do it through our online portal.” Note that the question is reported just for reference, and it’s not used for retrieval. The actual question is provided by the user through the voice interface and then matched in real-time with the answers in the database using Vector Search. This information is finally presented to the customer service operator in text form (see Fig. 3). The proposed architecture is simple but very powerful, easy to implement, and effective. Moreover, it can serve as a foundation for more advanced use cases that require complex interactions, such as agentic workflows , and iterative and multi-step processes that combine LLMs and hybrid search to complete sophisticated tasks. Figure 3: App interface, displaying what has been asked by the customer (left) and how the information is presented to the customer service operator (right) This solution not only impacts human operator workflows but can also underpin chatbots and voicebots, enabling them to provide more relevant and contextual customer responses. Building a better future for customer service By seamlessly integrating analytical and operational data streams, insurance companies can significantly enhance both operational efficiency and customer satisfaction. Our system empowers businesses to optimize staffing, accelerate inquiry resolution, and deliver superior customer service through data-driven, real-time insights. To embark on your own customer service transformation, explore our GitHub repository and take advantage of the Dataworkz free tier .

November 27, 2024