Elevate Your Python AI Projects with MongoDB and Haystack

Taylor Pacelli
July 8, 2024

MongoDB is excited to announce an integration with Haystack, enhancing MongoDB Atlas Vector Search for Python developers. This integration amplifies our commitment to providing developers with cutting-edge tools for building AI applications centered around semantic search and Large Language Models (LLMs).

We’re excited to partner with MongoDB to help developers build top-tier LLM applications. The new Haystack and MongoDB Atlas integration lets developers seamlessly use MongoDB data in Haystack, a reliable framework for creating quality LLM pipelines for use cases like RAG, QA, and agentic pipelines. Whether you're an experienced developer or just starting, your gen AI projects can quickly progress from prototype to adoption, accelerating value for your business and end-users.

Malte Pietsch, co-founder and CTO, deepset

Simplifying AI app development with Haystack

Haystack is an open-source Python framework that simplifies AI application development. It enables developers to start their projects quickly, experiment with different AI models, and to efficiently scale their applications.

Indeed, Haystack is particularly effective for building applications requiring semantic understanding and natural language processing (NLP), such as chatbots and question-answering systems. Haystack’s core features include:

  • Components: Haystack breaks down complex NLP tasks into manageable components, such as document retrieval or text summarization. With the new MongoDB-Haystack integration, MongoDB becomes the place where all your data lives, ready for Haystack to use.

  • Pipelines: Haystack lets you link components together into pipelines for more complex tasks. With this integration, your MongoDB data flows through these pipelines.

  • Agents: Haystack Agents use LLMs to resolve complex queries. They can decide which tools (or components) to use for a given question, leveraging MongoDB data to deliver smarter answers.

Atlas Vector Search: Enhance AI development with Haystack

At the heart of the new integration is MongoDB Atlas Vector Search, transforming how applications search and retrieve data. By leveraging vector embeddings, Atlas Vector Search goes beyond mere keyword matching: it interprets the intent behind queries, enabling applications to provide highly relevant, context-aware responses. This is a breakthrough for Python developers who aim to build applications that think and understand like humans.

Building on this foundation, the Atlas Vector Search and Haystack integration gives Python developers a powerful toolkit for navigating the complexities of AI application development. MongoDB becomes a dynamic document store within Haystack's framework, optimizing data storage, processing, and retrieval.

Additionally, the integration eases the use of advanced AI models from leading providers such as OpenAI and Cohere into your applications. Developers can thus create applications that do more than just answer queries—they grasp and act on the underlying intent, ensuring responses are both accurate and contextually relevant.

What this means for Python developers

For Python developers, this integration means:

  • Faster development: Developers can focus on building and innovating rather than spending time configuring and managing infrastructure. MongoDB's integration with Haystack means you can get up and running quickly, leveraging the best of both technologies to accelerate your development cycles.

  • Smarter applications: By utilizing Haystack's powerful Natural Language Processing tooling in combination with MongoDB Atlas Vector Search’s efficient data handling, developers can create applications that understand and process natural language more effectively. This results in applications that can provide more accurate and contextually relevant responses that resonate with user intent.

  • Access to pre-trained AI models: With seamless integration of leading generative AI models from providers like OpenAI, Anthropic, Cohere, Hugging Face, and AWS Bedrock, Python developers can easily incorporate advanced AI functionalities into their projects. This means developers can quickly adopt state-of-the-art models without the need for extensive training or fine-tuning, saving time and resources.

  • Flexible and scalable pipelines: Haystack's modular approach to building AI applications, through its use of components and pipelines, allows developers to create flexible and scalable solutions. With MongoDB data seamlessly flowing through these pipelines, you can easily adapt and expand your applications to meet growing demands and new challenges.

  • Robust search capabilities: Atlas Vector Search transforms the way applications retrieve and interpret data, going beyond simple keyword searches. It enables applications to perform high-precision searches that return more relevant and semantically rich results. This advanced search capability is crucial for developing applications that require high levels of semantic understanding and accuracy.

By integrating MongoDB with Haystack, Python developers are equipped with a powerful toolkit that not only simplifies the AI development process but also significantly enhances the intelligence and functionality of their applications. Whether you are building chatbots, search engines, or other AI-driven applications, this integration provides the tools you need to create innovative and impactful solutions.

Get started now

Start leveraging the MongoDB and Haystack integration for your AI development. Explore our tutorial, documentation, or check out our github repository to begin building smarter, more intuitive Python projects today!

← Previous

Nokia Corteca Scales Wi-Fi Connectivity to Millions of Devices With MongoDB Atlas

Nokia’s home Wi-Fi connectivity cloud platform was launched in 2019 as the Nokia WiFi Cloud Controller (NWCC). In 2023, it was renamed and relaunched as the Corteca Home Controller, becoming part of the Corteca software suite that delivers smarter broadband for a better experience. The Corteca Home Controller can be hosted on Amazon Web Services, Google Cloud, or Microsoft Azure, and is the industry’s first platform to support three management services—device management, Wi-Fi management, and application management. Supporting TR-369 (a standardized remote device management protocol) also allows the Home Controller to work in a multi-vendor environment, managing both Nokia broadband devices and third-party broadband devices. By solving connectivity issues before the end-user detects them, and by automatically optimizing Wi-Fi performance, the Home Controller helps deliver excellent customer experiences to millions of users, 24/7. During the five years that Nokia Corteca has been a MongoDB Atlas customer, the Home Controller has successfully scaled from 500,000 devices to over 4.5 million. There are now 75 telecommunications customers of Home Controller spread across all regions of the globe. Having the stability, efficiency, and performance to scale Nokia Corteca's solution is end-to-end, from applications embedded in the device, through the home, and into the cloud. Algorithms assess data extracted from home networks, based on which performance parameters automatically adjust as needed—changing Wi-Fi channels to avoid network interference, for example—thereby ensuring zero downtime. The Home Controller processes real-time data sent from millions of devices, generating massive volumes of data. With a cloud optimization team tasked with deploying the solution across the globe to ever more customers, the Home Controller needed to store and manage its vast dataset and to onboard new telecommunication organizations more easily without incurring any downtime. Prior to Nokia Corteca moving to MongoDB Atlas, its legacy relational database lacked stability and required both admin and application teams to manage operations. A flexible model with time series capabilities That's where MongoDB Atlas came in. Nokia was familiar with the MongoDB Atlas database platform, having already worked with it as part of a previous company acquisition and solution integration. As Nokia's development team had direct experience with the scalability, manageability, and ease of use offered by MongoDB Atlas, they knew it had the potential to address the Home Controller’s technical and business requirements. There was another key element: Nokia wanted to store time-series data—a sequence of data points in which insights are gained by analyzing changes over time. MongoDB Atlas has the unique ability to store operational and time series data in parallel and provides robust querying capabilities on that data. Other advantages include MongoDB's flexible schema, which helps developers store data to match the application's needs and adapt as data changes over time. MongoDB Atlas also provides features such as Performance Advisor that monitors the performance of the database and makes intelligent recommendations to optimize and improve the performance and resource consumption Fast real time data browsing and scalability made easy Previously, scaling the database had been time-consuming and manual. With MongoDB Atlas, the team can easily scale up as demand increases with very little effort and no downtime. This also means it is much more straightforward to add new clients, such as large telecommunications companies. Having started with 100GB of data, the team now has more than 1.3 terabytes, and can increase the disc space in a fraction of a second, positioning the team to be able to scale with the business. As the Home Controller grows and onboards more telcos, the team anticipates a strengthening relationship with MongoDB. “We have a very good relationship with the MongoDB team,” said Jaisankar Gunasekaran, Head of Cloud Hosting and Operations at Nokia. “One of the main advantages is their local presence—they’re accessible, they’re friendly, and they’re experts. It makes our lives easier and lets us concentrate on our products and solutions.” To learn more about how MongoDB can help drive innovation and capture customer imaginations, check out our MongoDB for Telecommunications page.

July 2, 2024
Next →

Exploring New Security, Billing, and Customization Features in Atlas Charts

MongoDB is excited to announce a few new updates to Atlas Charts that enable you to securely share insights, gain deeper visibility into expenses, and customize your most frequently used data visualizations. Based on specific feedback received from users of our native visualization tool, these significant improvements will make data analysis even more productive. We: Improved security in Atlas Charts for passcode-protected public dashboards Increased visibility into Atlas spending through an updated billing dashboard Introduced new customization for table charts through hyperlinks and hidden columns Secure insights with passcode-protected public dashboards First, there’s the new passcode-protected public dashboards feature that brings an extra layer of security to publicly shared dashboards—we understand that not everyone who benefits from Atlas Charts operates within MongoDB Atlas. Alongside the ability to schedule email reports and support for publicly-shared dashboards , we’re offering a new and secure way to spread insights with the launch of our latest feature. Add an extra layer of security to previously publicly shared dashboards, ensuring that only authorized users with the passcode can access your data. Enabling passcode protection on a dashboard is simple. As a dashboard owner, a new option is available to protect dashboard links with a passcode when sharing it publicly. Check the box to protect your public link with a passcode Once enabled, a passcode is automatically generated and can be copied to the clipboard (and regenerated on demand as needed). Viewers navigating to dashboards via the public link will see a new screen prompting them to enter a passcode. Once authenticated successfully, they can view the dashboard just as before. Easily access your dashboards by inputting your password when prompted Whether you're sharing insights with clients, stakeholders, or team members, rest assured that your data remains easily accessible yet secure. To learn more about the different ways we support dashboard sharing, check out our documentation . What’s new in the Atlas Charts billing dashboard Next, we continue to make enhancements to the MongoDB Atlas Charts billing dashboard , all of which provide insights into Atlas expenses. We are delighted to share that it’s now possible to see resource tags data, as well as billing data from all linked organizations inside the Atlas Charts billing dashboard. Additionally, users can now ingest billing data from another organization, provided they possess the organization’s API keys. These newly introduced features rely on the availability of billing data within the organization. And for those leveraging resource tags, the billing data will seamlessly integrate, empowering users to generate personalized charts or to incorporate tailored dashboard filters within the Atlas Charts billing dashboard. If cross-organization billing is enabled, editing the configuration will ingest the linked organization’s billing data for the last three months, with the option to extend this period to up to a year by creating a new ingestion. Project tags data in the Atlas Charts billing dashboard Resource tags are now seamlessly integrated into billing data and can be included in any of the charts or the dashboard filters inside the Atlas billing dashboard. For example, our MongoDB organization uses the Atlas auto-suggested tags “application” and “environment,” alongside a custom resource tag labeled "team." The following chart uses the tags data and shows the billing cost per team and per environment. A chart which depicts cost per team and environment using tags The subsequent chart presents the billing cost allocated per project and team, providing valuable insights into the primary cost drivers for each team's projects. A chart depicting cost allocated per project and team Users can also add a dashboard filter to the “tags” field, which will allow them to see the whole dashboard based on the selected tag values. In the next example, we have selected a specific “team” : “Charts” from the tags dashboard filter, so we can see all of the billing insights per team thanks to our custom tag. Billing insights filtered by specific "charts" team in an intuitive dashboard Linked organization’s data in the Atlas Charts billing dashboard For complex Atlas projects spanning multiple organizations, the Atlas Charts billing dashboard now seamlessly integrates billing data from all linked organizations. The most productive use case is to add a dashboard filter based on the "organizationId" to enable filtering data according to specific organizations for a more granular analysis of the spending. Dashboard filtered by the organizationID field to show insights for one organization Billing data from another organization Users can now ingest billing data from other organizations that are not directly linked, provided they possess authorization API keys, bringing the data you need to where you are. Provide the API key to ingest billing data from other organizations These new features in the Atlas Charts billing dashboard are designed to provide richer, more detailed insights into organization spend. Check out our documentation and our previous blog post to learn more about it. Hyperlinks and hidden columns for tables in Atlas Charts Of all the data visualization methods available in Atlas Charts, table charts rank as one of the most popular among our users. So it should come as no surprise that one of the most highly requested features from our customers is the ability to format columnar data as hyperlinks. We're excited to announce that this is now possible in Atlas Charts through the new hyperlink customization options available for table charts . With hyperlink customization, you can format columnar data as hyperlinks using any of the following URI protocols: http, https, mailto, or tel, and can be constructed statically or dynamically using encoded fields. Let’s assume we’ve created a table using the sample movies dataset in Atlas, with encodings like title, imdb.id, runtime, genre, poster_display—which is a calculated field —and more. Customization panel in Atlas Charts To turn movie titles into clickable links that direct users to their respective IMDB pages, navigate to the customization panel and click into the hyperlinking feature in the fields tab . We will format the title field as a hyperlink which links to the Internet Movie Database (IMDB) entry for that movie. IMDB URLs are formatted as follows, where id needs to be substituted with the value of the imdb.id field for each document. https://www.imdb.com/title/tt<id>/ Customize the “title” field in the table chart to link to IMDB using the “imdb.id” field in the URI input. Below, a preview displays the fully formatted URI with fields substituted for their values, helping to ensure it’s correct before we save it to be applied to the chart. Preview of URI in the hyperlinking panel Since we only need the imdb.id field to be encoded for the purpose of constructing the URI applied to the title field, we can hide the column from rendering using another new customization option. Select the imdb.id field in the customization panel, and toggle on the “Hide Column” option. Toggle "Hide Column" We also support using URI values directly from fields (provided they use one of the supported protocols). Let’s see this in action by creating a hyperlink to the movie poster. In the URI input, trigger the encoded field menu using the @ keyboard shortcut, and select the poster field. Similar to the previous example, a preview will be displayed. After saving and applying the hyperlink formatting, we can hide the rendering of the poster field as needed to keep the chart clean. Use the @ keyboard shortcut to trigger the encoded field menu All these options are accessible in the customization panel, making it straightforward to enhance table charts with interactive hyperlinks. For more detailed instructions, visit our documentation . As we conclude this roundup, we hope you’re as excited about these updates as we are. The Atlas Charts team is dedicated to continuously improving Atlas Charts to meet your needs and enhance your data visualization experience. Stay tuned for more updates, and happy charting! New to Atlas Charts? Get started today by logging into or signing up for MongoDB Atlas , deploying or selecting a cluster, and activating Charts for free.

September 5, 2024