How DataSwitch And MongoDB Atlas Can Help Modernize Your Legacy Workloads

Data modernization is here to stay, and DataSwitch and MongoDB are leading the way forward. Research strongly indicates that the future of the Database Management System (DBMS) market is in the cloud, and the ideal way to shift from an outdated, legacy DBMS to a modern, cloud-friendly data warehouse is through data modernization.

There are a few key factors driving this shift. Increasingly, companies need to store and manage unstructured data in a cloud-enabled system, as opposed to a legacy DBMS which is only designed for structured data. Moreover, the amount of data generated by a business is increasing at a rate of 55% to 65% every year and the majority of it is unstructured. A modernized database that can improve data quality and availability provides tremendous benefits in performance, scalability, and cost optimization. It also provides a foundation for improving business value through informed decision-making. Additionally, cloud-enabled databases support greater agility so you can upgrade current applications and build new ones faster to meet customer demand.

Gartner predicts that by 2022, 75% of all databases will be on the cloud – either by direct deployment or through data migration and modernization. But research shows that over 40% of migration projects fail.

This is due to challenges such as:

  • Inadequate knowledge of legacy applications and their data design

  • Complexity of code and design from different legacy applications

  • Lack of automation tools for transforming from legacy data processing to cloud-friendly data and processes

It is essential to harness a strategic approach and choose the right partner for your data modernization journey. We’re here to help you do just that.

Why MongoDB?

MongoDB is built for modern application developers and for the cloud era. As a general purpose, document-based, distributed database, it facilitates high productivity and can handle huge volumes of data. The document database stores data in JSON-like documents and is built on a scale-out architecture that is optimal for any kind of developer who builds scalable applications through agile methodologies. Ultimately, MongoDB fosters business agility, scalability and innovation.

Key MongoDB advantages include:

  • Rich JSON Documents

  • Powerful query language

  • Multi-cloud data distribution

  • Security of sensitive data

  • Quick storage and retrieval of data

  • Capacity for huge volumes of data and traffic

  • Design supports greater developer productivity

  • Extremely reliable for mission-critical workloads

  • Architected for optimal performance and efficiency

Key advantages of MongoDB Atlas, MongoDB’s hosted database as a service, include:

  • Multi-cloud data distribution

  • Secure for sensitive data

  • Designed for developer productivity

  • Reliable for mission critical workloads

  • Built for optimal performance

  • Managed for operational efficiency

To be clear, JSON documents are the most productive way to work with data as they support nested objects and arrays as values. They also support schemas that are flexible and dynamic.

MongoDB’s powerful query language enables sorting and filtering of any field, regardless of how nested it is in a document. Moreover, it provides support for aggregations as well as modern use cases including graph search, geo-based search and text search. Queries are in JSON and are easy to compose. MongoDB provides support for joins in queries.

MongoDB supports two types of relationships with the ability to reference and embed. It has all the power of a relational database and much, much more.

Companies of all sizes can use MongoDB as it successfully operates on a large and mature platform ecosystem. Developers enjoy a great user experience with the ability to provision MongoDB Atlas clusters and commence coding instantly. A global community of developers and consultants makes it easy to get the help you need, if and when you need it. In addition, MongoDB supports all major languages and provides enterprise-grade support.

Why DataSwitch as a partner for MongoDB?

Automated schema re-design, data migration & code conversion

DataSwitch is a trusted partner for cost-effective, accelerated solutions for digital data transformation, migration and modernization through a modern database platform. Our no-code and low-code solutions along with cloud data expertise and unique, automated schema generation accelerates time to market. We provide end-to-end data, schema and process migration with automated replatforming and refactoring, thereby delivering:

  • 50% faster time to market

  • 60% reduction in total cost of delivery

  • Assured quality with built-in best practices, guidelines and accuracy

Data modernization: How “DataSwitch Migrate” helps you migrate from RDBMS to MongoDB

DataSwitch Migrate (“DS Migrate”) is a no-code and low-code toolkit that leverages advanced automation to provide intuitive, predictive and self-serviceable schema redesign from a traditional RDBMS model to MongoDB’s Document Model with built-in best practices. Based on data volume, performance, and criticality, DS Migrate automatically recommends the appropriate ETTL (Extract, Transfer, Transform & Load) data migration process.

DataSwitch delivers data engineering solutions and transformations in half the timeframe of the existing typical data modernization solutions. Consider these key areas:

  • Schema redesign – construct a new framework for data management. DS Migrate provides automated data migration and transformation based on your redesigned schema, as well as no-touch code conversion from legacy data scripts to MongoDB Atlas APIs. Users can simply drag and drop the schema for redesign and the platform converts it to a document-based JSON structure by applying MongoDB modeling best practices. The platform then automatically migrates data to the new, re-designed JSON structure. It also converts the legacy database script for MongoDB. This automated, user-friendly data migration is faster than anything you’ve ever seen.

    Here’s a look at how the schema designer works.

  • Refactoring – change the data structure to match the new schema. DS Migrate handles this through auto code generation for migrating the data. This is far beyond a mere lift and shift. DataSwitch takes care of refactoring and replatforming (moving from the legacy platform to MongoDB) automatically. It is a game-changing unique capability to perform all these tasks within a single platform.

  • Security – mask and tokenize data while moving the data from on-premise to the cloud. As the data is moving to a potentially public cloud, you must keep it secure. DataSwitch’s tool has the capability to configure and apply security measures automatically while migrating the data.

  • Data Quality – ensure that data is clean, complete, trustworthy, consistent. DataSwitch allows you to configure your own quality rules and automatically apply them during data migration.

In summary: first, the DataSwitch tool automatically extracts the data from an existing database, like Oracle. It then exports the data and stores it locally before zipping and transferring it to the cloud. Next, DataSwitch transforms the data by altering the data structure to match the re-designed schema, and applying data security measures during the transform step. Lastly, DS Migrate loads the data and processes it into MongoDB in its entirety.

Process Conversion

Process conversion, where scripts and process logic are migrated from legacy DBMS to a modern DBMS, is made easier thanks to a high degree of automation. Minimal coding and manual intervention are required and the journey is accelerated. It involves:

  • DML – Data Manipulation Language

  • CRUD – typical application functionality (Create, Read, Update & Delete)

  • Converting to the equivalent of MongoDB Atlas API

Degree of automation DataSwitch provides during Migration

Schema

Migration Activities DS Automation Capabilities
Application Data Usage Analysis 70%
3NF to NoSQL Schema Recommendation 60%
Schema Re-Design Self Services 50%
Predictive Data Mapping 60%

Process

Migration Activities DS Automation Capabilities
CRUD based SQL conversion (Oracle, MySQL, SQLServer, Teradata, DB2) to MongoDB API 70%

Data

Migration Activities DS Automation Capabilities
Migration Script Creation 90%
Historical Data Migration 90%
2 Catch Load 90%

DataSwitch Legacy Modernization as a Service (LMaas):

Our consulting expertise combined with the DS Migrate tool allows us to harness the power of the cloud for data transformation of RDBMS legacy data systems to MongoDB. Our solution delivers legacy transformation in half the time frame through pay-per-usage. Key strengths include:

● Data Architecture Consulting

● Data Modernization Assessment and Migration Strategy

● Specialized Modernization Services

DS Migrate Architecture Diagram

Contact us to learn more.

← Previous

Distinguish Data, Get Insights Faster with Conditional Formatting in Charts

The latest release of MongoDB Charts adds Conditional Formatting; an exciting new feature that enables chart authors to highlight important changes in their chart data, based on a set of rules that they define. Conditional Formatting rules can be applied to table charts and number charts . Why use Conditional Formatting? For table charts, the data is densely packed into the visualisation using rows and columns. This is great for comparing many values simultaneously, but as the density increases it may become more difficult to find and focus on the data that matters. Many authors use Number charts to track key individual metrics within their data. While the number itself can be useful, sometimes it isn’t enough to convey other necessary information for its context – for instance, is a high number good or bad? Conditional Formatting can aid users in understanding the data by applying different styles based on rules to highlight what is important, and to provide them with more context. See Conditional Formatting in Action with Formula 1 Data Formula 1 motorsport is what I like to refer to as the “sport of nerds”, because analyzing and understanding huge amounts of data, and being able to quickly make a decision on that analysis can be the difference between winning and losing. So let’s see how Conditional Formatting can help with this task using data from the 2021 FIA Formula 1 World Championship. Single Color Conditional Formatting Let’s start off with something simple. Below is a table showing the 2021 Drivers Championship after three rounds. A driver’s position in the championship is determined by the total number of points they have been awarded over successive rounds of the season. Let’s edit this chart and add Conditional Formatting to highlight the top three drivers in the championship with colors to represent 1st, 2nd, and 3rd place. Click on the Customize tab , and then click on the Conditional Formatting menu to expand the accordion. As you can see we haven’t yet defined any rules, so let’s add a new rule by clicking the + Add button. A drawer will open up from the left hand side of the screen displaying the Add Format Rules view. A conditional formatting rule must have at least one condition, and all conditions must match in order for the rule to be applied. Let’s highlight the row of the driver currently in 1st place by adding a single color rule with one condition. Since this rule will be determined by the driver’s current position in the championship, we need to add a condition to act on this data. We can target this field by selecting Pos from the Applies to select control. Now that we know what field we are targeting, we must next choose an operator to use for the comparison. Since we are only interested in data that matches a specific value, we select the Equal To numeric operator. Next we must provide an input for the operand to be compared to. For this rule we are only interested in highlighting the driver that is in first place in the championship, so we enter a value of 1 into the Input . You can think of this condition as saying; “is the value of the field Pos equal to the value of 1?” If it is, then apply the styling of this rule, otherwise do not. Finally we choose what styling changes should be applied by choosing from the options under Styling . In this example, we want to highlight the background color of the cell in a gold color to signify 1st place, and we will also apply a bold font weight to the text to make it more prominent. Additionally we also would like for these styles to be applied to the entire row, and not just the cell that the condition is applicable to, so we will check the Format entire row option. And that’s it! Once we save the rule, you’ll notice that the table re-renders in the Chart Builder Preview to show that the data is being evaluated correctly and the Conditional Formatting rule is applied. We then simply rinse and repeat this process to add additional rules to highlight the drivers in 2nd and 3rd place, resulting in the following output: Color Scale Conditional Formatting When comparing tabular data, sometimes it is desirable to use color to show where each value lies relative to other values in the column. The table below shows the race results for the third round of the 2021 FIA Formula 1 World Championship. Each row displays the final result for each driver taking part in the race. Let’s compare the Average Speed of each driver’s Fastest Lap using a Color Scale Conditional Formatting rule. Navigate to the Add Format Rule screen in the same way by going to “Customize > Conditional Formatting > + Add” , but this time select the Color Scale radio option. Note: Conditional Formatting Color Scale rules can only have one condition, and this condition can only be applied to fields in a Table Chart encoded as Value columns. Select the Fastest Lap Average Speed as the target for the condition. You’ll notice that unlike the discrete Single Color rules, there are no other settings to configure for the condition. This is because a Color Scale will compare the values across the documents in sort order, and will determine a background color to apply to the cell based on the rank of the value within the range. Since we are interested in finding the highest Average Speed across each driver’s fastest lap, we will select a sequential color scale, where higher values are colored green, and lower values are colored in white. Save the rule to see the changes applied. As you can see, for the third round of the 2021 FIA Formula 1 World Championship, the fastest lap average speed was set by Valtteri Bottas at a blistering speed of 209.74 km/h (130.32 mph)! And there we have it. I hope this brief introduction to Conditional Formatting has highlighted (pun intended) the capabilities of this exciting new feature! In this post we’ve only scratched the surface of what’s possible though, Conditional Formatting has many many more powerful operators than what we have demonstrated here, including matching values by range, regular expression and even ranks. Why not take it for a test drive yourself to see what is possible? If you haven’t tried Charts yet, it’s quick, easy and free to get started. Simply sign up for MongoDB Cloud , deploy a free Atlas cluster and click Charts in the top navigation bar. You can also ask questions on the MongoDB Developer Community Forums , or suggest new or improved features using the MongoDB Feedback Engine .

May 13, 2021
Next →

MongoDB Database Observability: Integrating with Monitoring Tools

This post is the final in a three-part series on leveraging database observability. Welcome back to our series on Leveraging Database Observability! Our previous post showcased a real-world use case highlighting how MongoDB Atlas’s observability tools effectively tackle database performance challenges. Whether you’re a developer, DBA, or DevOps engineer, our mission is to empower you to harness the full potential of your data through our observability suite . Integrating Atlas metrics with your central enterprise observability tools can simplify your operations. By seamlessly working with popular observability tools, our approach helps teams streamline workflows and enhance visibility across systems. Integrating MongoDB Atlas with third-party monitoring tools MongoDB’s developer data platform combines all essential data services for building modern applications within a unified experience. Our purpose-built observability tools for Atlas environments offer automatic monitoring and optimization, guiding diagnostics tailored specifically for MongoDB. Additionally, we extend Atlas metrics into your existing enterprise observability stack, enabling seamless integration without replacing your current tools. This creates a consolidated, single-pane view that unifies Atlas telemetry with other tech and application metrics, ensuring comprehensive visibility into both database and full-stack performance. This integration empowers you to monitor, receive alerts, and make data-driven decisions within your existing workflows, driving greater efficiency. Below is a quick guide to modifying integration settings through the Atlas UI and the popular integrations we support: Navigate to the Project Integrations page in Atlas. Choose the organization and project you want to configure from the navigation bar. On the Project Integrations page, select the third-party services you’d like to integrate. Configure the chosen services with the required API keys and regions. Critical integrations for your observability platform With Atlas’s Datadog and Prometheus integrations, you can send critical MongoDB metrics to these platforms, empowering detailed, real-time monitoring. Through Datadog , you can track database operation counts, query efficiency, and resource usage, ideal for pinpointing bottlenecks and managing resources. Similarly, Prometheus enables you to monitor essential metrics like query times, connection rates, and memory usage, supporting flexible tracking of database health and performance. Both integrations facilitate proactive detection of issues, alert configuration for resource thresholds, and a cohesive view of Atlas data when visualized in Grafana. Atlas’s integration with PagerDuty streamlines incident management by sending metrics like performance alerts, billing anomalies, and security events directly to PagerDuty. This integration records incidents automatically, notifies teams upon alerts, and supports two-way syncing, ensuring resolved alerts in Atlas are reflected in PagerDuty. It enables efficient incident response and resource allocation to maintain system stability. With Atlas integrations for Microsoft Teams and Slack, you can route key metrics—such as query latency, disk usage, and throughput—to these channels for timely updates. Teams can use these insights for real-time performance monitoring, incident response, and collaboration. Notifications through these platforms ensure your team stays informed on database performance, storage health, and user activity changes as they occur. Use case: Centralized observability with MongoDB Atlas, Datadog, and Slack Let’s walk through a hypothetical scenario for ShopSmart, an e-commerce company that leverages MongoDB Atlas to manage its product catalog and customer data. As traffic surges, the DevOps team faces challenges in monitoring application performance and database health effectively. To tackle these challenges, the team leverages MongoDB Atlas’ integration with Datadog and Slack, creating a powerful observability ecosystem. Integrating MongoDB Atlas with Datadog: The team pushes key MongoDB Atlas metrics into Datadog, such as query performance, connection counts, and Atlas Vector Search metrics. With Datadog, they can visualize these metrics and correlate overall MongoDB performance with their other applications. Out-of-the-box monitors and dedicated dashboards allow the team to track metrics like throughput, average read/write latency, and current connections. This visibility helps pinpoint bottlenecks in real time, ensuring optimal database performance and improving overall application responsiveness. Setting up alerts in Datadog: The team configures alerts for critical metrics like high query latency and increased error rates. When thresholds are breached, Datadog instantly notifies the team. This proactive approach allows the team to address potential performance issues before they impact customers. Integrating Datadog with Slack: To ensure fast communication, alerts are sent directly to the dedicated Slack channel, “ShopSmart-Alerts.” This integration fosters seamless collaboration, enabling the team to discuss and resolve issues in real-time. With these integrations, ShopSmart’s engineering team can monitor performance quickly and address issues efficiently. The unified observability approach enhances operational efficiency, improves the customer experience, and supports ShopSmart’s competitive edge in the e-commerce industry. By leveraging MongoDB Atlas, Datadog, and Slack, the team ensures scalable performance and drives continuous innovation. Conclusion MongoDB Atlas empowers developers and organizations to achieve unparalleled observability and control over their database environments. By seamlessly integrating with central enterprise observability tools, Atlas enhances your ability to monitor performance metrics and ensures you can do so within your existing workflows. This means you can focus on building modern applications confidently, knowing you have the insights and alerts necessary to maintain optimal performance. Embrace the power of MongoDB Atlas and transform your approach to database management—because your applications can thrive when your data is observable. And that wraps up our Leveraging Database Observability series! We hope you learned something new and found value in these discussions. Sign up for MongoDB Atlas , our cloud database service, to see database observability in action. To dive deeper and expand your knowledge, check out this learning byte for more insights on the MongoDB observability suite and how it can enhance your database performance.

November 14, 2024