Consider Your Cloud Backup Strategy on World Backup Day, March 31

Steve Jurczak
March 31, 2022

World Backup Day, marked every March 31, reminds all of us of the risk and vulnerability of the data stored on our devices and systems.

Data loss happens every day for a variety of reasons. While human error is the most common cause, ransomware is fast becoming the most expensive one. A big reason is because ransomware criminals have moved on from targeting individuals to attacking businesses, where data is far more sensitive and valuable. Businesses have more to lose if their data is compromised or exposed, and they have deeper pockets to pay larger ransoms. Having a backup copy of data can help a business recover clean copies of data after it’s been corrupted by malware, accidentally deleted, or destroyed by a fire or flood.

Backup could also save the day during cloud outages. For businesses, any type of data loss is extremely costly. So having a backup plan as part of an overall disaster recovery strategy is critical for the survival of every business.

Backup benefits

A backup and disaster recovery strategy is necessary to protect your mission critical data against these types of risks. With such a strategy in place, you'll gain peace of mind knowing that if your data ever becomes accidentally deleted or infected by malware, you'll be able to recover it and avoid the cost and consequences of data loss. You'll also satisfy important regulatory and compliance requirements by demonstrating that you've taken a proactive approach towards data safety and business continuity.

Taking regular backups offers other advantages as well. The backups can be used to create new environments for development, staging, or QA without impacting production. This practice enables development teams to quickly and easily test new features, accelerating application development and ensuring smooth product launches.

Backup and the cloud factor

Back when business systems were mostly on-premises, there was a simple framework for disaster recovery planning: the 3-2-1 backup rule. It essentially recommends keeping three copies of your data, in at least two different form factors, with one of them kept at an offsite or remote location. In practice, this might mean creating a bare metal image of an entire server and storing it on a secondary server in the same server room. In addition, you would also make a copy of all of your server data on magnetic tape and transport it to an offsite, secure location. So that’s three copies of data (the original, the bare metal image, and the tape backup) in two different form factors (disc and tape), with one stored offsite.

The 3-2-1 backup rule has worked well for a lot of businesses for decades. But systems have changed. First, barely anyone uses tape anymore. The time it takes to transport a tape backup to where it can be used for disaster recovery is more than most businesses can tolerate. The criticality of systems has shrunk recovery time objectives (RTO) to just hours or minutes. Tape backups are simply not practical for most modern IT environments. But the cloud is.

Hyperscale cloud providers can provide service levels that are just as reliable, if not moreso, than traditional on-premises environments. But customers shouldn’t make the mistake of thinking that by deploying in the cloud, they’re off the hook when it comes to planning and implementing a disaster recovery strategy. Businesses can and do lose data in the cloud. And hyperscale cloud providers do experience outages. Businesses must have a backup strategy, but it needs to be updated to factor in the cloud workloads that have become ubiquitous.

Cross-cloud data protection

An outage at any of the major cloud providers could take your databases offline. Keeping extra backup copies with different cloud providers is a good way to ensure you can still access your data in the event of an outage with a single cloud provider.

Recent outages at hyperscale cloud providers have underscored the need for cross-cloud backup. “Most cloud outages are related to software bugs rather than physical catastrophes,” says Chris Shum, Atlas product lead at MongoDB. “We've always protected ourselves against physical catastrophes by distributing across data centers or regions, but no one really protects themselves against the software bug.” Shum says by backing up workloads on different clouds, you could tolerate a cloud provider going down and your database and your application would still be up.

Getting around the cloud backup skills gap

Data gravity keeps many businesses locked into a single cloud provider. Becoming fluent with a particular cloud provider is a skill to acquire just like anything else. And once you become comfortable with one provider, its hardware configurations, operational tools, and its offerings, it can be hard to venture out to a different provider with different hardware and pricing. But developing flexibility in the cloud is critical if you hope to leverage the best features, functionality, and cost efficiencies from each cloud provider. MongoDB has made it possible to do just that.

“With Atlas, we’ve made it our mission to abstract as much of the management away as possible,” Shum says. “It’s all available as a fully managed service. So things like hardware asymmetry between different cloud providers, offerings being different, prices being different, how you’d set up networking infrastructure, all of those things that to you as a consumer might be different, we’ve abstracted it away for you. And because of the abstraction, you are then free to move nodes to whichever cloud provider or region you want.”

Data protection with Atlas

MongoDB Atlas provides point in time recovery of replica sets and cluster-wide snapshots of sharded clusters. It’s simple to restore to precisely the moment you need, quickly and safely.

Backups can be restored automatically to the existing MongoDB Atlas cluster or downloaded to be manually archived or restored on a different infrastructure. MongoDB Atlas provides:

  • Security features to protect access to your data

  • Built in replication for always-on availability, tolerating complete data center failure

  • Backups and point in time recovery to protect against data corruption

  • Fine-grained monitoring to let you know when to scale — additional instances can be provisioned with the push of a button

  • Automated patching and one-click upgrades for new major versions of the database, enabling you to take advantage of the latest and greatest MongoDB features

  • A choice of cloud providers, regions, and billing options

Backup wrap-up

If you’re still depending on a single cloud provider to keep your critical workloads online and available, consider World Backup Day as your reminder to identify and close the gaps in your cloud disaster recovery strategy. For information on how to back up and restore cluster data in MongoDB read this article in our documentation.

← Previous

Introducing the Newest Version of the MongoDB Spark Connector

MongoDB has just released an all-new version of our Spark Connector. This article discusses the background behind the MongoDB Spark Connector and some of the key features of the new release. Why a new version? The current version of the MongoDB Spark Connector was written in 2016 and is based on Version 1 of the Spark Data Source API. This API is still supported, but Databricks has released an updated version of the API, making it easier for data sources like MongoDB to work within the Spark ecosystem. By using Version 2 of the MongoDB Spark Connector, you’ll immediately benefit from capabilities such as tighter integration with Spark Structured Streaming. MongoDB will continue to support Version 1 until Databricks deprecates its Datasource API, but no new features will be implemented, and upgrades to the Connector will include only bug fixes and support for the current version. Which version should I use? The new Spark Connector (Version 10.0) is not intended to be a direct replacement for applications that use the current MongoDB Spark Connector. Note that the new connector uses a different namespace, “com.mongodb.spark.sql.connector.MongoTableProvider”, versus the original Spark Connector, which uses “com.mongodb.spark.DefaultSource”. Having a different namespace makes it possible to use both versions of the Connector within the same Spark application. This is helpful in unit testing your application with the new Connector and making the transition on your timeline. Also note a change with versioning of the MongoDB Spark Connector. The current version of the existing MongoDB Spark Connector is 3.0. Up until now, as MongoDB released versions of the connector, the number was aligned with the version of Spark that was supported—i.e., Version 2.4 of the MongoDB Spark Connector works with Spark 2.4. Going forward, this will not be the case. MongoDB's documentation will make clear which versions of Spark the Connector supports and provide the appropriate information. Structured Streaming to MongoDB Apache Spark comes with a stream processing engine called Structured Streaming, which is based on Spark's SQL engine and DataFrame APIs. Spark Structured Streaming treats each incoming stream of data as a microbatch, continually appending each microbatch to the target dataset. This makes it easy to convert existing Spark batch jobs into streaming jobs. Structured Streaming provides maximum throughput via the same distributed capabilities that have made Spark such a popular platform. In the following example, we’ll show you how to stream data to MongoDB using Structured Stream. Consider a CSV file that contains natural gas prices. The following PySpark code will read the CSV file into a stream, compute a moving average, and stream the results into MongoDB. from pyspark.sql.types import StructType, DateType, StringType, TimestampType, DoubleType from pyspark.sql import functions as F from pyspark.sql.window import Window from pyspark.sql.functions import lit, count sc.setLogLevel('DEBUG') readSchema = ( StructType() .add('Type', StringType()) .add('Date', TimestampType()) .add('Price', DoubleType()) ) ds = (spark .readStream.format("csv") .option("header", "true") .schema(readSchema) .load("daily*.csv")) slidingWindows = (ds .withWatermark("Date", "1 minute") .groupBy(ds.Type, F.window(ds.Date, "7 day")) .avg() .orderBy(ds.Type,'window')) dsw = ( slidingWindows .writeStream .format("mongodb") .queryName("7DaySlidingWindow") .option("checkpointLocation", "/tmp/pyspark/") .option("forceDeleteTempCheckpointLocation", "true") .option('spark.mongodb.connection.uri', 'MONGODB CONNECTION HERE') .option('spark.mongodb.database', 'Pricing') .option('spark.mongodb.collection', 'NaturalGas') .outputMode("complete")) query = dsw.start() query.processAllAvailable() query.stop() For more information and examples on the new MongoDB Spark Connector V10.0, check out our documentation . Ask questions and give feedback on the MongoDB Community Forum. The Connector is open sourced; feel free to contribute at GitHub .

March 31, 2022
Next →

MongoDB and Partners: Building the AI Future, Together

If you’re like me, over the past year you’ve closely watched AI’s developments—and the world’s reactions to them. From infectious excitement about AI’s capabilities, to impatience with its cost and return on investment, every day has been filled with AI twists and turns. It’s been quite the roller coaster. During the ride, from time to time I’ve wondered where AI falls on the Gartner hype cycle, which gives "a view of how a technology or application will evolve over time." Have we hit the "peak of inflated expectations" only to fall into the "trough of disillusionment?" Or is the hype cycle an imperfect guide, as The Economist argues? The reality is that it takes time for any new technology—even transformative ones like AI—to take hold. And every advance, no matter how big, has had its detractors. A famous example is that of Picasso (!), who in 1968 said, “Computers are useless. They can only give you answers.” (!!) For our part, MongoDB is convinced that AI is a once-in-a-generation technology that will enhance every future application—a belief that has been reinforced by the incredible work our partners have shared at MongoDB’s 2024 events. Speeding AI development MongoDB is committed to helping organizations of all sizes succeed with AI, and one way we’re doing that is by collaborating with the MongoDB partner ecosystem to create powerful, user-friendly AI development tools and solutions. For example, Fireworks.ai —which is a member of the MongoDB AI Applications Program ecosystem —created an inference solution that hosts gen AI models and supports containerized deployments. This tool makes it easier for developers to build and deploy powerful applications with a range of easy-to-use tools and customization options. They can choose to use state-of-the-art, open-source language, image, and multimodal foundation models off the shelf, or they can customize and fine-tune models to their needs. Jointly, Fireworks.ai and MongoDB provide a solution for developers who want to leverage highly curated and optimized open-source models and combine these with their organization’s own proprietary data—and to do so with unparalleled speed and security. “MongoDB is one of the most sophisticated database providers, and it’s very easy to use,” said Benny Chen , cofounder of Fireworks.ai. "We want developers to be able to use these tools, and we want to work with providers who enable and empower developers." Nomic , another MAAP ecosystem member, also enables developers with best-in-class solutions across the entire unstructured data workflow. Their Embed offering, available through the Nomic API , allows users to vectorize large-scale datasets for use in text, image, and multimodal retrieval applications, including retrieval-augmented generation (RAG), using only their web browser. The Nomic-MongoDB solution is a highly efficient, open-weight model that developers can use to visualize the unstructured datasets they store in MongoDB Atlas . These insights help users quickly discover trends and articulate data-driven value propositions. Nomic also supported the recently announced vector quantization in MongoDB Atlas Vector Search , which reduces vector sizes while preserving performance. Last—but hardly least!—there’s our new reference architecture with MAAP partners AWS and Anthropic. Announced at MongoDB.local London , the reference architecture supports building memory-enhanced AI agents, and is designed to streamline complex processes and develop smarter, more responsive applications. For more—including a link to the code on Github— check out the MongoDB Developer Center . Making AI work for anyone and everyone The companies MongoDB partners with aren’t just making gen AI easier for developers—they’re building tools for everyone. For example, Capgemini has invested $2 billion in gen AI and is training 100,000 of its employees in the technology. GenYoda, a solution that helps insurance professionals with their daily work, is a product of this investment. GenYoda leverages MongoDB Atlas Vector Search to analyze large amounts of customer data, like policy statements, premiums, claims history, and health information. Using GenYoda, insurance professionals can quickly analyze underwriters’ reports to make informed decisions, create longitudinal health summaries, and streamline customer interactions to improve contact center efficiency. GenYoda can ingest 100,000 documents in just a few hours and respond to users’ queries in two to three seconds—a metric on par with the most widely used gen AI models. And it produces results: in one example, by using Capgemini’s solution an insurer was able to increase productivity by 15%, add new reports 25% faster (thus speeding decision-making), and reduce the manual effort of searching PDFs, increasing efficiency by 10%. Building the future of AI together So, what’s next? Honestly, I’m as curious as you are. But I’m also incredibly excited. At MongoDB, we’re active participants in the AI revolution, working to embrace the possibilities that lie ahead. The future of gen AI is bright, and I can’t wait to see what we’ll build together. To learn more about how MongoDB can accelerate your AI journey, explore the MongoDB AI Applications Program .

November 4, 2024