From the 1980s to the internet era in the late 1990s, SQL databases dominated the development landscape. Large commercial applications, niche products, and custom applications of all types were based on SQL.
But the rise of the internet has changed application development profoundly. The amount of data, the structure of the data, the scale of applications, the way applications have developed have all changed dramatically.
These changes have led organizations of all sizes to adopt NoSQL database technology.
This article seeks to explain why NoSQL is growing in popularity and when is NoSQL a good choice for building applications.
There is always a large amount of innovation in database technology from both academia and industry. NoSQL came out of industry, driven by the needs of the successful pioneers of both web-scale applications and infrastructure for search and advertising. The demands of these applications could not be served by SQL technology and each of the early companies developed new databases to meet their needs. Most of these were developed in-house and were later published as open source. Some stayed proprietary.
The founders of MongoDB experienced frustration with SQL technology while building out DoubleClick, an early internet advertising firm that is now part of Google. After leaving DoubleClick, Dwight Merriman, Eliot Horowtiz, and Kevin Ryan founded MongoDB in 2007 to create the NoSQL database they always wanted.
At the same time NoSQL databases started appearing, the public cloud was invented and has become an important way that database technology is delivered.
What has happened in a broad sense is that the frustrations that led the early internet companies to create NoSQL databases are now shared by almost every organization.
Digital transformation is the name for the trend toward serving customers using scalable, customizable internet and mobile applications. These applications are often hard to build and evolve rapidly using SQL technology. For this reason, from the mid-2000s to 2020 we have seen a steady rise in the adoption of NoSQL database technology.
The rise of NoSQL is an important event in computer science and in application development because SQL has been so dominant for so long. Many other forms of database technology have come and gone, but few have had the wide adoption of NoSQL.
By understanding the rise in popularity of NoSQL databases, we should be able to shed light on when it makes sense to use NoSQL.
NoSQL covers a lot of different database structures and data models. (See Types of NoSQL Databases for more detail.) This discussion treats them all as a group and identifies the main reasons people are turning to NoSQL databases in general.
NoSQL database technology is usually adopted for one or more of the following reasons:
The pace of development with NoSQL databases can be much faster than with a SQL database.
Because NoSQL databases often allow developers to be in control of the structure of the data, they are a good fit with modern Agile development practices based on sprints, quick iterations, and frequent code pushes. When a developer must ask a SQL database administrator to change the structure of a database and then unload and reload the data, it can slow development down.
The structure of many different forms of data is more easily handled and evolved with a NoSQL database.
NoSQL databases are often better suited to storing and modeling structured, semi-structured, and unstructured data in one database.
NoSQL databases often store data in a form that is similar to the objects used in applications, reducing the need for translation from the form the data is stored in to the form the data takes in the code.
The amount of data in many applications cannot be served affordably by a SQL database.
NoSQL databases were created to handle big data as part of their fundamental architecture. Additional engineering is not required as it is when SQL databases are used to handle web-scale applications. The path to data scalability is straightforward and well understood.
NoSQL databases are often based on a scale-out strategy, which makes scaling to large data volumes much cheaper than when using the scale-up approach the SQL databases take.
The scale of traffic and need for zero downtime cannot be handled by SQL.
The scale-out strategy used by most NoSQL databases provides a clear path to scaling the amount of traffic a database can handle.
Scale-out architectures also provide benefits such as being able to upgrade a database or change its structure with zero downtime.
The scale-out architecture is one of the most affordable ways to handle large volumes of traffic.
New application paradigms can be more easily supported.
The scalability of NoSQL databases allows one database to serve both transactional and analytical workloads from the same database. In SQL databases, usually, a separate data warehouse is used to support analytics.
NoSQL databases were created during the cloud era and have adapted quickly to the automation that is part of the cloud. Deploying databases at scale in a way that supports microservices is often easier with NoSQL databases.
NoSQL databases often have superior integration with real-time streaming technologies.
NoSQL databases support polyglot persistence, the practice of mixing various types of NoSQL databases depending on the needs of particular segments of an application. For example, some applications store most of their data in a document database like MongoDB, but supplement that with a graph database to capture inherent connections between people or products.
If you have these requirements, then you should consider adopting a NoSQL database like MongoDB.
Companies of all sizes, from small startups to established Fortune 100 companies, build leading-edge applications on MongoDB.
The easiest way to get started for free with the most popular NoSQL database, MongoDB, is by trying out MongoDB Atlas, a fully managed version of MongoDB that runs on all the popular cloud platforms.
It is important to point out that while NoSQL has shown rapid growth, SQL databases are a great choice for many applications. There is widespread knowledge about SQL, a massive selection of products and supporting tools, and a huge community of trained developers and consultants.
The discussion in Comparing MongoDB vs. PostgreSQL provides a more detailed comparison of the strengths of SQL vs. MongoDB.
The key point of this article is that even in the face of SQL’s many strengths, NoSQL databases are making rapid progress for the reasons stated above.