The old Chinese expression “May you live in interesting times” has never been more applicable in my lifetime than in 2020 and 2021. The global pandemic combined periods of great anxiety with long stretches of mind-numbing lockdown boredom. But it certainly kept our attention!
During this period, the world of databases moved slowly but consistently in a number of interesting directions. Perhaps looking at these trends can help us anticipate what might be coming next in the world of database systems.
The Growth of Cloud-Based Solutions
Before the pandemic, we had already reached a cloud database “tipping point,” with most new workloads likely to be deployed on the cloud, and with all major database vendors focusing on cloud-oriented business models. The pandemic served to accelerate all things cloud due to the relative difficulty of maintaining on-premise systems during lockdowns. So during the last 2 years, we’ve seen all significant vendors focus an increasing share of their efforts on developing and marketing cloud-based solutions.
You might think that genuinely cloud-native databases would be the primary beneficiary of this accelerated cloud adoption. However, while it’s true that cloud-native systems such as Amazon DynamoDB, Microsoft Cosmos, and Google Spanner are showing healthy adoption, there is just as much energy behind the cloud versions of on-premise systems such as MongoDB, Oracle, SQL Server, and others.
Key Trends
Until recently, “on-premise in the cloud” offerings were little more than the same on-premise software running in a cloud hosted VM. However, they are increasingly transforming into truly elastic, co-tenanted, pay-as-you-go offerings. The “serverless” models adopted by MongoDB, CockroachDB, and DataStax are examples of this transformation.
Another trend that became clear over the past few years is the resurgence of the SQL language. The “NoSQL” movement—which emerged in 2009 and is therefore almost a teenager now—led to a plethora of database systems that omitted the SQL language in favor of non-relational APIs. Some of these—MongoDB in particular—have very healthy adoption. However, the SQL-based systems have remained dominant, while new SQL-based systems such as Snowflake and CockroachDB are some of the fastest-growing new databases.
The Rise of ‘Source Available’ Licenses
Open source has become a controversial topic in database systems over the past few years. To prevent mega-cloud vendors from exploiting their systems, some vendors have adopted “source available” licenses that forbid offering the database as a service. These new licenses are not truly open source, and, to many in the open source community, they are a step backward.
Regardless of what one might think about these new licenses, it’s definitely the case that open source or source available databases are strong and growing in market share. MySQL remains incredibly popular, and PostgreSQL adoption is rapidly increasing, as is MongoDB (an “source available” system). By most measures, three of the top five database systems are currently open source.
What’s Ahead
Predicting that open source, cloud-based systems, mostly incorporating SQL, will dominate over the next few years seems a safe prediction. What else might emerge? There doesn’t seem to be a significant paradigm shift in database systems on the horizon. Quantum computing, blockchain, edge computing, and revolutions in storage technology all stand to influence the next generation of database systems.
But for the time being, the increasing prevalence of open or source-available databases in the cloud will probably be the dominant trend.