This year is an expansive one for the database ecosystems that have evolved around the major platforms. Artificial intelligence (AI), machine learning, the Internet of Things (IoT), and cloud computing are now mainstream offerings seen within the constellations of database vendors, partners, and integrators.
One thing is certain: This is a new day for relational database management systems, their vendors, and the huge ecosystem that has been built up over the years to support them. “All the RDBMS players already extend themselves beyond traditional relational-model databases to support different unstructured data and columnar formats,” said Soumendra Mohanty, executive VP and chief data analytics officer at LTI. In addition, he added, they also provide database as a service and data as a service. “While still evolving, clients are increasingly exploring these options and moving away from deploying just a database. They are now thinking about how and where data can be stored and consumed as a service irrespective of the underlying database. The major RDBMS players have responded and offer solutions addressing these emerging trends, albeit with limited feature sets.”
AI and Machine Learning Shape the Market
Perhaps the trend that is generating the most buzz across database markets this year is the use of AI and machine learning—which is heavily data-dependent, and thus, directly impacting discussions and acquisition choices. “Most companies want to do something with AI and machine learning, but they have zero idea what that something is,” said Robert Reeves, co-founder and CTO of Datical. “However, database administrators need to be aware that it is coming. If they want to be strategically aligned with the C-suite, they need to be prepared to deliver data locked in RDBMS providers to a data platform that supports machine learning.”
For his part, Mohanty sees “incredible excitement” about AI and machine learning from clients in various sectors. “They are eager to introduce AI-driven solutions, and many are already on the way to full deployments. As with any AI solution, the system learns and improves continuously, and some clients take a cautionary approach. Because the accuracy of AI solutions keeps improving with more learning, customers want to ensure they take advantage of the most current version for optimal performance and ROI.”
Many enterprises are already making progress with AI and machine learning, according to Monte Zweben, CEO of Splice Machine. “Companies have completely operationalized AI now,” he said. “Machine learning models are routinely used by companies to personalize interactions.”
Nonetheless, this shift to AI and machine learning brings a whole host of problems as well, Reeves pointed out. One is securing and handling personally identifiable information and another concerns the data source and metadata around it in the data lake. “A big danger is delivering two data dumps when you normally deliver one. Thus, the results based on that double data will be skewed and lead the company to make bad decisions.” The challenge to data professionals, then, said Reeves, is to be able to “identify current tasks they handle that can be automated or eliminated so they have the resources to be ready when the C-suite does need that machine learning infrastructure.”
Data managers need to exercise caution as they consider how best to implement AI in their organizations. “Some technology companies are claiming to have AI embedded in their applications, but their claim is based on a much older, broader definition of the term previously in use,” said HK Bain, CEO of Digitech Systems. He added that AI falls into one of three primary areas: natural language processing, robotics, and machine learning. “AI engines can explore an exponentially large number of inputs to arrive at a previously unknown output through the recognition of patterns and categories of data. The engines even learn and adapt; they are not taught.”
Accordingly, these are the early days, other industry experts caution. “An extremely small number—less than 10%—of non-digital natives are using AI and machine learning effectively,” said Suketu Gandhi, a partner in the Digital Transformation practice at A.T. Kearney. “Fifty percent of enterprises are piloting and 40% are evaluating,” he related. “What that means is productizing of AI and machine learning is in a very small number of companies.” Companies leading the way are in the financial services, telecom, and retail sectors, he observed.
“Enterprises are still in the very early days of getting value from AI and machine learning,” agreed Joe Pasqua, EVP for MarkLogic, who predicted that adoption of these technologies “is going to happen in phases.” An example of AI at work today is the use of chatbots for customer support, he added. Overall, AI adoption “has been slower than expected because of data integration and quality issues. You can’t make informed decisions about customers if the data about them is in 10 different data silos and your machine learning algorithm only has access to one. Most enterprises are not there yet.”
The Value of IoT
IoT—until recently part of projects seen at the fringes of enterprises—is also growing in adoption, industry observers stated. This is another data-rich technology that demands a range of new solutions and capabilities from today’s databases. “Many of our customer use cases are architecturally supported by an IoT design,” said Robin Schumacher, senior vice president at DataStax. “General-purpose use cases such as inventory management, asset monitoring, supply chain management, logistics, and others are IoT-based. Portions of customer experience systems such as Customer 360 and personalization and recommendation can also be IoT in nature.”
IoT is “another type of network within which we can create applications that blend what people do within that network,” said Jeff Morris, head of product marketing at Neo4j. At the same time, “it’s not enough for businesses to just accumulate data—they also have to be able to act on it,” Yu Xu, CEO of TigerGraph, pointed out. “Today, most IoT users—businesses, governments—are collecting the data but have challenges making sense of it to drive value. Imagine a city that is better able to program traffic lights to improve traffic flow based on in-the-moment feedback after an accident. Saving 30 minutes in traffic is quite meaningful.”
Vendors well beyond the database and IT space are adapting and upgrading their offerings to become part of this new interconnected world. “Engineered systems are now completely instrumented with sensors,” said Zweben. “In the supply chain, vehicles, pallets, cartons, and products themselves have radio tags and other location sensors that report where inventory is at all times.” Energy grids, telecommunication networks, and oil and gas rigs have equipment reporting their status across many metrics in real time. This creates an invaluable flow of data to predict outages before they happen so that service personnel can be proactively dispatched to install spares, he noted.
Clouds Keep Forming
Much of the new capabilities emerging within data enterprises are made possible through a single technology initiative: the cloud. “Cloud has impacted enterprises in significant ways through improved business agility and operations management, effective utilization of resources, and heightened data security/privacy,” said Prasad Venkatachar, senior product manager of database solutions for Lenovo Data Center Group. “Cloud is becoming more ubiquitous with enterprise users expecting self-service as part of many enterprise business processes and workflows.” Several on-premise applications and middleware platforms are also now being offered on internal, private clouds.
The cloud is reshaping enterprise data strategies, Kevin Petrie, senior director product marketing for Attunity, agreed. “Cloud-based data warehouses, data lakes, and RDBMS [relational database management system] platforms have become a mainstream component of enterprise environments. Most organizations are improving efficiency by moving production and analytics workloads to cloud offerings such as AWS and Azure.” The data lake is an especially good fit with the cloud, given that both technologies are “scalable, efficient, and economic,” Petrie noted.
“In our enterprise customer base, I can think of only one who does not have a strategic plan for utilizing cloud in their infrastructure,” said Schumacher. “Naturally, some are much further along than others. Some are only in the stage of not actively pursuing more enterprise-owned data centers while others are already operating full-blown hybrid-cloud infrastructures that support first-line digital business applications.”
At the same time, the rise of the cloud also creates new data integration requirements. “Enterprises need to stay flexible, moving data and workloads between cloud or hybrid platforms based on their learning curve and changing business requirements,” said Petrie. “It is becoming fairly common for enterprises to change their cloud strategy and data sources in the middle of a project, so they may use the most agile cloud solution.”
With this enhanced flexibility and access to greater scale and capabilities, “expectations are becoming higher and higher for the enterprise user,” said Reeves. “Data professionals need to deliver data to consumers as quickly as they can create cloud compute instances,” he said. “Simply telling internal customers that a request will take days or weeks to complete is no longer acceptable. SLAs [service level agreements] need to change for the database in the cloud due to the comparison to other parts of the application stack that the cloud provides.”
Are Vendors Adapting Fast Enough?
How are major database vendors managing this new world? “Digital transformation is putting pressure on database vendors as customers are seeking to do more with their data assets,” said Xu. “These vendors have focused on collect-first strategies. While this approach has its benefits, they break down in the new world of personalization, IoT, and fraud [detecton] where real-time analytics is essential. This is pushing them to not only rethink their data use approach but data access.”
Leading RDBMS vendors have largely taken their eyes off the ball in order to focus on their apps or to lower the cost of the database in the cloud, or their AI technology, rather than advancing the database story, according to Pasqua. Moreover, he emphasized, the major RDBMS players are not dealing with “the fundamental challenges of a model that requires rigid, inflexible structure, in a world that requires the opposite.”
At the same time, the huge investments the vendors are making in next-generation database technologies to move into this new world are not escaping anyone’s attention. “In varying ways, each is investing in cloud, IoT, and AI opportunities,” said Petrie.
Microsoft, in particular, gets called out for its ongoing commitment to the cloud model. “Microsoft is making great strides with its Azure platform and is challenging other market leaders,” Petrie observed. “We find that many enterprises are moving SQL Server data and workloads to Azure to efficiently address IoT, AI, and machine learning use cases.” Microsoft “is the most innovative with a massive focus on cloud-based distributed systems on Azure,” Zweben added.
“Microsoft is the only RDBMS provider that gets it,” Reeves acknowledged. “You can run any database you like in Azure. With their ‘cloud first’ strategy, they still make money even if an Azure customer chooses a database other than SQL Server.”
The changes shaping the data market may call for no less than a major redesign of the relational database model. “Modern hybrid-cloud applications have absolutely mandated the retooling and redesign of the database,” said Schumacher. “Because legacy RDBMS designs are fundamentally ill-equipped to support a distributed application that needs to seamlessly operate in a hybrid and/or multi-cloud fashion, you see some of the RDBMS vendors delivering solutions that address parts of this equation,” Schumacher noted. “Vendors have so far taken the head-in-the-sand approach where recognizing the real database requirements of hybrid-cloud apps is concerned. Beyond cloud, you see the RDBMS vendors attempting to support multi-model in their legacy engines, with some having more success than others, although all struggle somewhat because of the rigidness found in the relational model.”