Data platforms, in all their forms and configurations, are becoming more real-time, as well as highly intelligent and highly scalable. But they need to be able to keep up with the burgeoning wave of AI and related intelligence applications that are being demanded by organizations. Can data managers keep up as well?
Adoption of data streaming for agentic AI will reach beyond 60% by 2028, from less than 15% in 2025, according to a recent study from Gartner. This calls for data systems that can support more rapid decision making and automation. “These technologies are responding to live conditions in real time,” says Pat McClain, senior director of product management for Skylar Advisor at ScienceLogic. This raises questions of AI safety, as lags or shortfalls in underlying data make it riskier to grant AI agents or systems autonomy.
Is everyone ready for this new agentic-driven world? Not quite. A majority of companies (53%) say their data foundations are only “somewhat ready” for AI, according to a report published by MIT Technology Review Insights in partnership with Databricks. Only 22% of respondents say they are “very ready” for the AI onslaught.
We may be witnessing what is essentially “the collapse of the modern data stack,” warns Karthik Ramakrishnan, VP of engineering for Google Cloud’s agentic data platform. “It’s failing the stress test of agentic AI,” he says. The idea that if data is clean and structured enough for an executive dashboard, it is ready for AI “is a mistaken belief.”
Importantly, the time to act is now. “Data executives have to stop thinking they can fix the data platform at a later date,” advises Phillip Merrick, CEO and co-founder of pgEdge. “Teams need to start making deployment decisions well in advance of a security review or a stalled AI project.”
THE ISSUES WITH DATA PLATFORMS
There are a number of difficulties that stand in the way of enabling what today’s and tomorrow’s data platforms require to meet the agentic AI challenge and all related initiatives. Let’s take a look at some of them.
Gaps in data quality, access, trust, and lineage lead to AI failures. Most enterprises (52%) report half or fewer of their AI initiatives succeeding, according to a survey conducted by Unisphere Research and Radiant Advisors, which was sponsored by Quest. The root causes include data and the gaps in quality, access, trust, and lineage that continue to limit outcomes and prevent AI from scaling.
Business context is still locked away in traditional systems. What’s missing from this equation is “decades of business context and intuition” that humans have already built into their existing systems and dashboards, Ramakrishnan says. “AI agents don’t have that inference—they only have the raw data.”
Plumbing is still batch mode. “In over 14 years building data platforms at large enterprises, I’ve seen the same pattern,” states Abhilash Rao Mesala, senior data engineer with Meta. “AI initiatives get bolted onto architectures originally designed for overnight batch reporting. Your plumbing is still nightly ETL, siloed warehouses, and schemas that break the moment upstream systems change.”
Data is still in silos across enterprises. “When data is spread across different systems and its provenance or traceability isn’t clear, it becomes harder to know whether AI outputs can be trusted,” says Scott Gnau, SVP of data platforms at InterSystems.
Many organizations maintain separate systems for operational data, analytics, and AI. “That’s much more consequential when an agent needs current context to approve a transaction, update an account, or execute a workflow in real time,” states Adam Luciano, VP of product management for MariaDB. “As AI becomes more autonomous, access to current, trusted data becomes more important, not less.”
AI agents behave much differently than traditional systems. The pace of change in AI is blindingly fast. “Models, agent frameworks, and other tools are evolving quickly, creating problems if an organization’s architecture is too tightly tied to one technology or approach,” says Gnau.
Current data environments are not ready for the coming AI deluge, but not for the reasons one might think. “The technology isn’t what’s missing,” says Mike Freedman, co-founder and CTO of Tiger Data. “You provisioned a database, tuned it, and it ran the same workload for years. AI breaks that assumption. Agents don’t behave like human developers. A human might spin up a staging environment once a week. An agent wants a fresh one every few minutes.”
Architectural fragmentation hampers free data movement. Think about the way enterprise architectures tend to evolve, advises Luciano. The process is gradual, “with different systems added for the transactions, analytics, performances, and now AI. That means data is constantly being copied, moved, and synchronized just to make it available somewhere else.”
Data needs to be understood before redesigning around it. Today’s organizations are overrun with large volumes of files, documents, messages, images, and other content, which are often not classified or governed, says Rick Vanover, VP of product strategy at Veeam. “Much of that data is redundant, outdated, trivial, or sensitive in ways the business may not fully understand.” If the agent is drawing from this kind of data, it will give the business faster answers, “but not necessarily better or safer ones,” he adds. “They need to know what data they have, whether it can be trusted, how it is protected, and how quickly it can be restored when something goes wrong.”
A data platform needs to provide “a comprehensive overview over what data an organization has, how it is being used, who owns it, what value it delivers to the business, and what level of performance it requires,” says Greg Knieriemen, VP of marketing and senior technology advocate at Quantum. “Without that insight, valuable high-performance resources can be consumed by inactive, duplicated, or misplaced data.”
THE ROAD AHEAD FOR DATA PLATFORMS
There is healthy demand for integrated AI capabilities in data management and analytics solutions. About three-fourths of enterprises (74%) are looking for “AI systems that require deeper integration with existing data infrastructure and workflows,” according to research from IDC. Interestingly, 70% of respondents cite needing “AI platforms that are more tailored and integrated into their specific data and analytics systems.” Lastly, 73% of respondents say they are using or integrating generative AI capabilities into analytics solutions, but only 29% state that they’re currently these capabilities.
The basics of data management 101 still matter a lot. It’s important to assure that data contracts, lineage, and observability are all up-to-speed before investing in AI models. “AI on top of untrusted data is a liability,” says Mesala. Be selective about real-time initiatives, he urges. “Identify the handful of use cases where minutes actually matter, and let the rest stay batch. Treat AI as a user of your platform, not a feature of it.” The most successful AI initiatives won’t be the ones employing the flashiest models, “but the ones whose data is clean enough to trust them.”
Data first—versus AI first—is the best policy, Gnau agrees. “Make sure the data is clean, connected, and governed. Understand where it comes from, how it’s being transformed, and who has access to it. Also, assess how quickly it needs to be updated for the business problem you’re trying to solve and whether your current systems can meet that requirement.”
Data modernization and AI adoption need to be conducted in unison. Audit what data your agents need to consume, then work backward to close the gaps. The goal is to achieve an “AI-ready foundation built around governed data products,” says Ashwin Patil, principal and AI and data practice leader at Deloitte. This also should incorporate “a semantic layer that intelligent agents can trust and consume, [as well as] real-time pipelines.”
Architecture needs to be front and center from the start. Too often, data pipelines are duct-taped together. “A data architecture needs to be AI-ready,” Ramakrishnan advises. This can be accomplished through a concerted set of strategies, including batch processing and real-time, zero-copy architectures, which cut down on delays to data movement, he says.
It is vital to make every effort to reduce unnecessary data movement and “[bring] AI closer to trusted operational data,” states Luciano. Rather than creating another AI silo, think in terms of having operational, analytical, and agentic workloads sharing a common data foundation.”
Data architectures have to support context. AI transactions need to be structured around the context of the business problems and opportunities at hand. Next-generation platforms now include automated context engines that support semantic layers “that continuously infer business meaning and map relationships,” says Ramakrishnan. Such a structure is also capable of extracting unstructured dark data. This adds the power of business logic to agentic AI transactions.
Organizations need an end-to-end view across all of their data platforms and environments. AI models, agents, infrastructure, applications, and business services require “enough context to understand how those pieces connect,” says McClain. “That context provides the evidence needed to explain decisions, verify outcomes, and establish effective guardrails. We need to understand what informed its actions, what it did, and whether the outcome matched the intent.”
Workloads should be understood before redesigning around them. Questions that need to be asked include these: “ ‘What are your write rates?’ ‘How much of your data gets queried in the first hour versus the first year?’ and ‘Which queries actually matter to the business?’ ” Freedman illustrates. “Correctness, predictable performance, and access discipline matter more now, not less. The fundamentals didn’t go away. The cost of ignoring them went up.”
Some legacy platforms are perfect for the AI job. There are already some components in place that can be adapted to AI requirements. For example, “Postgres is about 40 years old and wasn’t designed with AI in mind,” Freedman points out. “But it was designed for extensibility, and that turns out to matter more. Vector search, time-series analytics, and forkable storage can all be added to Postgres without replacing what already works.”
As a result, for many organizations, “the data layer is closer to ready than most people assume,” Freedman continues. “The gap is in the systems around it. Teams that evolve that infrastructure around a database they trust will get there faster than teams that swap in something new every time a workload changes.”
Data security and AI governance go hand in hand. When configuring an architecture or system for AI, “start by asking what you could prove to a board or regulator today if an AI agent made a bad decision on your data, and work backward from there,” says Vanover. At the same time, of course, a successful AI effort will have plenty of business owners stepping forward to take the credit. “Focus on the quality of data for all initiatives, including AI.”
Portability is the key to the future. In the era of fast-moving AI and agentic AI, workloads have to be capable of being moved on short notice. That’s why it’s important to build flexibility into a data architecture, Gnau advises. “Organizations need to be able to test new models, swap components, and adapt their systems without having to rebuild everything. The goal is to make AI part of the existing data environment and business workflow, rather than treating it as a separate project.”
Preserve your options, says Jon Lucas, director and co-founder of Hyve. “Nobody can say with any certainty what the AI landscape will look like several years from now. The model you favor today might not be the one you want tomorrow. The same goes for approaches to platforms, hardware, and infrastructure. I’d be very wary of designing an AI strategy that depends too heavily on a single technology, platform, or way of doing things.” n