Data needs to be understood by everyone, said Warden. “Getting to this point requires an architecture that provides for high data understanding regardless of the data literacy of an organization. A business analyst with access to data fields with names like ‘MC1,’ ‘Temp,’ and ‘LFRWQ’ will likely not know how to use the data in these fields for any meaningful analysis.”
Data architectures that support “distributed data management and analytics in real time are proving most responsive to business needs,” said Scott Gnau, vice president of data platforms at InterSystems. “Legacy databases were simply not built to be distributed, and, no matter how much they are modified, they cannot escape their stovepipe past.” Now, architectures that “bring the composable stack and distributed data together for actionable real-time insights are vital to meeting the data demands of businesses,” said Gnau.
A number of industry leaders point to smart data fabrics or meshes that make data resources and insights available on an enterprise scale. The approach presents a “reference architecture that provides the capabilities needed to discover, connect, integrate, transform, analyze, manage, utilize, and store data assets,” said Gnau. This helps reduce complexity from previous approaches such as data lakes.
“It’s essential to place an emphasis on the quality of data that is collected and fed into powerful tools such as AI and machine learning,” Gnau added. “The speed of today’s digital organizations is forcing AI and machine learning to be embedded into modern data architectures, accelerating application development, data analysis, and decision making. AI and machine learning can’t be an afterthought.”
Each data architecture has a purpose and fits for specific IT requirements. “If a narrow-functional-scope IT system has a limited user base, it could still be supported by a data warehouse data architecture,” said Peter Burggraaff, MACH Alliance ambassador and partner and director with Boston Consulting Group. “In organizations with modern microservices technology architectures and extensive core—often legacy—IT landscapes, it’s beneficial to develop a mesh data architecture with clearly defined data domains. Digital products will consume and feed relevant data domains, which are owned and managed by dedicated cross-functional data product teams.”
“Distributed data is the future,” said Carr. “An architecture focused on a single point for data integration or analytics—whether that is in a traditional data center, the cloud, or out at the edge—will only take a business so far. The key to future business agility will be the ability to collect, process, and analyze data at the point of action based on data from all other relevant points. This requires distributed data architectures for integration, management, and analytics.”
DESIGNING A DATA ARCHITECTURE
Determining the right tools and technologies can be challenging, said Grant Fritchey, product advocate with Redgate Software. “With the incredible growth in platforms and technologies for managing data, planning the architecture is frequently a factor of playing catch-up rather than leading the parade. Because data could be stored locally, or on one of several cloud platforms, or even on more than one, a single set of tools for tracking and maintaining the architecture doesn’t really exist. Instead, it’s down to the data management team doing some hard work to create documents that enable them to track what’s happening.”