Combining all your data from disparate sources is the first step in turning it into business value, according to Informatica. The company recently held a special webinar focused on leading trends in modern data architecture and integration.
Part of the “Back to Basics Webinars: Data Integration” series, Makesh Renganathan, principal product manager, R&D cloud, Informatica, and John O'Brien, principal advisor and CEO, Radiant Advisors, discussed what’s next in XOps (DataOps, MLOps, etc.), the impact of data fabric, autonomous data integration and serverless processing, and more.
Data lakehouse architecture via Databricks and others is thriving. This next generation technology offers an open-standard format for data, supports multiple cloud compute engines, maintains consistency among users and more, O’Brien explained.
“We keep hearing from our customers about data fabric and data mesh,” Renganathan said.
Data fabric and data mesh democratizes data and offers consistent data services across the enterprise, he explained. This improves productivity and governance.
As an industry analyst watching this space, O’Brien noted that he’s seeing the same interest in data fabric and data mesh.
“As all enterprises are dealing with more data and more integration, the architecture is turning into a data mesh strategy,” O’Brien said. “This is as a result of what companies are dealing with.”
Data warehouse or data lake architecture supports data catalogs and data governance. This type of architecture supports data discovery, lineage, and glossaries.
Autonomous data integration is another top integration trend that O’Brien and Renganathan are seeing this year. Active metadata-driven intelligent autonomous data management “understands” data lineage, improve data quality, and enables self-service data integration.
“[People are] moving to data pipelines and breaking things down to understand the difference between ingestion and processing patterns,” O’Brien said.
ETL and ELT processing are additional data processing approaches that companies can utilize to transform data. Both have its benefits and need to be considered for different situations, O’Brien noted.
Event streaming architecture increases the opportunity for ML and AI benefits. This type of platform can reuse data pipeline segments to increase agility and quality.
DataOps is a journey for agile delivery teams to increase analytic output for customers, O’Brien and Renganathan noted. Continued talent shortages drive tool/platform adoption for faster development and training, and MLOps is needed as ML models can be trained and deployed faster throughout the enterprise.
An archived on-demand replay of this webinar is available here.