Data integration technologies, capabilities, and methodologies are evolving. It can be difficult to figure out what solutions or approaches may be best for the business and its digital transformation.
Informatica recently held a special webinar titled “Data Integration Trends for 2023,” with Makesh Renganathan, director product management, Informatica, and John O'Brien, principal advisor and CEO, Radiant Advisors, to discuss data integration trends to improve data management.
Looking back at 2022, O’Brien emphasized that several trends would continue this year including the evolution of data lakehouses, the rise of data mesh/fabric, continued adoption and movement to the cloud, and more.
“Data integration, architectural patterns are being rethought due to cloud data,” O’Brien said. “Companies want to do things better and faster.”
Data management is a large umbrella of key use cases of data capabilities or business needs, O’Brien stressed. There are multiple ways to do this, but it doesn’t change the fact that enterprises need data management.
There is more of a need to access to data in a quicker way, opening up a path to data democratization, Renganathan noted.
Once again ETL and ELT processing are being hotly debated in organizations. ETL and ELT are additional data processing approaches that companies can utilize to transform data. Both have its benefits and need to be considered for different situations, O’Brien said.
The leading buzz this year is mostly around lakehouses as it continues this long-term path of separation between compute platforms and services.
“It’s really the architecture pattern of the future,” O’Brien said.
The continuity of workflows is the number one challenge of modernizing data platforms. The main drivers in interest toward data lakehouses are data science workloads, whereas data warehouses and BI workloads tend to favor more of a two-tiered warehouse/data lake.
The concept of a data fabric/mesh continues to be an interesting topic, O’Brien and Renganathan agreed. Companies need to deliver data faster and not wait months for the completion of projects. Creating a data fabric or data mesh enables people to be more data driven.
“Data mesh isn’t something you buy, it’s something you build,” O’Brien said.
Open architecture is another major theme for 2023, O’Brien and Renganathan said. Open standards from both the architecture side and the integration side, companies are saying they want to go faster, empower more teams, and have more portability and interoperability to leverage other people’s works via API or a services approach to data integration. There is an architecturally growing separation between ingestion and processing/transformation.
“There is a new paradigm and it’s hard to transition to automate and orchestrate,” O’Brien said. “Keep in mind we are making an important shift into open architecture. As companies continue down that path, the benefits are worth the journey.”
An archived on-demand replay of this webinar is available here.