Data-driven organizations are adopting augmented data management to deal with complexity, keep the ability to innovate, and iterate quickly. CTOs and CIOs of successful companies are facing one major challenge: the skyrocketing complexity of their data stack (data pipelines). Combined with a shortage of engineering talent, it limits their ability to cope with the fast pace of changes, negatively impacts innovation initiatives, increases the risk of data incidents, causes reputation issues, and leads to non-compliance with regulatory requirements.
“Active metadata” and “data lineage” are key cornerstones of any augmented data management initiative. While data lineage started as a simple way to describe the “journey of data”, it has now evolved and become the main tool for organizations to map, understand, and gain insights into their data pipelines.
The Ultimate Guide to Data Lineage in 2022 presents a vast array of data lineage perspectives and outlines several approaches to its discovery, each with its advantages and disadvantages. The guide also puts data lineage into the broader perspective of current data management trends and provides actionable insights into the practices of successful data-driven institutions.