What is Data Lineage?

I think Data Lineage can be a very valuable thing when it comes to doing Data Governance but I think the term perhaps makes it sound more scary than it is! So, what is Data Lineage? Watch my latest Ask the Data Governance Coach video to find out.

Want more support through your Data Governance journey?

[NissimOhayon]

Good points Nicola. I would point out however that the use-case for data lineage for the data governance team is different from the use-case for the data engineering team. They need lineage for different purposes. The governance related need is primarily to follow the path to know where their data comes from, and perhaps something about how the transformations may be impacting the final output on the reporting layer.

On the other hand, the data developers and engineers use lineage to be sure that they know what will be impacted by the changes they make to the data flows.

As such, the technical lineage you described as being far to granular will be difficult for a business users trying to get the basics from the lineage map, but critical to the engineers to ensure that the data flows as designed, and thereby ensuring top data quality.

With all that said, a good data lineage solution provides both ways to consume the information, and in a format that is digestible for the business users, while technically sound enough for the data engineers.

Of course, this is coming from a vendor of Data Lineage Automation (Octopai.com), but we also have a data catalog that is designed for the data governance types too. As such, our approach is to provide both perspectives in a way that will not only be easily understood, but also double as a way for the two stakeholders (IT and Business) to collaborate around the use of the data elements in the landscape.

Care to learn more? Ping us on octopai.com.