Daragh Fitzpatrick on 29 Jan 2015 22:39:47
When working with data that needs to be shared, it is of utmost importance to understand what the semantic definition of it is (i.e., the proper usage of it) and also where it came from (both sources and transformations). This would be published as a data dictionary with lineage information. There are many other aspects to meta- and master-data, and this is one of the biggest barriers to PowerBI becoming adopted as an enterprise-level tool.
Eyal Carmel (administrator) on 12 Jan 2023 09:33:45
There is the lineage view in which the lineage information is avaialbe - Introducing Power BI lineage view | Microsoft Power BI Blog | Microsoft Power BI
The metadata is exposed through the Scanner API or in Purview - Announcing Scanner API (Admin REST APIs) enhancements to include dataset tables, columns, measures, DAX expressions, and mashup queries | Microsoft Power BI Blog | Microsoft Power BI, , Use Power BI with Azure Purview to achieve better data governance and discovery | Microsoft Power BI Blog | Microsoft Power BI
If there is a need to expose more granualr information in one or all of these capabilities please do open a new idea/s with the specific asks so we address them
- Comments (6)
RE: stronger metadata & lineage support
thanks @billanton
RE: stronger metadata & lineage support
Powerbi API needs much stronger lineage capability. Currently it is only possible to find, for a report, the server that the dataset is using. It's not a preety process to get....
So that means, we can get server level lineage. Not database, not table and certainly not column lineage. This is very poor. This means that anytime we do an impact analysis, we have to open all the datasets to see if the table/column etc is used. At minimum, the SQL executed should be exposed (still means we'd need to parse the sql but its a step closer). What would be idea is knowing the sql fields and the mapping to other queries measures etc. (ie column level lineage to the dataset). If we could then also get what measures are used in the report/dashboard etc we'd have relatively full tracability. Been able for 3rd party tools such as Informatica EDC / Alation to be able to be able to download this metadata is vital. Impact analysis is just 1 use case. Without column lineage, how do you know which reports you need to know have been view/printed/download in the case of GDR/CCP/breach etc (if you know the columns with identifiable info, you should be able to trace to where its used.)
RE: stronger metadata & lineage support
Fundamental to users experiencing the data in the best way and helping them know and trust what they are using.
RE: stronger metadata & lineage support
i believe this is being solved by Data Catalog... see this PBI Idea: https://ideas.powerbi.com/forums/265200-power-bi-ideas/suggestions/6864081-data-catalog-integration-in-powerbi-designer
RE: stronger metadata & lineage support
This would be a great step, but besides that. What about your business rules / transformations that you apply to your data? When creating an new document there is no way to re-use those transformations. Ideal would be if you could store the result of a query/transformation and use that in multiple PIBX files. (Also more efficient to your source db-system)
RE: stronger metadata & lineage support
Bump. Ability to extract metadata for a tool like Adaptive, MetaCenter, or Rochade is a showstopper in some environments. I'm also interested in the plans in this area. It's supported by other Microsoft platforms, like SSRS.