Bamakus Montreuil on 18 Apr 2024 11:07:54
Improve DATAFLOW GEN2 around KIMBALL Patterns [1# SNAPSHOTS]
- Offer something similar on DBT cloud to deal with rows snapshots (PK , values , Start_date , End_date)
- Based a primary key and columns , we need to track changes from Source (security==> SQL instructions to occur ==> only if datetime a business column value on source is >= targeted row one)
- "BIG revolution" , here on power query as we are on UPSERT or MERGE + INSERT , according to UPDATE an END_DATE system column value on current rows
and create a new ROW based new values ... similar primary KEY (PK , values , Start_date system , End_date system )
Administrator on 30 Sep 2024 21:30:04
Today you can leverage this article to implement your own Slowly Changing Dimension technique:
https://learn.microsoft.com/en-us/fabric/data-factory/slowly-changing-dimension-type-two
If perhaps the article covers the request, let us know and share your feedback and what gaps it might have.
- Comments (1)
RE: Improve DATAFLOW GEN2 around KIMBALL Patterns [1# SNAPSHOTS]
Also ADD a surrogate key ;>) Will help to manage SLOW CHANGE DIMENSION on the semantic model Snapshot table : surrogate key + business key + rows columns ,where we need to track changes from Source Merged via LEFT JOIN to a full refreshed table