Skip to main content
Microsoft Idea

Power BI

Under Review

Dataflow, load to an existing Datamart

Vote (45) Share
Mimoune Djouallah's profile image

Mimoune Djouallah on 29 May 2022 07:39:00

Currently when you create a dataflow, it give you the option to either create a new dataflow or a new datamart, which is a great option for a new user as you get a full Online experience from Data Prep, Modelling to Final reports, and a good option to start a new standalone report

but for existing users with a bit of experience, we prefer another Workflow, give us an option to load a dataflow to an existing Datamart, we want to use a Datamart as a central Database to store a lot of tables for a specific Business Topic, Currently the only way to do it is to create a separate dataflow and load data from there, the problem is, if you have a Datamart with 100 Tables, and only one table has changed, you need to reload all the 99 tables, that's not a very good usage of PowerBI resources Neither for customers nor Microsoft.

When Loading Data from Dataflow to Datamart, we prefer those option in order of Priority.

  • Overwrite
  • Incremental refresh
  • append
  • Upsert using PK etc

This is Microsoft opportunity to redefined ETL for mass market, and blur the line between Enterprise ETL and Self service Data preparation.

Comments (3)
Mimoune Djouallah's profile image Profile Picture

Mimoune Djouallah on 09 Aug 2022 05:21:41

RE: Dataflow, load to an existing Datamart

@ Steven Fullerboth the idea is to decouple data from datamart for maximum flexibility, now if you want to refresh a table, you have to refresh the whole model, that does not work in any serious scenario

Mimoune Djouallah's profile image Profile Picture

Steven Fuller on 13 Jun 2022 20:38:30

RE: Dataflow, load to an existing Datamart

Mimoune, are you referring to previously existing Dataflows as well? or just new dataflows? Personally I think what you're suggesting would be great, especially for an existing dataflow that someone wants to use in a datamart. I'm sure I'm not the only one who already has many active dataflows that I would love to be able to work with in a datamart, but don't really want to duplicate all of them just for the datamart capabilities.

Mimoune Djouallah's profile image Profile Picture

James Fancke on 29 May 2022 08:14:22

RE: Dataflow, load to an existing Datamart

Allowing dataflows to choose their destination simplifies the story for users:a dataflow is a way to get data from source systems into a storage destination. you can pick either basic datalake storage (CSVs + manifest/typing) or richer relational database storage (Azure SQL DB).