41

Currently when you create a dataflow, it give you the option to either create a new dataflow or a new datamart, which is a great option for a new user as you get a full Online experience from Data Prep, Modelling to Final reports, and a good option to start a new standalone report


but for existing users with a bit of experience, we prefer another Workflow, give us an option to load a dataflow to an existing Datamart, we want to use a Datamart as a central Database to store a lot of tables for a specific Business Topic, Currently the only way to do it is to create a separate dataflow and load data from there, the problem is, if you have a Datamart with 100 Tables, and only one table has changed, you need to reload all the 99 tables, that's not a very good usage of PowerBI resources Neither for customers nor Microsoft.


When Loading Data from Dataflow to Datamart, we prefer those option in order of Priority.


  • Overwrite
  • Incremental refresh
  • append
  • Upsert using PK etc


This is Microsoft opportunity to redefined ETL for mass market, and blur the line between Enterprise ETL and Self service Data preparation.

STATUS DETAILS
Under Review

Comments

M

@ Steven Fuller


both the idea is to decouple data from datamart for maximum flexibility, now if you want to refresh a table, you have to refresh the whole model, that does not work in any serious scenario

M

Mimoune, are you referring to previously existing Dataflows as well? or just new dataflows?


Personally I think what you're suggesting would be great, especially for an existing dataflow that someone wants to use in a datamart. I'm sure I'm not the only one who already has many active dataflows that I would love to be able to work with in a datamart, but don't really want to duplicate all of them just for the datamart capabilities.

M

Allowing dataflows to choose their destination simplifies the story for users:

  • a dataflow is a way to get data from source systems into a storage destination.
  • you can pick either basic datalake storage (CSVs + manifest/typing) or richer relational database storage (Azure SQL DB).