Data Factory

Planned

Enable to pass variables from pipelines as parameters into a Dataflow Gen2

Vote (262)

Miguel Escobar on 01 Jun 2023 17:44:52

Enable to modify the mashup document of a Dataflow Gen2 by passing a variable from a pipeline before an execution of the Dataflow

Administrator on 22 Aug 2024 17:14:52

This is currently in planning. Be sure to check out the release plan for timelines around this feature once they become available:

What's new and planned for Data Factory in Microsoft Fabric - Microsoft Fabric | Microsoft Learn

Comments (28)

Comments (28)

Lackshu Balasubramaniam on 31 Oct 2024 02:25:20

RE: Enable to pass variables from pipelines as parameters into a Dataflow Gen2

I don't see it in the release plan link. Do we have a general timeline?

Michael Del-Papa on 21 Aug 2024 20:08:30

RE: Enable to pass variables from pipelines as parameters into a Dataflow Gen2

Honestly, the fact that you can plop a Dataflow Gen2 into a pipeline and connect it to activities is a complete joke since you are unable to pass output, parameters, or variables. A random dataflow sitting in the middle of your pipeline but not talking in any way to the other activities. Yay! PowerQuery is touted for being able to connect to over 300 different sources, but none of those are Pipeline objects? What genius at Microsoft thought that would get any traction? How dumb do they think PowerBI users are?Instead, I see most users bypassing Dataflows altogether in favor of Notebooks, and now I can see why. This needs to be fixed pronto because I can't see too many organizations willing to sign up for Fabric if they still need to deploy code-based solutions for their ETL inside Fabric.PowerQuery in Azure Data Factory? Who cares? It doesn't work. If you need to do complex transformations of your data, Notebooks IS THE ONLY SOLUTION.

René Valentin Panning on 21 Aug 2024 10:31:02

RE: Enable to pass variables from pipelines as parameters into a Dataflow Gen2

I just noticed that the backend for this feature seems to exist already. When starting a dataflow from a pipeline, defined parameters show as input in the pipeline monitor. I have trouble understanding just why passing parameters would be more difficult to implement.