Data Factory
PlannedData Pipelines - Create Relative Connection to Warehouse/Lakehouse/KQL DB
Koen Verbeeck on 14 Jun 2024 07:39:29
When you create a connection to a warehouse, lakehouse or KQL DB, the connection is "absolute", meaning that the workspace ID is hardcoded into the connection.
This gives issues when you deploy the pipeline to another workspace with deployment pipelines, because ADF will try to resolve the connection using the original workspace, instead of the new workspace.
It can't find it, so the connection gives an error.
You can work around the issue by parameterizing the connection entirely, as described in this blog post:
https://sqlkover.com/dynamic-warehouse-lakehouse-connections-in-microsoft-fabric-data-pipelines/
However, it would be better if you could create a "relative" connection. This means: find the warehouse with the same name in the current workspace.
Administrator
This is in our roadmap
- Comments (2)
RE: Data Pipelines - Create Relative Connection to Warehouse/Lakehouse/KQL DB
This was previously listed on the public roadmap with an ETA of Q4 2024 but has since been removed entirely.When will it be delivered?
RE: Data Pipelines - Create Relative Connection to Warehouse/Lakehouse/KQL DB
Something like .\mywarehouse, instead of myworkspace\mywarehouse. Very similar to how SSIS environment references work (for some reason, I cannot put the URL to the doc page, because it is "malicious input".