Skip to main content

Data Factory

Needs Votes

Data Pipelines - Create Relative Connection to Warehouse/Lakehouse/KQL DB

Vote (46) Share
Koen Verbeeck's profile image

Koen Verbeeck on 14 Jun 2024 07:39:29

When you create a connection to a warehouse, lakehouse or KQL DB, the connection is "absolute", meaning that the workspace ID is hardcoded into the connection.

This gives issues when you deploy the pipeline to another workspace with deployment pipelines, because ADF will try to resolve the connection using the original workspace, instead of the new workspace.

It can't find it, so the connection gives an error.


You can work around the issue by parameterizing the connection entirely, as described in this blog post:

https://sqlkover.com/dynamic-warehouse-lakehouse-connections-in-microsoft-fabric-data-pipelines/


However, it would be better if you could create a "relative" connection. This means: find the warehouse with the same name in the current workspace.


Administrator on 27 Nov 2024 08:31:10

This is in our roadmap

Comments (2)
Koen Verbeeck's profile image Profile Picture

Tim S on 08 Nov 2024 05:20:07

RE: Data Pipelines - Create Relative Connection to Warehouse/Lakehouse/KQL DB

This was previously listed on the public roadmap with an ETA of Q4 2024 but has since been removed entirely.When will it be delivered?

Koen Verbeeck's profile image Profile Picture

Koen Verbeeck on 14 Jun 2024 07:40:22

RE: Data Pipelines - Create Relative Connection to Warehouse/Lakehouse/KQL DB

Something like .\mywarehouse, instead of myworkspace\mywarehouse. Very similar to how SSIS environment references work (for some reason, I cannot put the URL to the doc page, because it is "malicious input".