Christian Harrington on 03 Nov 2023 19:08:25
The Lakehouse shortcuts are great for sure. But why can't I create some from a Warehouse? This should be available as well. Many scenarios will focus only on warehouse not lakehouse.
Or make Lakehouse shortcuts available from the Warehouse? Yes I know we can query them from the Warehouse like this: Lakehouse.schema.shortcut. That works (althought the intellisense works only 50% of the time...)
But it's not the same, you don't get visibiliy on them from the warehouse, I mean from the object explorer. I can add other warehouse with button "Add Warehouse" but this doesn't persist, you need to add it each time you have a new session. (this is another idea, coming soon)
Please add shortcuts available on Warehouse as well. This would be great.
Peri Rocha (administrator)
Thanks for posting your idea. Can you further expand on the scenarios where you would use shortcuts in Data Warehouse? Data ingestion? Querying external data? Any other scenarios you could share?
- Comments (11)
RE: Lakehouse Shortcuts available in Warehouse also
Planning to share common tables from Warehouse with other workspaces. The warehouses are created with case-insensitive (CI) collation, then we create shortcuts on lakehouse the tables in lakehouse changed to case-sensitive again. So it is better to allow create shortcuts on Warehouse, so the other workspaces can be created with case-insensitive (CI) collation and accesss the data without any case sensitive issues.otherwise having one copy is not possible with current options.
RE: Lakehouse Shortcuts available in Warehouse also
When Organization is selecting DWH Option. Keeping Source/Landing tables in One workspace. Target Dimensions and Facts in Different workspace. Cross Workspace querying is not allowed. In Order to load validate that data of Source/Landing is loaded into Target, we are unable to write T SQL Scripts. Shortcut in DWH will allow test team to write T SQL Scripts by querying Different workspace tables and confirm data in Target layer in different workspace.
RE: Lakehouse Shortcuts available in Warehouse also
The modern data warehouse should support shortcuts for creating common tables. These shortcuts allow me to share these tables easily with other teams or business units, simplifying their access and reference
RE: Lakehouse Shortcuts available in Warehouse also
Shortcuts between Datawarehouse's would help me in the way of structuring my organization inside workspaces, as I would create different workspaces per Department, and also I would be available to create a 'General' Warehouse where I would be able to put all the common tables needed in my company as for example, the table Calendar. I would like to have it just replicated it one time in the General Datawarehouse, I don't want to have the table in 7 different workspaces as it's the same data.
RE: Lakehouse Shortcuts available in Warehouse also
ShortCuts from LakeHouse to Warehouse is important, to get one copy of data. If not is possible technical this point, the I suggest to use CREATE EXTERNAL TABLE to point to the Delta Table in the LakeHouse, and the table will be only read. I think this table types in the warehouse is usually only for read and make relations between warehouse tables and Lakehouse tables. Also is necessary the work with these tables with Direct Lake.
RE: Lakehouse Shortcuts available in Warehouse also
It´s definitely a must have. No sense to manage warehouse and lakehouses differently based on the "One copy of Data" mantra. Warehouses should not be "second" division artifacts.....Many ThanksAlfons
RE: Lakehouse Shortcuts available in Warehouse also
I totally agree with this idea. Shortcuts between data warehouses would enforce the conformed dimension pattern. Going further, I we could add a shortcut to a lakehouse table, that would be better. For example, I have created a date dimension using a python script in a lakehouse, with shortcuts, I would be able to add it to most of my warehouses.Shortcuts in warehouses would make Fabric an even more compelling environment for data warehouses.
RE: Lakehouse Shortcuts available in Warehouse also
Another vote from me. I use warehouses for different departments and i want to use the employee table from the hr warehouse for other warehouses. At the moment i have to duplicate the table or use a shortcut in a lakehouse with a warehouse view on top, which excludes the possibility of direct lake.
RE: Lakehouse Shortcuts available in Warehouse also
I agree - we are trying not to duplicate data when we have it segmented in multiple warehouses / lakehouses ( think medallion architecture).
RE: Lakehouse Shortcuts available in Warehouse also
Fabric is marketed as "One Copy Of Data".If this is the case, how do we maintain shared DIMs across different Data Warehouses without shortcuts?Without shortcuts, we need to create clone tables in each warehouse that needs the DIM as part of the ETL process or create a View.But having the ability to create shortcuts from other warehouse and lakehouse tables in a warehouse is key to keep the "One copy of data" strategy consistent.It was available during preview - last I checked in Nov 2023.