Offer one new Fabric admin API around items usage suring each time points (item ,datetimes start & end , timepoint , user , CUs , darations , throttling , status etc ... ) on Capacities
Improve the copy tast on data pipelines
We are only waiting for data pipeline COPY task improvement
1) Different modes to load data to destination
Types & use cases
--------------------
[Overwrite] - Fact and dimension SQL tables (Fabric DWH & LH)
[Append] - Fact like a stock table snapsh...
Security on FABRIC
𝐖𝐚𝐫𝐞𝐡𝐨𝐮𝐬𝐞 & 𝐋𝐚𝐤𝐞 𝐡𝐨𝐮𝐬𝐞 𝐒𝐐𝐋 𝐬𝐜𝐡𝐞𝐦𝐚𝐬
Hi ideas team
✔️ Provide on SQL table schema a new column type = Record to deal with nested data like JSON
✔️ Offer a "clear contract" for storage, ETL ingestion tools & SQL Queries consumers
✔️ Improve compliance
✔️ Avoid all JSON arrays storage on a s...
Improve Data factory (pipeline & dataflow gen2)
Hi Fabric ideas team
✔️Improve the copy (data pipeline & dfgen2) with these use case
Native management avoiding currents work arounds
- [Full Overwrite] ==> Ingestion all layers (RAW/ Staging / bronze / silver / gold tables)
- [Append]...
Manage SQL authorized views on Lake house and DWH
✔️ Provide something like Bigquery authorized views
- Managed access for the SQL GOLD view (Gold DWH or LH) to underlying RAW tables (another DWH or LH on same Workspace)
- RAW LH or DWH owner gives access to the GOLD view
𝐂𝐨𝐫𝐞 𝐭𝐨𝐩𝐢𝐜 : 𝐝𝐚𝐭𝐚 𝐬𝐨𝐮𝐫𝐜𝐞𝐬 𝐬𝐞𝐜𝐮𝐫𝐢𝐭𝐲 Push integration to the maximum by creating on domains areas (spaces ?) , "ready to use" and "multi artifacts usage" external sources connections
Push integration to the maximum by creating on domains areas (spaces ?) , "ready to use" and "multi artifacts usage" external sources connections
✔️ Avoid to share ERP & API & cloud & on premise & folders credentials to several E.T.L items creators
✔️ Why to ...
𝐂𝐨𝐫𝐞 𝐭𝐨𝐩𝐢𝐜 - 𝐒𝐞𝐜𝐮𝐫𝐢𝐭𝐲 : 𝐨𝐟𝐟𝐞𝐫 𝐚 𝐛𝐲 𝐝𝐨𝐦𝐚𝐢𝐧 𝐚𝐧𝐝 𝐠𝐥𝐨𝐛𝐚𝐥 𝐒𝐞𝐜𝐮𝐫𝐢𝐭𝐲 𝐀𝐮𝐝𝐢𝐭 𝐩𝐚𝐠𝐞 (𝐩𝐮𝐫𝐯𝐢𝐞𝐰 𝐨𝐧𝐥𝐲 𝐡𝐞𝐥𝐩 𝐨𝐧 𝐝𝐚𝐭𝐚-𝐜𝐚𝐭𝐚𝐥𝐨𝐠)
𝐂𝐨𝐫𝐞 𝐭𝐨𝐩𝐢𝐜 - 𝐒𝐞𝐜𝐮𝐫𝐢𝐭𝐲 : 𝐨𝐟𝐟𝐞𝐫 𝐚 𝐛𝐲 𝐝𝐨𝐦𝐚𝐢𝐧 𝐚𝐧𝐝 𝐠𝐥𝐨𝐛𝐚𝐥 𝐒𝐞𝐜𝐮𝐫𝐢𝐭𝐲 𝐀𝐮𝐝𝐢𝐭 𝐩𝐚𝐠𝐞 (𝐩𝐮𝐫𝐯𝐢𝐞𝐰 𝐨𝐧𝐥𝐲 𝐡𝐞𝐥𝐩 𝐨𝐧 𝐝𝐚𝐭𝐚-𝐜𝐚𝐭𝐚𝐥𝐨𝐠)
✔️All items (data source , WS , DWH , LH , One lake , Semantic model etc ..) access...
𝐃𝐚𝐭𝐚 𝐟𝐚𝐜𝐭𝐨𝐫𝐲 𝐰𝐨𝐫𝐤𝐥𝐨𝐚𝐝 improvements
✔️ Improuve data pipeline CICD and data pipeline parameters or variables transmission to children TASKS (input & output results)
- Accept parameters for DFGEN2 artifacts runs
- On deployment pipeline offer Workspace rules to updates data pipeline parameters values
𝐖𝐚𝐫𝐞𝐡𝐨𝐮𝐬𝐞 𝐚𝐧𝐝 𝐓 𝐒𝐐𝐋 𝐬𝐮𝐩𝐩𝐨𝐫𝐭 - Imprive & avoid limitations
𝐖𝐚𝐫𝐞𝐡𝐨𝐮𝐬𝐞 𝐚𝐧𝐝 𝐓 𝐒𝐐𝐋 𝐬𝐮𝐩𝐩𝐨𝐫𝐭
✔️ Any news about T-SQL DDL ==>
- alter table add/remove column
- Identity Columns
- Primary keys
- RECORD Columns (store nested fields like json structures)
- temporary tables (useful on stored proc)
STATUS DETAILSPlanned