Bob Duffy on 20 Mar 2024 16:44:08
Its great that Fabric gives us many ways to write delta-parquet files, but for meta data and logging we really need a different storage. One that has super low latency and optimised for small non-analytical queries.
Ideally, what we need here is to be able to create a Fabric "SQLDB" and link or replicate it into a warehouse or Lakehouse
This would let us store some data in a row store but also query it from Fabric in conjunction with Fabric tables.
The killer use case here is meta data often which is the heart of any ETL automation solution.