Jefferson Bauer on 19 Dec 2024 20:25:28
I was excited about the possibility of using Data Flow Gen 2 to perform incremental refreshes. However, I noticed that Lakehouse is grayed out as an option. I then read Lakehouse isn't supported. Hopefully this feature is coming soon. The alternatives are not great. It would be very helpful to not have to manage all the moving parts of incremental refresh through Data Copy, Variables, Notebooks, Watermark tables, etc...
Please, please, please add support for Lakehouse destinations.
- Comments (2)
RE: Data Flow Gen 2 - Support for incremental refresh to Lakehouse
Agree. With the large datasets that my team is working on, I cannot see us making full use of DataFlow Gen 2 until incremental refresh is supported into a Lakehouse is supported.
RE: Data Flow Gen 2 - Support for incremental refresh to Lakehouse
Yes, we need this!