Andre Fomin on 27 Mar 2024 21:43:15
Why are we limiting Fabric to only support TSQL endpoints? It would be great if I could connect to my Lakehouse and use Spark SQL to interact with it. We have a Script activity in Data Factory that only supports TSQL, it would be amazing if we could also have a script activity that supported Spark SQL or if I could connect to a Lakehouse from a Dataflow and also just paste a Spark SQL statement in it for a specific Lake House just like I am able to do with a SQL data source, I can pick a table or I can start with a SQL statement. Spark SQL and Spark should enjoy the same support in Fabric as anything TSQL based.