Skip to main content

Data Science

New

Support better and/or "global" setting of spark.conf.set properties

Vote (2) Share
Scott Powell's profile image

Scott Powell on 13 Mar 2024 22:56:40

I've seen that by default Notebooks write table names in Lakehouses in all lower case. Setting the following solves the problem - but I'd like this setting to either be the default (I don't understand why anyone would "want" things to default to lower case) or to set some sort of "global parameter" so I wouldn't have to specify this in every notebook:


spark.conf.set('spark.sql.caseSensitive', True)


Thanks!

Scott