on
RE:
Here's another code snippet that the spark.sql() function seems to break, allthough the code is able to run. The code below gives the red squiggly underline errors "no viable alternative at input '\n'" and "extraneous input '.' expecting {, 'assert', 'async'...". But the code still runs. The errors can be solved by using backslashes on each line for line continuation, but that makes the code messy. When commenting or removing the spark.sql() line, the errors disappear.df = spark.sql(query)df1 = (df .withColumn("col1", lit(1)) .select(col('*')) )
RE:
Any updates here? Is this going to take months or years?How can you guys make call this stuff "GA" without any sort of CI/CD or git integration? You guys need to eat your own dogfood. It is shameful that you build this platform for hosting software solutions, yet have no meaningful plan for supporting CI/CD or source control. In past decades, Microsoft used to build very high quality development tooling for data engineers. Now I just want to avoid anything from Microsoft, and seek open-source alternatives which are much more robust, have better tooling, and have better support.
RE:
See this LCS issue for an explanation of the issue: https://fix.lcs.dynamics.com/Issue/Details?kb=0&bugId=942445&dbType=3&qc=dee038f98b9885f472762e3e5192749bb0a6eacf49631031a1725fb21620c9d9