Skip to main content
Microsoft Idea

Synapse

New

Enable automated testing of Spark Notebooks

Vote (6) Share
Stefan Janse van Rensburg's profile image

Stefan Janse van Rensburg on 09 Feb 2024 09:55:16

As a Python developer I need to run unit tests on all my PySpark notebook code. All the functionality already exists to run these tests automatically but the naming convention of Notebook files completely ignore python module and package naming rules. There are two issue that would be fantastic to address:


  1. The Notebook folder created from git integration should not have periods in the name, eg "my_notebook.Notebook".
  2. The notebook python script created from git integration should not have dashes in the name, eg "notebook-content.py"


My idea is to simply replace these symbols with underscores. I hope it's a small enough change and it would have a massive impact on enabling automated testing for all Notebook scripts