PowerShell Export Databricks Workspace Items – Recurse

Building on the excellent PowerShell Databricks module created by Gerhard Brueckl here, I’ve added another layer of code to recursively export all items in a given Databricks workspace using PowerShell.

I accept this does need to be hardened as a PowerShell cmdlet on its own and added to a module. However, I wanted to share the first version sooner rather than later to support a couple of ongoing projects where Databricks Notebook CI/CD is becoming an important issue.

The code snippet requires four parameters as follows:

• Token – An access token for your Databricks service which can be generated from the User Setting menu.
• The URL – Your service. In the case of Azure this can just be the region.
• OutputPath – Where you want the exported items to land in your local environment.
• WorkspacePath – The level within your Databricks workspace where you want to recursively export items from.

For the first use you’ll probably need to install/import the DatabricksPS module.

Download the full script from my GitHub repo here:
https://github.com/mrpaulandrew/BlogSupportingContent/

Limitation

Beware that currently the export of libraries (eg. jar files) is not support by the workspace API.

Full API details here: https://docs.azuredatabricks.net/api/index.html