Azure Data Factory Resource Limitations

Hello friends, I'm creating this post hopefully to raise awareness for my followers of the service limitations for Azure Data Factory. Like most resources in the Microsoft Cloud Platform at various levels (Resource/Resource Group/Subscription/Tenant) there are limitations, these are enforced by Microsoft and most of the time we don't hit them, especially when developing. That … Continue reading Azure Data Factory Resource Limitations

My Script for Peer Reviewing Code

With this post I wanted to write something a little different, something that isn't directly focused on a particular technology, but is just as important when delivering an enterprise grade solution. Or any solution for that matter! Typically we all work in a team delivering Data Platform solutions for our internal/external customers. In order for … Continue reading My Script for Peer Reviewing Code

Structuring Your Databricks Notebooks with Markdown, Titles, Widgets and Comments

Just a short post following a recent question I got from my delivery team... Are there any best practices for structuring our Databricks Notebooks in terms of code comments and markdown? Having done a little Googling I simply decided to whip up a quick example that could be adopted as a technical standard for the … Continue reading Structuring Your Databricks Notebooks with Markdown, Titles, Widgets and Comments

Interacting with SQL Server 2019 Big Data Clusters

Just a very quick post with lots of pictures πŸ™‚ SQL Server 2019 Big Data Clusters are still in private preview and I'm currently running with version CTP 2.1. However, I wanted to share some of the exciting new features we can expect from this latest big data tech in terms of tools, management and … Continue reading Interacting with SQL Server 2019 Big Data Clusters