ADFv2 Variable String Interpolation – REST API Body

As we know Azure Data Factory (ADF) Version 2 now has the ability to use expressions, parameters and system variables in various components throughout the service. This is brilliant and the best thing about v2 allowing us to build some very dynamic parameter driven pipelines. However, the syntax isn't always that obvious when you use … Continue reading ADFv2 Variable String Interpolation – REST API Body

Azure Data Lake Store – Files Dashboard

A tool I've used in the past to query my local disk drives is WinDirStat. It's great for finding out where large files are hiding in your directories using a heat map visual. In this post I talk you through how I've created a similar tool for Azure Data Lake Store using PowerShell, SQL Server … Continue reading Azure Data Lake Store – Files Dashboard

Inject into Azure Logic Apps ARM Templates with PowerShell

Continuous delivery of data services has always been a battle for us data analytics peeps. Things are getting better when we think about business intelligence and dev ops, however there are still gaps. This blog post is an account of one such issue I recently faced and how I worked around the problem. Problem While … Continue reading Inject into Azure Logic Apps ARM Templates with PowerShell

Azure Data Factory v2 Developer Interface Launched

Its finally here!! After being teased for nearly 4 months with all the great new functionality available in Azure Data Factory version 2 we finally have the tools to start developing it. Microsoft, it appears, have leap frogged my long standing request and feedback votes to support ADF templates in Visual Studio 2017 and gone … Continue reading Azure Data Factory v2 Developer Interface Launched

Azure Data Factory v1 & v2 Service Principal Authentication for Azure Data Lake

Sadly this post has been born out of frustration, so please accept my apologies from the outset if that comes across in the tone. That said, I imagine anyone reading it will empathise and will have come across the post while searching for an understanding of some inconsistencies when using service principals to authenticate against … Continue reading Azure Data Factory v1 & v2 Service Principal Authentication for Azure Data Lake