Azure Data Analytics End to End Let's start with a story, not a 'once upon a time story', a story for your backlog 🙂 As a solution architect I need to design and build an Azure data analytics platform end to end to deliver data insights for my customer. In February 2021 I delivered a … Continue reading An Architects Guide to Delivering Data Insights Using the Microsoft Azure Data Platform
Tag: Azure
Microsoft Ignite 2019 Announcements – The Ignite Book of News 2019
How nice is this!... All the Microsoft Ignite announcements in one place. Very cool. Data Platform and AI things in chapter 3, starting on page 39. Sadly not that many new things for us data people this year, but still, its all great news. Azure Synapse Analytics seems to be the main data platform news, … Continue reading Microsoft Ignite 2019 Announcements – The Ignite Book of News 2019
PowerShell Export Databricks Workspace Items – Recurse
Building on the excellent PowerShell Databricks module created by Gerhard Brueckl here, I've added another layer of code to recursively export all items in a given Databricks workspace using PowerShell. I accept this does need to be hardened as a PowerShell cmdlet on its own and added to a module. However, I wanted to share the … Continue reading PowerShell Export Databricks Workspace Items – Recurse
Inject into Azure Logic Apps ARM Templates with PowerShell
Continuous delivery of data services has always been a battle for us data analytics peeps. Things are getting better when we think about business intelligence and dev ops, however there are still gaps. This blog post is an account of one such issue I recently faced and how I worked around the problem. Problem While … Continue reading Inject into Azure Logic Apps ARM Templates with PowerShell
Azure Data Factory v2 Developer Interface Launched
Its finally here!! After being teased for nearly 4 months with all the great new functionality available in Azure Data Factory version 2 we finally have the tools to start developing it. Microsoft, it appears, have leap frogged my long standing request and feedback votes to support ADF templates in Visual Studio 2017 and gone … Continue reading Azure Data Factory v2 Developer Interface Launched
Azure Data Factory v1 & v2 Service Principal Authentication for Azure Data Lake
Sadly this post has been born out of frustration, so please accept my apologies from the outset if that comes across in the tone. That said, I imagine anyone reading it will empathise and will have come across the post while searching for an understanding of some inconsistencies when using service principals to authenticate against … Continue reading Azure Data Factory v1 & v2 Service Principal Authentication for Azure Data Lake
Using U-SQL Split to Handle Variable Width Datasets
Happy New Year folks and welcome back to work! I hope you had a nice break. 🙂 I decided to kick off the new year of blogging with something fairly simple following some support work I did helping out a friend in the community. The problem encountered was twofold when dealing with data in Azure … Continue reading Using U-SQL Split to Handle Variable Width Datasets
Controlling U-SQL Job AU’s with Azure Data Factory v2 Pipeline Parameters
This post came about really from me playing around with Azure Data Factory Version 2 (ADFv2) and trying to gain a better appreciation of why and how you can now use parameters and expressions in ADFv2 to control your pipelines. Below is the first thing I tested out and hopefully gives you a taster of … Continue reading Controlling U-SQL Job AU’s with Azure Data Factory v2 Pipeline Parameters
The Azure Icon Game
If you've ever seen me at a community event you'll probably know that I like to warm up the crowd by playing the Azure Icon Game (as I've named it). When moving around the Microsoft cloud platform portal you will be greeted my many graphics, some are obvious, some are strange, others are a bit … Continue reading The Azure Icon Game
