For my first appearance on the Altius Data Lounge show I talk about and demonstrate the 'out-of-the-box' capabilities within Azure Data Factory for Git integration and how to publish developed pipelines. In my demo you'll see what each of the developer UI toolbar functions means in the context of the code repository and what happens … Continue reading Deploying Azure Data Factory in a Single Service Environment (Option 1)
Structuring Your Databricks Notebooks with Markdown, Titles, Widgets and Comments
Just a short post following a recent question I got from my delivery team... Are there any best practices for structuring our Databricks Notebooks in terms of code comments and markdown? Having done a little Googling I simply decided to whip up a quick example that could be adopted as a technical standard for the … Continue reading Structuring Your Databricks Notebooks with Markdown, Titles, Widgets and Comments
Get Any Azure Data Factory Pipeline Run Status with Azure Functions
Microsoft.Azure.Management.DataFactory.Models.RunFilterParameters So I recently found myself with code withdrawal… Yes, its a real thing! Anyone in an architect type role will understand, I hope… You can’t spend all your time drawing pictures of how things “should” work without occasionally getting hands-on with some code. Anyway, my recent episode of code withdrawal led me to creating … Continue reading Get Any Azure Data Factory Pipeline Run Status with Azure Functions
Microsoft Ignite 2019 Announcements – The Ignite Book of News 2019
How nice is this!... All the Microsoft Ignite announcements in one place. Very cool. Data Platform and AI things in chapter 3, starting on page 39. Sadly not that many new things for us data people this year, but still, its all great news. Azure Synapse Analytics seems to be the main data platform news, … Continue reading Microsoft Ignite 2019 Announcements – The Ignite Book of News 2019
Azure Data Factory – Pipeline Hierarchies (Generation Control)
Best practice for structuring your Azure Data Factory? Hello friends, this post and my next few posts have been born out of a community talk I've delivered a couple of times now at various conferences. In my session I talk about using Azure Data Factory (ADF) in production - inspired by my own lessons learnt … Continue reading Azure Data Factory – Pipeline Hierarchies (Generation Control)
Are you an MVP in the making?
Hi friends, just a casual reflective post from me. Within the data platform community you get to meet so many great people who are true experts in their respective fields. This made me wonder how many people are still out there that are truly the most valuable professionals? These people might be valuable to their … Continue reading Are you an MVP in the making?
Another Career Milestone – Joining Altius
Today marks another milestone in my career as I leave Adatis after a really excellent 18 months and start my next challenge with Altius. Through our brilliant data platform community, my growing reputation has given me the privilege of choosing my next employer. Something that I feel incredibly humbled by, as I have been approached … Continue reading Another Career Milestone – Joining Altius
