Visio Stencils – For the Azure Solution Architect

I feel like I've mentioned this collateral to loads of people, but only recently releaised that I've never actually blogged about it! So, just a quick post to help anyone that likes Visio and drawing pictures. In short, my brains works best with pictures. I like drawing architecture diagrams and can represent almost anything visually … Continue reading Visio Stencils – For the Azure Solution Architect

What is Microsoft’s Project Volterra?

Burning the candle at both ends in the UK I followed along with as many sessions as possible from this year's Microsoft Build conference. While my primary focus has and always will be on the Data Platform tech. I couldn't help but take note when Satayla Nednella introduced Project Volterra. The small boxes shown in … Continue reading What is Microsoft’s Project Volterra?

Building a Data Mesh Architecture in Azure – Part 4

Data Mesh vs Azure -Theory vs practice Use the tag Data Mesh vs Azure to follow this blog series. As a reminder, the four data mesh principals: domain-oriented decentralised data ownership and architecture.data as a product.self-serve data infrastructure as a platform.federated computational governance. Source reference: https://martinfowler.com/articles/data-mesh-principles.html A Quick Recap In Part 1, Part 2 and … Continue reading Building a Data Mesh Architecture in Azure – Part 4

Building a Data Mesh Architecture in Azure – Part 3

Data Mesh vs Azure -Theory vs practice Use the tag Data Mesh vs Azure to follow this blog series. As a reminder, the four data mesh principals: domain-oriented decentralised data ownership and architecture.data as a product.self-serve data infrastructure as a platform.federated computational governance. Source reference: https://martinfowler.com/articles/data-mesh-principles.html In Part 1 and Part 2 of this blog … Continue reading Building a Data Mesh Architecture in Azure – Part 3

Building a Data Mesh Architecture in Azure – Part 2

Data Mesh vs Azure -Theory vs practice Use the tag Data Mesh vs Azure to follow this blog series. As a reminder, the four data mesh principals: domain-oriented decentralised data ownership and architecture.data as a product.self-serve data infrastructure as a platform.federated computational governance. Source reference: https://martinfowler.com/articles/data-mesh-principles.html In Part 1 of this blog series we looked … Continue reading Building a Data Mesh Architecture in Azure – Part 2

Building a Data Mesh Architecture in Azure – Part 1

Data Mesh vs Azure - Theory vs practice Use the tag Data Mesh vs Azure to follow this blog series. Context and Background The concepts and principals of a data mesh architecture have been around for a while now and I've yet to see anyone else apply/deliver such a solution in Azure. I'm wondering if … Continue reading Building a Data Mesh Architecture in Azure – Part 1

Creating a Simple Staged Metadata Driven Processing Framework for Azure Data Factory Pipelines – Part 1 of 4

In this four part blog series I want to share my approach to delivering a metadata driven processing framework in Azure Data Factory. This is very much version 1 of what is possible and where can we build on this to deliver a hardened solution for our platform orchestrator. For the agenda of this blog … Continue reading Creating a Simple Staged Metadata Driven Processing Framework for Azure Data Factory Pipelines – Part 1 of 4

Azure Data Factory Resource Limitations

Hello friends, I'm creating this post hopefully to raise awareness for my followers of the service limitations for Azure Data Factory. Like most resources in the Microsoft Cloud Platform at various levels (Resource/Resource Group/Subscription/Tenant) there are limitations, these are enforced by Microsoft and most of the time we don't hit them, especially when developing. That … Continue reading Azure Data Factory Resource Limitations

My Script for Peer Reviewing Code

With this post I wanted to write something a little different, something that isn't directly focused on a particular technology, but is just as important when delivering an enterprise grade solution. Or any solution for that matter! Typically we all work in a team delivering Data Platform solutions for our internal/external customers. In order for … Continue reading My Script for Peer Reviewing Code

Structuring Your Databricks Notebooks with Markdown, Titles, Widgets and Comments

Just a short post following a recent question I got from my delivery team... Are there any best practices for structuring our Databricks Notebooks in terms of code comments and markdown? Having done a little Googling I simply decided to whip up a quick example that could be adopted as a technical standard for the … Continue reading Structuring Your Databricks Notebooks with Markdown, Titles, Widgets and Comments