An Architects Guide to Delivering Data Insights Using the Microsoft Azure Data Platform

Azure Data Analytics End to End

Let’s start with a story, not a ‘once upon a time story‘, a story for your backlog 🙂

As a solution architect
I need to design and build an Azure data analytics platform end to end
to deliver data insights for my customer.

In February 2021 I delivered a talk as part of the Scottish Summit conference on how you could/should build an end to end data platform solution in Azure to deliver data insights and analytics. This is one of my favourite sessions so thought it worth re-sharing the recording here.

Talk Abstract

The resources on offer in Azure are constantly changing, which means as data professionals we need to constantly change too. Updating knowledge and learning new skills. No longer can we rely on products matured over a decade to deliver all our solution requirements. Today, data platform architectures designed in Azure with best intentions and known good practices can go out of date within months. That said, is there now a set of core components we can utilise in the Microsoft cloud to ingest and deliver insights from our data? When does ETL become ELT? When is IaaS better than PaaS? Do we need to consider scaling up or scaling out? And should we start making cost the primary factor for choosing certain technologies? In this session we’ll explore the answers to all these questions and more from an architects viewpoint. Based on real world experience lets think about just how far the breadth of our knowledge now needs to reach when starting from nothing and building a complete Microsoft Azure Data Platform solution.



Below is a view of the outcome from the session and the overall architecture that I talk about the delivery of and build up throughout the talk.

Thanks for watching

9 thoughts on “An Architects Guide to Delivering Data Insights Using the Microsoft Azure Data Platform

  1. Hi Paul,
    You’ve mentioned in your video about extending pipeline for sensitive data. Can you please explain that more or maybe demo on how to use metadata to identify PII data using ADF pipeline.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.