Building a Data Mesh Architecture in Azure – Part 4

Data Mesh vs Azure -Theory vs practice Use the tag Data Mesh vs Azure to follow this blog series. As a reminder, the four data mesh principals: domain-oriented decentralised data ownership and architecture.data as a product.self-serve data infrastructure as a platform.federated computational governance. Source reference: https://martinfowler.com/articles/data-mesh-principles.html A Quick Recap In Part 1, Part 2 and … Continue reading Building a Data Mesh Architecture in Azure – Part 4

Azure Data Integration Pipelines – Advanced Design and Delivery

SQLBits 2022 - Full Day of Training 9th March – Online or In Person at the London Excel https://sqlbits.com https://youtu.be/41eiT-Qp-bA In this full day training data session, we'll quickly cover the fundamentals of data integration pipelines before going much deeper into our Azure resources. Within a typical Azure data platform solution for any enterprise grade … Continue reading Azure Data Integration Pipelines – Advanced Design and Delivery

SQLBits 2022 – My Sessions

This year at SQLBits I'm honoured to be presenting a full training day and two regular sessions throughout the week. It'll be my 6th time at SQLBits, my 3rd time delivering a full training day and my 6th/7th time delivering sessions, I think! Full agenda here: https://sqlbits.com/information/schedulegrid My Sessions Azure Data Integration Pipelines - Advanced … Continue reading SQLBits 2022 – My Sessions

Building a Data Mesh Architecture in Azure – Part 3

Data Mesh vs Azure -Theory vs practice Use the tag Data Mesh vs Azure to follow this blog series. As a reminder, the four data mesh principals: domain-oriented decentralised data ownership and architecture.data as a product.self-serve data infrastructure as a platform.federated computational governance. Source reference: https://martinfowler.com/articles/data-mesh-principles.html In Part 1 and Part 2 of this blog … Continue reading Building a Data Mesh Architecture in Azure – Part 3

Building a Data Mesh Architecture in Azure – Part 2

Data Mesh vs Azure -Theory vs practice Use the tag Data Mesh vs Azure to follow this blog series. As a reminder, the four data mesh principals: domain-oriented decentralised data ownership and architecture.data as a product.self-serve data infrastructure as a platform.federated computational governance. Source reference: https://martinfowler.com/articles/data-mesh-principles.html In Part 1 of this blog series we looked … Continue reading Building a Data Mesh Architecture in Azure – Part 2

Building a Data Mesh Architecture in Azure – Part 1

Data Mesh vs Azure - Theory vs practice Use the tag Data Mesh vs Azure to follow this blog series. Context and Background The concepts and principals of a data mesh architecture have been around for a while now and I've yet to see anyone else apply/deliver such a solution in Azure. I'm wondering if … Continue reading Building a Data Mesh Architecture in Azure – Part 1

An Introduction to Delta Lakes and Delta Lake-Houses

Recording from the Hybrid Virtual Group MeetUp Abstract Once upon a time, there was a data warehouse and it lived happily as a set of tables within our relational database management system (RDMS) called Microsoft SQL Server. The data warehouse had three children known as extract, transform, and load. One day a blue/azure coloured cloud … Continue reading An Introduction to Delta Lakes and Delta Lake-Houses

How Interchangeable Are Integration pipelines Between Azure Data factory and Azure Synapse Analytics?

Inspired by an earlier blog where we looked at 'How Interchangeable Delta Tables Are Between Databricks and Synapse' I decided to do a similar exercise, but this time with the integration pipeline components taking centre stage. As I said in my previous blog post, the question in the heading of this blog should be incredibly … Continue reading How Interchangeable Are Integration pipelines Between Azure Data factory and Azure Synapse Analytics?

An Introduction to Azure Data Integration Pipelines

Azure Data Factory & Azure Synapse Analytics Following a recent talk I did for the MeetUp 'Let's Talk - Data Engineering' , here is the recording on YouTube for those that didn't catch the live broadcast. I squeezed a lot of content into an hour for those that are new to Azure Data Factory and … Continue reading An Introduction to Azure Data Integration Pipelines

Scaling Azure Data Integration Pipelines – Decoupling Data Extract And Transform

Scaling Azure Data Factory & Azure Synapse Analytics Pipelines Context Back in May 2020 I wrote a blog post about 'When You Should Use Multiple Azure Data Factory's'. Following on from this post with a full year+ now passed and having implemented many more data platform solutions for some crazy massive (technical term) enterprise customers … Continue reading Scaling Azure Data Integration Pipelines – Decoupling Data Extract And Transform