This year at SQLBits I'm honoured to be presenting a full training day and two regular sessions throughout the week. It'll be my 6th time at SQLBits, my 3rd time delivering a full training day and my 6th/7th time delivering sessions, I think! Full agenda here: https://sqlbits.com/information/schedulegrid My Sessions Azure Data Integration Pipelines - Advanced … Continue reading SQLBits 2022 – My Sessions
Recording from the Hybrid Virtual Group MeetUp Abstract Once upon a time, there was a data warehouse and it lived happily as a set of tables within our relational database management system (RDMS) called Microsoft SQL Server. The data warehouse had three children known as extract, transform, and load. One day a blue/azure coloured cloud … Continue reading An Introduction to Delta Lakes and Delta Lake-Houses
Having worked with Azure Data Factory and now Azure Synapse Analytics data integration pipelines ever since the resources first appeared in the Microsoft cloud platform, I've definitley earned (more than) my fair share of experience using the products. Both good and bad. As a data engineer and data archtiect. To help soften those bad experiences, … Continue reading Azure Data Integration Pipelines – A Deep Dive
Inspired by an earlier blog where we looked at 'How Interchangeable Delta Tables Are Between Databricks and Synapse' I decided to do a similar exercise, but this time with the integration pipeline components taking centre stage. As I said in my previous blog post, the question in the heading of this blog should be incredibly … Continue reading How Interchangeable Are Integration pipelines Between Azure Data factory and Azure Synapse Analytics?
Azure Data Factory & Azure Synapse Analytics Following a recent talk I did for the MeetUp 'Let's Talk - Data Engineering' , here is the recording on YouTube for those that didn't catch the live broadcast. I squeezed a lot of content into an hour for those that are new to Azure Data Factory and … Continue reading An Introduction to Azure Data Integration Pipelines
Scaling Azure Data Factory & Azure Synapse Analytics Pipelines Context Back in May 2020 I wrote a blog post about 'When You Should Use Multiple Azure Data Factory's'. Following on from this post with a full year+ now passed and having implemented many more data platform solutions for some crazy massive (technical term) enterprise customers … Continue reading Scaling Azure Data Integration Pipelines – Decoupling Data Extract And Transform
Background Building on a previous blog post where I explored what a possible Azure Synapse Analytics logical architecture might look like in terms of end-to-end data curation/enrichment, here: Thinking about an Azure Synapse Analytics Logical Architecture v1 Now, with the recently added Synapse 'Managed Virtual Network' I'm exploring the same architecture from a physical network connectivity … Continue reading Thinking About an Azure Synapse Analytics Physical Architecture v1
Hi folks, as some of you may be aware it was the Microsoft MVP Summit in March, for MVP's this is an exciting time where Microsoft product teams share plans for the coming year and talk about features on the roadmap. That said, I have the privilege of knowing what the future looks like for … Continue reading Thinking about an Azure Synapse Analytics Logical Architecture v1
Azure Data Analytics End to End Let's start with a story, not a 'once upon a time story', a story for your backlog 🙂 As a solution architect I need to design and build an Azure data analytics platform end to end to deliver data insights for my customer. In February 2021 I delivered a … Continue reading An Architects Guide to Delivering Data Insights Using the Microsoft Azure Data Platform
A few weeks ago I delivered an introduction session on Azure Synapse Analytics as part of the Data Toboggan event. In my session I explored what Synapse is? And given the answer isn't that straight forward or obvious I thought it was probably worth re-sharing here as the recording is now available on YouTube. In … Continue reading What is Azure Synapse Analytics?