Category: Data Factory

Data Integration, Architecture and Real-Time Analytics Full Training Days with Me 2024

Posted on by mrpaulandrew

SQL Bits, UK, go to >> SQL Day, Poland, go to >> Data Platform Next Steps, Denmark, go to >> Big Data Europe, Lithuania, go to >> SQLBits | March | Online & In Person Microsoft Data Integration Pipelines – The Fundamentals to Level 300 Event Link: https://sqlbits.com/attend/the-agenda/full-day-training/Microsoft_Data_Integration_Pipelines__The_Fundamentals_to_Level_300 Overview In this full day of training, … Continue reading Data Integration, Architecture and Real-Time Analytics Full Training Days with Me 2024 →

procfwk is Getting An Upgrade to CF.Cumulus

Posted on by mrpaulandrew

CF.Cumulus.Control Early Release Running a company (Cloud Formations) is very much a balance between steering the ship, while building the ship! As a result, my open source metadata driven orchestration framework (procfwk.com) got neglected. To the point where the Azure Functions, the middleware, originally written in .Net 3.1 Core is no longer supported. Additionally, the … Continue reading procfwk is Getting An Upgrade to CF.Cumulus →

SQLBits 2024 – My Sessions

Posted on by mrpaulandrew

This year at SQLBits I'm once again honoured to be presenting a full training day, a 100 minute session and a regular session during the week. It'll be my 8th time at SQLBits and my 4th time delivering a full training day. Full agenda here: https://sqlbits.com/attend/the-agenda My Hat-Trick Of Sessions Microsoft Data Integration Pipelines – … Continue reading SQLBits 2024 – My Sessions →

What Is Microsoft Fabric? My Point of View

Posted on by mrpaulandrew

Bringing the Data Platform Workspace and the Power Platform workspace Together Over the last 18 months or so I've been watching and supporting the development of Microsoft's new Software as a Service (SaaS) offering that has now been named at MSBuild 2023 as Microsoft Fabric. As a Microsoft MVP I've be aware of this offering … Continue reading What Is Microsoft Fabric? My Point of View →

Training Events in October 2022

Posted on by mrpaulandrew

Azure Data Factory & Azure Synapse Analytics Pipeline Training Get your tickets for more in person training with me this October. Limited places available, don't miss out. Azure Data Integration Pipelines: Fundamentals to Level 300 When: Monday, 10th October Where: Lamot Conference & Heritage Centre - Mechelen, Belgium Host: dataMinds Connect Tickets: https://tickets.datamindsconnect.be/ Azure Data … Continue reading Training Events in October 2022 →

SQLBits 2022 – My Sessions

Posted on by mrpaulandrew

This year at SQLBits I'm honoured to be presenting a full training day and two regular sessions throughout the week. It'll be my 6th time at SQLBits, my 3rd time delivering a full training day and my 6th/7th time delivering sessions, I think! Full agenda here: https://sqlbits.com/information/schedulegrid My Sessions Azure Data Integration Pipelines - Advanced … Continue reading SQLBits 2022 – My Sessions →

Azure Data Integration Pipelines – A Deep Dive

Posted on by mrpaulandrew

Having worked with Azure Data Factory and now Azure Synapse Analytics data integration pipelines ever since the resources first appeared in the Microsoft cloud platform, I've definitley earned (more than) my fair share of experience using the products. Both good and bad. As a data engineer and data archtiect. To help soften those bad experiences, … Continue reading Azure Data Integration Pipelines – A Deep Dive →

How Interchangeable Are Integration pipelines Between Azure Data factory and Azure Synapse Analytics?

Posted on by mrpaulandrew

Inspired by an earlier blog where we looked at 'How Interchangeable Delta Tables Are Between Databricks and Synapse' I decided to do a similar exercise, but this time with the integration pipeline components taking centre stage. As I said in my previous blog post, the question in the heading of this blog should be incredibly … Continue reading How Interchangeable Are Integration pipelines Between Azure Data factory and Azure Synapse Analytics? →

An Introduction to Azure Data Integration Pipelines

Posted on by mrpaulandrew

Azure Data Factory & Azure Synapse Analytics Following a recent talk I did for the MeetUp 'Let's Talk - Data Engineering' , here is the recording on YouTube for those that didn't catch the live broadcast. I squeezed a lot of content into an hour for those that are new to Azure Data Factory and … Continue reading An Introduction to Azure Data Integration Pipelines →

Scaling Azure Data Integration Pipelines – Decoupling Data Extract And Transform

Posted on by mrpaulandrew

Scaling Azure Data Factory & Azure Synapse Analytics Pipelines Context Back in May 2020 I wrote a blog post about 'When You Should Use Multiple Azure Data Factory's'. Following on from this post with a full year+ now passed and having implemented many more data platform solutions for some crazy massive (technical term) enterprise customers … Continue reading Scaling Azure Data Integration Pipelines – Decoupling Data Extract And Transform →