Microsoft recently announced that we can now make our Azure Data Factory (ADF) v2 pipelines even more dynamic with the introduction of parameterised Linked Services. This now completes the set for our core Data Factory components meaning we can now inject parameters into every part of our Data Factory control flow orchestration processes. Pipelines ✔ Activities … Continue reading Using Data Factory Parameterised Linked Services
Category: Data Factory
Creating an Azure Data Factory v2 Custom Activity
Many moons ago and in a previous job role I wrote a post for creating an Azure Data Factory v1 Custom Activity here. According to Google Analytics this proved to be one of my most popular blog posts on that site. I therefore feel I need to do an update post with the same information … Continue reading Creating an Azure Data Factory v2 Custom Activity
What is Azure Data Factory Data Flow?
There were many great announcements to come out of the Microsoft Ignite 2018 conference, but the main one for me was the introduction of Azure Data Factory Data Flow. As a business intelligence person with many years experience working with SSIS packages this new feature is very exciting! Since I first saw the mock up … Continue reading What is Azure Data Factory Data Flow?
Preparing My Community Sessions for 2019
Hey friends & SQL family, its been a great year of tech and events so far, but now with several big conferences planning next years agendas its time to think ahead. Below I've prepared titles and abstracts for talks I'd like to deliver to our awesome data platform community in 2019. 5x regular sessions. 2x … Continue reading Preparing My Community Sessions for 2019
Execute All SSIS Packages in Parallel with ADFv2 Lookup and ForEach Activities
Just a quick post friends following a talk I did on Azure Data Factory (ADF) v2 at SQL Saturday Manchester. In the talk I demonstrated using ADF more dynamically now version 2 supports expressions and parameters... Amongst other cool stuff I specifically showed how you could run your SSIS packages, scaled out across all the … Continue reading Execute All SSIS Packages in Parallel with ADFv2 Lookup and ForEach Activities
How To Control the SSIS IR From Your ADFv2 Pipelines – Start/Stop
Hello friends, in this post I want to share with you a couple of methods I've developed for controlling the start up and stop of the Azure Data Factory (ADF) v2 SSIS Integration Runtime (IR) from your pipelines. This requirement came about mainly because of cost... The SSIS IR underneath is simply a cluster of … Continue reading How To Control the SSIS IR From Your ADFv2 Pipelines – Start/Stop
Using the ADFv2 Until Activity
Hello friends, with this post I wanted to give you a quick 'how to' guide when working with the Azure Data Factory (ADF) v2 Until activity... Having implemented the activity several times in production I've repeatedly found the current Microsoft documentation here falling a little short in its explanations and examples. So, I've created my … Continue reading Using the ADFv2 Until Activity
