There were many great announcements to come out of the Microsoft Ignite 2018 conference, but the main one for me was the introduction of Azure Data Factory Data Flow. As a business intelligence person with many years experience working with SSIS packages this new feature is very exciting! Since I first saw the mock up … Continue reading What is Azure Data Factory Data Flow?
Just a quick post friends following a talk I did on Azure Data Factory (ADF) v2 at SQL Saturday Manchester. In the talk I demonstrated using ADF more dynamically now version 2 supports expressions and parameters... Amongst other cool stuff I specifically showed how you could run your SSIS packages, scaled out across all the … Continue reading Execute All SSIS Packages in Parallel with ADFv2 Lookup and ForEach Activities
Hello friends, with this post I wanted to give you a quick 'how to' guide when working with the Azure Data Factory (ADF) v2 Until activity... Having implemented the activity several times in production I've repeatedly found the current Microsoft documentation here falling a little short in its explanations and examples. So, I've created my … Continue reading Using the ADFv2 Until Activity
This is just a quick post, mainly for my own reference. When working with code generated solutions we often need to convert datasets from SQL Server (T-SQL) data types to Azure Data Lake Analytics (U-SQL) data types. As you probably know U-SQL has a hybrid syntax of T-SQL and C# which uses .Net data types. … Continue reading T-SQL to U-SQL Data Type Conversion
As we know Azure Data Factory (ADF) Version 2 now has the ability to use expressions, parameters and system variables in various components throughout the service. This is brilliant and the best thing about v2 allowing us to build some very dynamic parameter driven pipelines. However, the syntax isn't always that obvious when you use … Continue reading ADFv2 Variable String Interpolation – REST API Body
Happy New Year folks and welcome back to work! I hope you had a nice break. 🙂 I decided to kick off the new year of blogging with something fairly simple following some support work I did helping out a friend in the community. The problem encountered was twofold when dealing with data in Azure … Continue reading Using U-SQL Split to Handle Variable Width Datasets