Like so many events SQLBits 2020 was delivered from our homes... Training sessions and content delivered via webcam and recordings. Catching up with friends done via chat rooms and breakout sessions. Hanging out in the coffee line done my hanging out in your kitchen with a headset. Banter and consume sharing done via social media. … Continue reading SQLBits 2020 – My Review & Thoughts
A Day Full of Azure Data Factory
Join me for a full day of training on Azure Data Factory. If we want to achieve any data processing in Azure you need an umbrella service to manage, monitor and schedule your solution. For a long time when working on premises, the SQL Agent has been our go-to tool, combined with T-SQL and SSIS … Continue reading A Day Full of Azure Data Factory
ADF.procfwk v1.8.5 – Execution Precursor
Code Project Overview This open source code project delivers a simple metadata driven processing framework for Azure Data Factory (ADF). The framework is made possible by coupling ADF with an Azure SQL Database that houses execution stage and pipeline information that is later called using an Azure Functions App. The parent/child metadata structure firstly allows … Continue reading ADF.procfwk v1.8.5 – Execution Precursor
Using the PowerShell Module ‘azure.datafactory.tools’ to Deploy ADF.procfwk
From my YouTube channel: https://www.youtube.com/c/mrpaulandrew Special guest Kamil Nowinski talks about how you can use his PowerShell Module 'azure.datafactory.tools' to deploy Azure Data Factory with ease and in a highly flexible way. Kamil kindly helped me simplify the deployment steps needed for publishing my ADF.procfwk Data Factory pipelines via PowerShell and using his Azure DevOps … Continue reading Using the PowerShell Module ‘azure.datafactory.tools’ to Deploy ADF.procfwk
ADF.procfwk v1.8.4 – Database Schema Reorganise and Restructuring
Code Project Overview This open source code project delivers a simple metadata driven processing framework for Azure Data Factory (ADF). The framework is made possible by coupling ADF with an Azure SQL Database that houses execution stage and pipeline information that is later called using an Azure Functions App. The parent/child metadata structure firstly allows … Continue reading ADF.procfwk v1.8.4 – Database Schema Reorganise and Restructuring
ADF.procfwk v1.8.2 – Optionally Store SPN Details in Azure Key Vault
Code Project Overview This open source code project delivers a simple metadata driven processing framework for Azure Data Factory (ADF). The framework is made possible by coupling ADF with an Azure SQL Database that houses execution stage and pipeline information that is later called using an Azure Functions App. The parent/child metadata structure firstly allows … Continue reading ADF.procfwk v1.8.2 – Optionally Store SPN Details in Azure Key Vault
Using NUnit to Automate the Testing of Azure Data Factory Pipelines
From my YouTube channel: https://www.youtube.com/c/mrpaulandrew Special guest Richard Swinbank talks about how you can use an NUnit project in Visual Studio to automate the testing of Data Factory pipelines. Richard kindly helped me apply this and test the pipelines in the ADF.procfwk to ensure post release everything still works in the open source code project. … Continue reading Using NUnit to Automate the Testing of Azure Data Factory Pipelines
How To Use ‘Specify dynamic contents in JSON format’ in Azure Data Factory Linked Services
Make any Azure Data Factory Linked Service dynamic! In a few different community circles I've been asked 'how to handle dynamic Linked Service connections in Azure Data Factory if the UI doesn't naturally support the addition of parameters'. Depending on the Linked Service the support for this varies. Well, the answer, or should I say, … Continue reading How To Use ‘Specify dynamic contents in JSON format’ in Azure Data Factory Linked Services
ADF.procfwk v1.8 – Complete Pipeline Dependency Chains For Failure Handling
Code Project Overview This open source code project delivers a simple metadata driven processing framework for Azure Data Factory (ADF). The framework is made possible by coupling ADF with an Azure SQL Database that houses execution stage and pipeline information that is later called using an Azure Functions App. The parent/child metadata structure firstly allows … Continue reading ADF.procfwk v1.8 – Complete Pipeline Dependency Chains For Failure Handling
ADF.procfwk v1.7.2 – NULL Pipeline Parameters Handled
Code Project Overview This open source code project delivers a simple metadata driven processing framework for Azure Data Factory (ADF). The framework is made possible by coupling ADF with an Azure SQL Database that houses execution stage and pipeline information that is later called using an Azure Functions App. The parent/child metadata structure firstly allows … Continue reading ADF.procfwk v1.7.2 – NULL Pipeline Parameters Handled
Welcome to the Blog of Paul Andrew 