Hi friends, just a casual reflective post from me. Within the data platform community you get to meet so many great people who are true experts in their respective fields. This made me wonder how many people are still out there that are truly the most valuable professionals? These people might be valuable to their … Continue reading Are you an MVP in the making?
Another Career Milestone – Joining Altius
Today marks another milestone in my career as I leave Adatis after a really excellent 18 months and start my next challenge with Altius. Through our brilliant data platform community, my growing reputation has given me the privilege of choosing my next employer. Something that I feel incredibly humbled by, as I have been approached … Continue reading Another Career Milestone – Joining Altius
Azure Data Factory – Web Hook vs Web Activity
As Azure Data Factory continues to evolve as a powerful cloud orchestration service we need to update our knowledge and understanding of everything the service has to offer. Mainly, so we can make the right design decisions when developing complex, dynamic solution pipelines. In this post I want to explore and share the reasons for … Continue reading Azure Data Factory – Web Hook vs Web Activity
PowerShell Export Databricks Workspace Items – Recurse
Building on the excellent PowerShell Databricks module created by Gerhard Brueckl here, I've added another layer of code to recursively export all items in a given Databricks workspace using PowerShell. I accept this does need to be hardened as a PowerShell cmdlet on its own and added to a module. However, I wanted to share the … Continue reading PowerShell Export Databricks Workspace Items – Recurse
Interacting with SQL Server 2019 Big Data Clusters
Just a very quick post with lots of pictures 🙂 SQL Server 2019 Big Data Clusters are still in private preview and I'm currently running with version CTP 2.1. However, I wanted to share some of the exciting new features we can expect from this latest big data tech in terms of tools, management and … Continue reading Interacting with SQL Server 2019 Big Data Clusters
Using Data Factory Parameterised Linked Services
Microsoft recently announced that we can now make our Azure Data Factory (ADF) v2 pipelines even more dynamic with the introduction of parameterised Linked Services. This now completes the set for our core Data Factory components meaning we can now inject parameters into every part of our Data Factory control flow orchestration processes. Pipelines ✔ Activities … Continue reading Using Data Factory Parameterised Linked Services
Creating an Azure Data Factory v2 Custom Activity
Many moons ago and in a previous job role I wrote a post for creating an Azure Data Factory v1 Custom Activity here. According to Google Analytics this proved to be one of my most popular blog posts on that site. I therefore feel I need to do an update post with the same information … Continue reading Creating an Azure Data Factory v2 Custom Activity
