Welcome to the Blog & Website of Paul Andrew
Technical Leadership Centred Around the Microsoft Data Platform
Databricks vs Synapse Analytics As an architect I often get challenged by customers on different approach's to a data transformation solutions, mainly because they are concerned about locking themselves into a particular technology, resource or vendor. One example of this is using a Delta Lake to deliver an Azure based warehousing/analytics platform. Given this context, … Continue reading How Interchangeable Are Delta Tables Between Azure Databricks and Azure Synapse Analytics?
Azure Data Factory & Azure Synapse Analytics Integrate Pipelines In this post I want us to explore and understand the difference between an internal and external activity when using our favourite orchestration pipelines. I'll focus predominately on Azure Data Factory (ADF), but the same applies to Azure Synapse Analytics. *Warning: this is a fairly dry, … Continue reading Pipelines – Understanding Internal vs External Activities
Just last week we heard the announcement from Microsoft that Azure Synapse Analytics is now generally available (GA)... A full year on, plus a few weeks, since first seeing Synapse at the big USA conferences in November 2019. Today I've been attempting to use the resource with a view to implementing it for several customer … Continue reading Trying to Deploy Azure Synapse Analytics Using ARM Templates
I've been playing around with Azure Synapse Analytics for a while now exploring the preview features and trying to find a meaningful use case for the 'single pane of glass' capabilities. In this post I'm exploring one possible option/idea for creating a very simple self service approach to dataset ingestion and consumption. Full disclosure, the … Continue reading An Idea for Self Service Using Azure Synapse Analytics
With Azure Synapse Analytics now in public preview is was time to find out how compatible my Azure Data Factory metadata driven processing framework (ADF.procfwk) is with the Synapse Orchestrate features. Firstly, as Synapse doesn't yet have any source control or DevOps support I had to manually rebuild the framework pipelines in the browser, copying … Continue reading ADF.procfwk and Azure Synapse Orchestrate – Preview Limitations
Exciting times friends, today (25th March) the lovely people at Microsoft granted me access to the private preview MVP workspace for Azure Synapse Analytics 😀 In this quick blog I wanted to share my experience so far, which I'm basically writing as I'm playing around... https://azure.microsoft.com/en-gb/services/synapse-analytics/ The main reason I wanted access to Synapse is … Continue reading First Time Playing with Spark.Net on Azure Synapse Analytics
Welcome to the Blog & Website of Paul Andrew 