Updated – 10th Feb 2021
Following my previous blog post Is Azure Synapse Analytics Ready for Production? I was asked to summarize this understanding/opinion of Synapse into a simpler view.
This is what I did…
As a PDF if you prefer here.
Discuss 🙂
Thanks
Please feel free to share this:
Published by mrpaulandrew
Group Manager & Analytics Architect specialising in big data solutions on the Microsoft Azure cloud platform.
Data engineering competencies include Azure Synapse Analytics, Data Factory, Data Lake, Databricks, Stream Analytics, Event Hub, IoT Hub, Functions, Automation, Logic Apps and of course the complete SQL Server business intelligence stack.
Many years’ experience working within healthcare, retail and gaming verticals delivering analytics using industry leading methods and technical design patterns.
STEM ambassador and very active member of the data platform community delivering training and technical sessions at conferences both nationally and internationally.
Father, husband, swimmer, cyclist, runner, blood donor, geek, Lego and Star Wars fan!
View all posts by mrpaulandrew
Hi Paul,
great overview, i would be interessted into Delta Lake Version differences between Databricks and Synapse. Especially now as DBRE 8.0 with 3.1 Spark is already in the preview.
Cheers and thanks,
Gapy
LikeLike
https://mrpaulandrew.com/2021/01/21/how-interchangeable-are-delta-tables-between-azure-databricks-and-azure-synapse-analytics/
LikeLike
SQL-on-demand
* No CI/CD and no build possible!
* No caching mechanism when data did not change
The first is in my opinion a red flag. The second, annoying but can live with it.
LikeLike
You should definitely change the status of Azure Spark Pools from yellow to red. I’m supposed to be implementing Azure Synapse Analytics at a customer and I’ve stumbled across some issues. I could live with the Spark version for now, however there is a much bigger issue: my Spark cluster has no queue. When I call a notebook in a loop, the job aborts if there are not enough nodes available. Why doesn’t it just wait and then start the job? Furthermore, each notebook starts a new session which takes 5 minutes to start.
Also, a lot of features for the delta tables are missing.
We will switch to Mapping Data Flows, which will run hopefully better.
LikeLike
Yes, great points. Happy to update the traffic lights 😊
LikeLike