Last Updated – 28th may 2021
Following my previous (more detailed) blog post on ‘Is Azure Synapse Analytics Ready for Production?‘ I was asked to summarize this understanding/opinion of Synapse into a simpler view.
Updated: I’ve tweaked a couple of the traffic lights and narrative based on the annoucements this week from the Microsoft Build conference. The main change being the support for Delta Tables from the SQL Serverless compute.
As a PDF if you prefer here.
Discuss 🙂
Thanks
Hi Paul,
great overview, i would be interessted into Delta Lake Version differences between Databricks and Synapse. Especially now as DBRE 8.0 with 3.1 Spark is already in the preview.
Cheers and thanks,
Gapy
LikeLike
https://mrpaulandrew.com/2021/01/21/how-interchangeable-are-delta-tables-between-azure-databricks-and-azure-synapse-analytics/
LikeLike
SQL-on-demand
* No CI/CD and no build possible!
* No caching mechanism when data did not change
The first is in my opinion a red flag. The second, annoying but can live with it.
LikeLike
You should definitely change the status of Azure Spark Pools from yellow to red. I’m supposed to be implementing Azure Synapse Analytics at a customer and I’ve stumbled across some issues. I could live with the Spark version for now, however there is a much bigger issue: my Spark cluster has no queue. When I call a notebook in a loop, the job aborts if there are not enough nodes available. Why doesn’t it just wait and then start the job? Furthermore, each notebook starts a new session which takes 5 minutes to start.
Also, a lot of features for the delta tables are missing.
We will switch to Mapping Data Flows, which will run hopefully better.
LikeLike
Yes, great points. Happy to update the traffic lights 😊
LikeLike