The Microsoft Intelligent Data Platform

My Thoughts Following MSBuild 2022

The MSBuild conference was excellent. The effort Microsoft put into making the experience work across time zones, in different languages and hybrid (in person/online) so very evident. My hat off to the organisers!

The Data Annoucements

During the keynote at MSBuild we were introduced to the Microsoft Intelligent Data Platform. We know what the Microsoft Data Platform is. What now makes it intelligent?

Satya Nadella’s keynote speech from MSBuild 2022

Sorry Satya, I didn’t capture the best facial expression. Or maybe I did! Are you saying ‘Oooo, check out my Intelligent Data Platform!’ 🙂

This was supported immediately by Microsoft Purview (recently renamed from Azure Purview).

If you missed MS Build 2022, check out the following link for publicly available on demand videos and content. Otherwise, you will need to register to get other session recordings etc.

https://news.microsoft.com/build2022/

Also, here is the direct link to the Intelligent Data Platform blog:

https://azure.microsoft.com/en-us/blog/introducing-the-microsoft-intelligent-data-platform

An Evolution of the Data Platform

So, let’s think about this as an evolution of data product offerings from Microsoft and others, mainly Databricks. My personal timeline of icons looks a little bit like the below (because I’m older than some, but younger than others). I’ve included a few significant dates, but not everything. You get the idea when thinking about the technical journey we’ve been on and continue to ride, aligned to the Microsoft mothership. Hopefully, the iconography versioning gives you some context as well.

Click to enlarge

I pondered this view for a while after drawing it. Then realised that if I drew the inverse of this it would represent a very obvious increase in technical capabilities offered as data products mature, leading to the intelligence we now desire. Something like this, perhaps…

As a side note, I left off the Purview icon because the launch of the product is very blurred in my mind. It was once Data Catalogue, then Data Catalogue v2, then Microsoft acquired BlueTalon in July 2019 etc etc. Some will know this story well. Purview and general data governance implicit throughout.

Anyway, what comes next? As an ultimate user of the Microsoft platform, I’ve been thinking about several things here and ways forward given what MSBuild 2022 delivered.

Are Microsoft now lighting up technical capabilities across the product stack that means we will soon get an end-to-end integrated experience for all the great data offerings. Was/is the unified experience of Azure Synapse Analytics just the beginning? It feels like Microsoft have a much bigger picture in mind for the new ‘Intelligent Data Platform’.
MSBuild 2022 gave us more Synapse Link capabilities, this time for SQL Server 2022, similar the existing Cosmos DB integration. This means data ingestion pipelines could become a thing of the past in favour of an almost turnkey enabled analytics feed for our OLTP datasets. Lots of room for improvement and lots more source database products will need to support this, but a roadmap becomes clear for this.
Data Marts for Power BI allows business users and data professionals to easily create/explore source datasets in a no-code/low-code environment. All using a nice browser-based interface. Could this mean an end to the massive industry dependency on data engineering skills?
We heard how Microsoft Azure super computers trained the AI Codex model meaning natural language can now be used to write code! With demonstrations at MSBuild instructing Minecraft bots to perform tasks, building virtual metaverse classrooms and drawing complex 3D visuals of things like the solar system. What else is just around the corner that natural language can now be used for?
As a final thought, will the principals of a data mesh architecture become common place for this intelligent platform and even embedded into the technology delivered?

Adding to this, in the other MSBuild sessions Rohan Kumar talked about the Intelligent Data Platform in the context of the wider Microsoft Cloud. Interestingly that had the Power Platform separate to the Data Platform, although Power BI is listed in the Data Platform. Probably just a marketing mix up! Or is the Data Platform a layer below the Power Platform? Or am I just reading the picture wrong?

Current Conclusion

Microsoft aren’t slowing down. Microsoft CTO Kevin Scott said the same when interviewed. They clearly have a plan and theme for the various product offerings coming together to make everything we do more intelligent. As the saying goes, work smarter not harder. When will I no longer need a keyboard for development and can speak to my computer for everything, Star Trek style? It feels like it’s now on the horizon.

The side effects of this are that the pace of change in the industry is really becoming nauseating. Anti-sickness medication almost certainly needed for those just starting out.

Lots more to come here as Microsoft reveal more about the Intelligent Data Platform.

Many thanks for reading.