procfwk is Getting An Upgrade to CF.Cumulus

CF.Cumulus.Control Early Release

Running a company (Cloud Formations) is very much a balance between steering the ship, while building the ship! As a result, my open source metadata driven orchestration framework (procfwk.com) got neglected. To the point where the Azure Functions, the middleware, originally written in .Net 3.1 Core is no longer supported. Additionally, the Data Factory NuGet packages have been deprecated! So, my friends, over the Christmas 2023 period and during the January New Year slow down it was time to show the framework some love. Also, who knew the open-source project and hobby was going to run for this long, receiving a lot of attention and use in production! Definitely a proud moment.

Today, it gives me great pleasure to tell you that there is a lot of new designs and plans for the framework in the pipeline. Pun intended 🙂

The framework will evolve to form the control flow component of a larger product and data platform accelerator.

---

As an initial set of steps for the “legacy” ProcFwk control flow components and to get our house in order, we have:

• Forked the repository from my GitHub account, under mrpaulandrew to Cloud Formations and renamed the namespaces.
• Upgraded the Azure Functions all the way to .Net 8. Covering Azure Data Factory and Azure Synapse Analytics interactions.
• Implemented a new Microsoft Fabric service and helper classes ready to manage ‘Fabric Pipelines’ once the APIs become available.
• Refactored the authentication to only use Azure Managed Identities for improved and simplified security.
• Stripped out a lot of old helper schema code from the database deployment and simplified the initial set of metadata required.
• Overhauled the solution structure ready for expansion as the other parts of the product arrive.

Then to align with the motivations and branding for Cloud Formations (CF) we have renamed the entire product to Cumulus.

ProcFwk has therefore become CF.Cumulus.Control

---

Cumulus as a product will consist of the following metadata driven components:

• Control
• Ingest
• Transform

Importantly, the entire CF.Cumulus product will also be an open-source code base, under the same MIT licence. Following in the footsteps of my ProcFwk. Community collaboration, openness, and disruptive thinking in everything we do. See Our Vision blog post for more about the culture of Cloud Formations.

Going forward, please check out our new product landing page for CF.Cumulus via the link below and stay tuned for more content via the Cloud Formations blog.

https://www.cloudformations.org/cumulus

---

Many thanks for reading.