A Day Full of Azure Data Factory

Join me for a full day of training on Azure Data Factory.

If we want to achieve any data processing in Azure you need an umbrella service to manage, monitor and schedule your solution. For a long time when working on premises, the SQL Agent has been our go-to tool, combined with T-SQL and SSIS packages. It’s now time to upgrade our skills and start using cloud native services to achieve the same thing on the Microsoft Cloud Platform. Within a PaaS only Modern Data Platform, the primary component for delivering that orchestration is Azure Data Factory, combined with various other compute resources.

In this full day of training we’ll start with the basics and learn how to orchestrate your Azure Data Platform end to end. You will learn how to build our Azure ETL/ELT pipelines using all Data Factory has to offer. Plus, consider hybrid architectures, dynamic design patterns, think about lifting and shifting legacy packages, and explore complex bootstrapping to orchestrate everything within your solution.


Complete Session Agenda

  • Module 1: Data Factory Fundamentals
    • What is it and why use it?
    • Resource Components
    • Common Activities
    • Execution Dependencies
  • Module 2: Uploading Data to Azure
    • Integration Runtimes
    • Hosted IR Patterns
    • Private Endpoints
  • Module 3: Using SSIS Packages in Azure
    • SSIS Integration Runtime
    • Packages Running on PaaS
    • Scaling Out Package Execution
  • Module 4: Data Flows
    • Mapping Data Flows
    • Wrangling Data Flows
    • Configuration
    • Use Cases
  • Module 5: Metadata Driven Pipelines
    • Expressions
    • Dynamic Pipelines
    • Orchestration Framework – ADFprocfwk.com
  • Module 6: Monitoring Alerting Security
    • Logging – Kusto Queries
    • Alerting
    • Roll Based Access
    • Using Azure Key Vault
  • Module 7: Pricing & Limitations
    • Data Integration Units
    • Data Flow Compute
    • Wider Platform Orchestration & Cost Control
    • Resource Limitations
  • Module 8: CI/CD with Azure DevOps
    • Source Control vs Developer UI
    • ARM Template Deployments
    • PowerShell Deployments
  • Module 9: Data Factory in Production
    • Testing
    • Bootstrapping
    • Best Practices
  • Module 10: Wrap Up
    • Conclusions
    • Questions
    • Homework

If that’s not enough content for one day, you will also get access to a set of hands-on labs that you can work through at your own pace. Whether you are new to Azure Data Factory or have some experience, you will leave this workshop with new skills and ideas for your projects.

4 thoughts on “A Day Full of Azure Data Factory

    1. Sorry, probably not. There is already a 1 hour intro session on ADF available via my YouTube channel though. Maybe start with that if you think it’s a resource students would use.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.