Apache Airflow Job in Microsoft Fabric

Apache Airflow job in Microsoft Fabric provides an integrated, cloud-based platform for development, scheduling, and monitoring python-based data workflows, articulated as Directed Acyclic Graphs (DAGs). This innovation delivers a Software-as-a-Service (SaaS) experience for data pipeline development and management using Apache Airflow, making Apache Airflow runtime readily accessible for the development and operationalization of your data workflows.



🎙 Meet the Speakers:

👤 Guest Expert: Abhishek Narain, Principal Program Manager in Microsoft. Abhishek serves as a Product Manager at Microsoft Fabric's Data Factory. His primary area of focus is Data Factory, a tool that empowers data engineers to construct scalable, robust, and reliable ETLs.

👤 Host: Estera Kot, Principal Product Manager at Microsoft.

👍 Liked this video? Don't forget to hit the 'Like' button and share it with your peers!

[keenfive]

sounds awesome!
Two questions:
1) will this functionality also come to Azure Synapse?
2) Can I execute CopyActivity from Airflow (without wrapping it in a Data Pipeline)?

[Eklandasy]

Thank you for covering all relevant questions!
I tried Fabric IDE and able to run Fabric activities via DAGs. However with ADO storge, seems DAGs are not recognized in airflow. Tried different connection options (Service principal, PAT etc) and confirmed folder structure that we have dags->--.py.
Not sure if I missed something.