Airflow DAG Factory: Create DAGs dynamically with YAML

Airflow DAG Factory: Create DAGs dynamically with YAML

In this video, you will learn:
✅ How to create DAGs dynamically with YAML
✅ How to construct DAGs without knowing Python
✅ How to avoid duplicate code
✅ How to transform a DAG into a YAML file
✅ How to use dynamic task mapping in YAML
✅ How to use Datasets in YAML
✅ How to run Airflow in local with the Astro CLI

and much more!

Enjoy

[parthchoudhary]

I just input in csv and dag is generated,
Another dags reads that csv and generate a new dag in gcs.

[NovasVilla]

Q: Can using `dagfactory` lead to performance issues in the scheduler, similar to when dynamically publishing a list of DAGs to `globals()`?

[SaxonChuang]

I have no idea why peoples want want to do this, in fact you still have to write python script.