Airflow 2.0 Series - TaskFlow API - PART 4

Welcome in Airflow 2.0 series!

My name is Marc Lamberti, head of customer training at Astronomer. As you know, creating DAGs in Airlfow is time consuming. Even more when you manipulate XCOMs as you need to explicitly call xcom_pull and xcom_push. Wouldn't be nice to have a faster and easier way to create DAGs? Here is the TaskFlow API! Let's discover this!

Link to follow the video:

Content:
00:00 - Introduction
01:07 - How was it before (in Airflow 1.10.X)
02:22 - Introduction to the TaskFlow API
03:42 - Set up and run Airflow 2.0 with the Astro CLI
04:55 - Creating a DAG, the classic way
07:09 - Creating a DAG with the TaskFlow API, the new way
09:21 - Running the DAGs in Airflow 2.0
11:32 - What you've learned

Enjoy!

[viniciusdossantosmello]

Hi Marc, thank you for giving some details about this amazing feature! I have one question, how can we use this new method of creating DAGs and also have branchs that pull data from the same task?

[TheRn]

Is there some easy way to delete the xcom data after use?

[ashwinkumar]

How to call snowflake stored procedures from a python file when all operators are declared in DAG file

[alexfernandodossantossilva]

Hi Marc, does Taskflow allow us to share multiple xcoms between tasks? How would it be?

[garp]

most shocking thing is the price of bitcoin.