Airflow DAG: create dag with bash operator!

Airflow DAG: create dag with bash operator!

#Airflow #AirflowTutorial #Coder2j

========== VIDEO CONTENT 📚 ==========
Today I am going to show you how to create our first dag using the bash operator! By watching this video, you will know:
👉 How to create a dag and set parameters as your need
👉 Define tasks using BashOperator
👉 How to build the task dependencies

You don't have a running Airflow in your local setup? Check out the Apache Airflow 2.0 Installation Tutorials.

========== L I N K S 🔗 ==========

========== T I M E S T A M P ⏰ ==========
00:00 - Launch airflow using docker
01:08 - Remove all the airflow example dags
02:16 - Create our first dag and set the parameters
05:00 - Create dag tasks using bash operator
07:26 - Trigger our first dag run
08:05 - Define multiple tasks for the dag
10:15 - Three different ways to build tasks dependencies

========== Connect with me 👏 ==========

[helovesdata]

I really enjoyed this video. I learned more today than the past two weeks with another youtube video. Also I reinstalled Airflow with your instructions from another video and it has never been this stable. Thank you!

[JoyantoGhoshJG]

I am following your instructions and have been able to understand Airflow at an intermediate level.
I would love to see your writing DAGs for PythonOperator with Xcoms. Eagerly waiting for it!

[manimaran.v]

Nice AirFlow video ever in you Tube, heartfelt Thanks

[brunodupont]

Amazing videos, very simply explained

[khalidsultani]

Thanks for posting far easy and clearly explained then paid material . Please keep postings if possible on airflow integeration with aws services such as red shift and databricks .

[caiofelixreis]

Awsomee!!! Thank you for all this broo, its really helpful

[Akashkumar-eitm]

Nicee .. please create one for passing one variable from 1st function to another function using these python operators

[bimerap]

Please include how we can install more python package?

[VitaliyZlobin]

Hello) Cool videos! Thank you!
Why is the old version of DAG not rebooting, but a new one is created?
And how to remove old versions, leaving only the latter?

[Akshay-dnni]

i am getting error as- Import "airflow" could not be resolved. possible solutions plz

[follygee]

how do i import a json config file that is dependent on another python script in airflow

[universedimension]

大哥，用visual studio编辑DAG，然后在airflow portal

[hayathbasha]

Hi is it possible to get task/dag duration from airflow context

[simre]

Good stuff! Thanks for the tuto! I am running the dag which is status success but the task 1, 2, and 3 are blanks and if I select one task to run it independently, it tells me ''Only works with the Celery or Kubernetes executors, sorry'' any clues how to fix this? I should add some settings in the .yaml file?

[akmalhafiz]

when I try to import airflow, it says that "Import "airflow" could not be resolved". I looked up over the internet and it seems like the problem is my local python interpreter doesnt have airflow installed. How can I use python interpretor from the container instead in visual studio code so that I wont get this error?

[pythonmathui]

Sir the airflow webserver has already running but when i tried to import DAG from airflow it says no module named airflow, should i just pip install it? or did i miss something that make the airflow module not automatically installed?

[dhanasekar]

Hi, really good videos. Shall I know how to dm you. I want to ask a doubt regarding sql sensor and hive operator

[vilw]

Hi, am using bash operator and passing multiple commands inside bash_command.Its a mixture of if else loops and commands.I don't know how to correctly incorporate it.l am using" to separate it. Could you guide me here if you have worked with something similar please thanks in advance

[trantheluc]

i use BashOperator run bash_command='date > ./date.txt' but when i run task i cannot find 'date.txt' in folder. so where is it?

[Uxdkkkhcddvbh]

Hi this has been really helpful. However the error in the logfiles I get when I run my dag is that says it doesn't have access to my python file - INFO - python: can't open file '. I am using the Bash Operator and the bash command is 'python how can I fix this. I have tried all version of the writing the full path. How do I give airflow permission?