Run PySpark Job using Airflow | Apache Airflow Practical Tutorial |Part 4|Data Making|DM| DataMaking

Hi Friends, Good morning/evening.

Do you need a FREE Apache Spark and Hadoop VM for practice?

Happy Learning!

================================================================================

[mosa]

Thank you for this great introduction, good covering of the different important parts!

[siddharthat]

how to setup the connection if the spark cluster is not local, there is no option to enter username and password except host and port in the UI

[harshaaleti]

Thanks for the video. You didn't talk about the spark installation. Without that part, how can we submit the jobs?

[abhisekchowdhury]

Really nice video.
Is it possible to config python operator similar way so that the python code runs in a separate vm than the airflow's python

[mateen]

Thanks so much. It is really helpful ! Can you also please make videos on dag to dag dependencies, sensors, email operators in airflow?

[swagatdash]

Can you please share the code. Thanks for making such nice video

[vamshi]

Hi Datamaking, thanks for nice video.
I want to build a spark jar and run that in airflow, is it possible??

[pragtyagi]

What if i want to schedule a Azure Databricks notebook via airflow . Any suggestions ?

[osmannassar]

Without code this tutorial not benefit. Please provide us with the code

[rahulbhatia]

Hi, nice demonstration!, could you please provide the code you have used in this video?

[akramkhan]

Improve your English to prove your skills