Intro To Data Orchestration With Airflow

[dataecstasy]

what would be the github integration look like for if companies want to keep the dag code on github

[waterhill]

Can Airflow be used to orchestrate a spark streaming YARN job that pulls data from Kafka and writes to HDFS?.. the idea is if the spark streaming job queues and it can be monitored/alerted/detected and restarted automatically by Airflow?

[jugsma]

If possible, Could you share with us the repo (github) link of Dag's