Apache Airflow for Data Science #10 - How to Work with Variables

preview_player
Показать описание
Hardcoding values in your Airflow DAGs is a bad practice. Learn how to use Airflow variables instead.

00:00 Introduction
00:37 How to add Variables in Airflow through the UI
02:02 How to add Variables in Airflow through the Terminal
04:02 How to access Airflow Variables from the DAG
08:14 Outro

FOLLOW BETTER DATA SCIENCE

FREE “LEARN DATA SCIENCE MASTERPLAN” EBOOK

GEAR I USE
  1. Better Data Science
  2. apache airflow
  3. airflow
  4. data engineering
  5. python
  6. programming
Рекомендации по теме
Axel Goblet: Scheduling 0:14:06

Axel Goblet: Scheduling machine learning pipelines using Apache Airflow | PyData Eindhoven 2019

Talk: Daniel Imberman 0:15:52

Talk: Daniel Imberman - Bridging Data Science and Data Infrastructure with Apache Airflow

Luther Hill: Use 0:38:03

Luther Hill: Use Apache-Airflow to Build Data Workflows | PyData Indy 2019

Data Science Platform 0:44:24

Data Science Platform at PlayStation and Apache Airflow

Learn Apache Airflow 0:12:38

Learn Apache Airflow in 10 Minutes | High-Paying Skills for Data Engineers

Daniel Imberman & 0:33:47

Daniel Imberman & Gonzalo Diaz: Data Products, From 0 to data science pipeline | PyData Córdoba

Building the Data 0:29:31

Building the Data Science Platform with Airflow at Near

Apache Airflow for 0:11:05

Apache Airflow for Data Science #7 - How to Work with REST APIs

Apache Airflow for 0:08:57

Apache Airflow for Data Science #10 - How to Work with Variables

Apache Airflow at 0:16:02

Apache Airflow at Wise

Apache Airflow for 0:06:47

Apache Airflow for Data Science #4 - Migrate Airflow MetaData DB to Postgres and Enable Parallelism

Apache Airflow for 0:09:51

Apache Airflow for Data Science #1 - How to Install Airflow Locally in 10 Minutes

Building (Better) Data 0:11:41

Building (Better) Data Pipelines with Apache Airflow

Simplifying the Creation 0:25:39

Simplifying the Creation of Data Science Pipelines with Airflow

Apache Airflow Meetup: 0:25:49

Apache Airflow Meetup: Elegant Data Pipelining with Apache Airflow

Apache Airflow for 0:10:20

Apache Airflow for Data Science #3 - How to work With Databases (Postgres)

Data Science at 0:24:32

Data Science at Scale: From Notebook to Production with Airflow and Ray

Apache Airflow for 0:00:57

Apache Airflow for Data Engineering | Apache Airflow | Data Engineering | Intellipaat #Shorts

Apache Airflow for 0:18:23

Apache Airflow for Data Science #2 - How to Write Your First Airflow DAG (Data Pipeline)

Apache Airflow for 0:13:38

Apache Airflow for Data Science #5 - How to Run Tasks in Parallel

Airflow tutorial for 0:32:25

Airflow tutorial for beginners | Airflow tutorial python | Airflow tutorial 1

Machine Learning with 0:23:54

Machine Learning with Apache Airflow

Twitter Data Pipeline 0:40:21

Twitter Data Pipeline using Airflow for Beginners | Data Engineering Project

Automating Data Pipeline 0:25:44

Automating Data Pipeline using Apache Airflow by Mridu Bhatnagar

Комментарии
Автор

Hello, Please do video about performance tuning on airflow.
Handling the boilerplate, adding plugins and using python imports from files for templates.

chriskathumbi
Автор

Please upload the end to end machine learning project in airflow. Like ingesting data, feature selection, model trainning on various ml algorithms, hyperpameter tunning, model selection, model saving. I searched whole internet but not able to find it. Please upload one large video sir. It will be very helpful.

SanjeevKumar-ncrt