Airflow Connection connect to Postgres: Airflow Tutorial P10

Airflow Connection connect to Postgres: Airflow Tutorial P10

#Airflow #AirflowTutorial #Coder2j

========== VIDEO CONTENT 📚 ==========
Today I am going to show you what Airflow Connection is and how to use it with Postgres Operator. By watching this video, you will know:
👉 What is Airflow Connection
👉 How to use Postgres Operator to connect with Postgres DB
👉 Best Practice for building Airflow DAG

========== L I N K S 🔗 ==========

========== T I M E S T A M P ⏰ ==========
00:00 - What is Airflow Connection
00:52 - Manage Connections in Airflow Webserver UI
01:45 - Expose the Postgres Database ports
02:11 - Create a database using DBeaver
03:25 - Connect to Postgres
07:28 - Insert values use Postgres Operator
10:08 - Airflow DAG best practice

========== Connect with me 👏 ==========

[coderj]

Comment what Airflow topic do you want to see. I will pick the most liked one and show it in the future episode!👏

[SiDanil]

thanks for this ! would you mind show how can we use sensor on GCS service like google drive or bucket in the airflow?

[arijitchanda]

I literally have become fan of your tutorials. I request you to post a vdo on how to create and manage environments in airflow docker.

[sridhar]

how can we call the stoted procedure for a bq by using tge airflow and data is in postgres data base

[SiDanil]

hi @coder2j

I tried to follow this tutorial and encountered an error in the "test connection" part in dbeaver

it says:

Connection refused (Connection refused)

Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

[vladislavzadvornev]

Great tutorial. Thank you!

I have a problem, though. If I run "select * from pg_catalog.pg_database pd ;" I can see that both "airflow" and "test" databases exist. And I can perform all the necessary actions with "airflow" user. However, I can't see the required databases ("test" and "airflow") in dbeaver. Do you know, what can be the issue here?

[dhruvsingla]

Thanks, was stuck at host. I kept trying localhost. Can you tell where you found what value to put in host?

[eunheechoi]

can you please show how to connect to Oracle which is missing in connection in UI?

[bokistotel]

Hey great video. But I can't find nothing online on how to read from postresql database that is located on another server?? Is there a way?

[JDeSilentio]

Really good videos, like and subscribed! Quick question - when I attempt the dbeaver connection I'm getting the following error: FATAL: password authentication failed for user "airflow". But I am using the airflow user/pw per the video and yaml file, do you happen to know what's going on here?

[introvertmind]

Is there way we can save the data fetch from postgress using pghook to txt file and upload it to s3 bucket

[anketambekar]

How to configure AWS Elastic cache in airflow

[ashishvats]

I have a problem, when i run my dag for JDBC connection, i getting this error:

JClass requires str or java.lang.Class instance, not 'NoneType'

Could you please help me in this issue

[shafickrahman]

I'm getting below error while creating the conn id . can you kindly help with it ??
connection to server on socket failed: No such file or directory Is the server running locally and accepting connections on that socket?

[MozgopravTV]

Thank you for the video. However, "host.docker.internal" doesn't work because I'm using Linux.
Gives an error: could not translate host name "host.docker.internal" to address: Name or service not known
What hostname should I use? Name "localhost" is not working. Postgres db deployed via docker

[matiasleonel]

Hi, I'm having problem when importing Postgres operators using and another one when conecting to host.docker.internal... I'm using Linux.

[anhkhoaang]

I 've tried use the IP address for external database but it won't work for me. I'm doing it on azure machine.. Does it have to open the port to connect ??

[syedanwar]

Hi Im getting error as Negsignal.Sigkill in the postgres to s3 operator, kindly helpe on this

[jhoanmartinezsilva]

How can I insert a pandas dataframe into mysql database in airflow

[onyedikachiadigwe]

Hi, what happens if we close down the airflow services on docker. Do we loose the database that was created (Test) and if so how do we overcome this challenge. Sorry i am asking because i just started using airflow.