Pandas Dataframes and SQL [How to write dataframes into a sql database/get sql table to dataframe]

In this video I am showing how to get sql data into Pandas dataframes and also how to store dataframes into sql databases. I am working with SQLite in this video.

Get the Notebook/Source code by becoming a Tier-2 Channel member:

More information for other Database systems (e.g. MySQL or MS SQL server or Postgres):

Check out my introductory video on Python and SQL:

Please be so kind subscribing and liking the video in case it was helpful.

0:00 - 0:55 Intro
0:55 - 02:08 Create DB engine (read description for other DB)
02:08 - 03:49 read_sql (Both whole table and SQL syntax possible)
03:49 - 08:32 to_sql (and possible errors/mistakes/pains)
08:32 - 10:20 Pull and store financial data in a sql db
10:20 - 11:04 Outro / Potential topics

#Pandas #SQL #Python

[jessedayan]

Thank you! Looking forward to working through all of your videos!

[ihebbibani]

I really really appreciate your videos. I love you accent, how you explain, your method ..everything is great. Already, subscribed and I am sharing your videos ;)

[xilllllix]

EXACTLY what i was looking for, thank you!

[davidtillery]

Thank you! your videos have been a big help to get my Tranding Bot jurney started!

[scrambled_egggs]

Thank you. You gave me courage to challenge my project!

[jitkadyan]

Another informative video sir, thanks and also very thankful to you replying our comments and providing information as suggested by us by spending your precious time to create such video, we are looking for more video, thanks again sir.

[AbCdEf-vhjj]

I can't tell you how enjoyable it is to listen and practice to your vids! You're doing a fantastic job and enrich every participant with the huge amount of learnings you offer us enthusiasts :)
I will use your learnings for my own db (also retrieved from yfinance). 2 questions occured, which I would like to ask you (your answers would be very much appreciated):
(1) When combining a database of Japanese or Chinese stocks with American or European stocks, the trading days are different. How do you personally go for database calculations like the 3M performance with stocks with diverging dates (used as indices)?
(2) Would it be possible to say sth like: I would like to store the price data from 2000-01-01 to 2021-12-01 and then only append the price data of that very day? Through the help of your video, it seems doable. What's your opinion on that?
Thank you so much for your great community contribution, Algo! :-)

[hicarlee]

gr8 upload and understandings for pandas!!

[sz]

Another great video..thank you! In case anyone gets stuck at 7:17..appending df1 with error message reading "database is locked"...just restart your PC and try again. Seems to fix the issue.

[vazquezlunadiegojose]

Thanks, your videos are great
What if want to add primary or foreign keys to the columns ?

[oczko]

i did the "replace mistake" today! :D

[styvkygo]

You are providing very helpful content; I have been able to fix some issues in my codes thanks to you. I really appreciate what you are doing.
I have a question regarding resampling data if you do not mind assisting me on this please. what would be the best way to resample downloaded stock data from say 1m to 5m or 15m? Cheers

[chinmoybasak]

Thank you! for sharing! But How can we establish a relationship(keys) in between the imported tables? Would really appreciate it!

[Kig_Ama]

9:14 How would u download the fundamental data of GME into the db?

[mrunalcavale]

Hi, this was really helpful. I also wanted to know how to upsert data into sql server. Can you share some info about that? My requirement is, the table cannot be truncated at any point and therefore I cannot do truncate and load. I need to either update or insert the records into DB. Your suggestion is deeply appreciated.

[cvillejin]

Nice video and tutoria!. Do you know how I can do this with SQL Server? I don't really know what SQLlite and all this technical; stuff is, but I tried creating this on Spyder and I can't re-create the same results. I use SQL Server and SSMS. Your help would be greatly appreciated!

Edit: specifically, I am trying to recreate the part where you use yfinance to import the live ticker data into a SQL Server database that is made in SSMS

[felixbartschi]

Thanks for the nice tutorial.
Is there a way to update rows without defining a SQL schema?
something like: df.to_sql('table', engine, if_exists='update)?
I do have a .csv files with +800 columns writing sql update statement is a pain i have seen the ubsert solution but it includes SQL as well.

[nguyenduyta]

Thank so much. By the way I would like to creat a GUI that user can sign up and log in and have profiles of users to database with portfolio management. Please tell me if you have this course

[guilhermerezende]

in matters of time and processing, is that way batter than using pyodbc?

[lilymaths]

Hello there, thank you for uploading! I am struggling with loading my pandas data frame with millions of rows into my Postgres database and I keep having issues! Would you be able to help me?