DuckDB: Hi-performance SQL queries on pandas dataframe (Python)

DuckDB calls itself the "SQLite for Analytics" and is built to facilitate fast analytical query workload directly within the host process (for data scientists, read: R or Python). The DuckDB creator himself cites ease-of-use as a major factor in designing DuckDB as a database management system (DBMS) that data scientists don't hate to use.

Timestamps:
00:00 What is DuckDB? (Intro, Why DuckDB? Overview)
03:00 Mental Models: How database interface with applications
13:37 DuckDB intro / quick tour to DuckDB shell
28:42 DuckDB python package (usage w/ pandas)
36:02 Multiline SQL query on a pandas dataframe!
42:19 Tools for working with DuckDB

Other tutorials mentioned in the video:

1. Convert CSV in SQLite in less than 5 lines of code:

2. Hands on introduction to SQL

3. Running SQLite in VSCode:

4. Building Dashboards with PyScript + Altair (⌚ -🏃 | Part 1) using running data from Mi Band / MiFit fitness band

[GabsFerreira]

Thank you so much for this content, Samuel! I'm getting started with DuckDB and it helped me a lot.

[uoitauz]

very nice video. haven't expected it from such a small channel. I really liked the summary and visualisatzion of the other recources in the beginning

[pritamdodeja]

Excellent content! Your channel is bound to grow if you keep putting out stuff like this!

[AdamCysec]

very insightful and informational video! I plan to use duckDB in my next project.

[torque]

Great video! Thank you for the timestamps and sql against pandas!

[Davidkiania]

This is amazing learned so much and clarified alot of the fundermental knowledge. Much appreciated.

[buggingbee]

This is what I have been looking for. As a python user, I always wonder why do you need sql when pandas can do all the work with fewer lines?

And I think duckdb is the point that both sides could use the best of both worlds.

Been watching your video until finish, and wonder if there is a script to turn dataframe to sql table? Would love to see that work

[miltondiaz]

Excelente. A+1. Se puede conectar un base de datos de Foxpro + DuckDB con python

[sebas-]

Is there a way to connect duckdb to database so as not to use pandas functions 'pd.read_sql()' and 'pd.to_sql()' to improve execution times in jupyter notebooks?