Writing My Own Database From Scratch

First episode of the #mkown series.

In this video I write my own SQL database from scratch without using any libraries or third party dependencies. Only the standard library provided by the chosen programming language is allowed (Rust in this case). The end result is a basic ACID compliant database that can process only one transaction at a time, something similar to the first versions of SQLite from the early 2000s, but with less features (and probably much worse performance).

The goal of these videos is to provide a high level overview of how such complicated systems work internally, not explaining all the code in detail as that would require dozens of hours of content.

🌐 LINKS

CMU Intro to Database Systems 2023:

SQLite 2.8.1 Source Code:

✉️ CONTACT INFO

🎵 MUSIC

🕖 Time Lapses

🔉 Background

📖 CHAPTERS

00:00 What is Mkown?
00:37 What Do We Know About Databases?
02:18 Research Time Lapse
02:53 Initial Ideas
03:04 Naive Storage Format
05:31 Why Binary Trees Are Not Enough For Indexes
07:46 Why B-Trees Are Better Than Binary Trees
12:21 Software Architecture
13:17 Dev Time Lapse
14:13 Project Demo
18:23 Database Internals
19:24 File System Structure
20:31 Slotted Pages
23:45 Rows, Tables And Indexes
26:50 Overflow Pages
28:30 Sophisticated BTree Balancing Algorithm
29:53 Parsing and AST (Abstract Syntax Tree)
31:54 Query Planner
34:23 K-Way External Merge Sort
36:50 Cache Replacement Algorithm
38:19 Transactions: Commit & Rollback
39:27 Network Protocol
39:56 Final Thoughts

🏷️ HASHTAGS

#programming
#computerscience

[BearIsAwesome]

Finally some real software engineer who is making cool stuff. YouTube is flooded with too many leetcode tutorials.. finally someone is making cool stuff..

[WarchantUA]

First video on a channel is a 7-month project - you're a legend. Please continue!

[churchers]

Brave. Databases are one of those areas of computer science that gifted experts spend their entire career on

[lost_martian]

This was such a hidden gem.. lucky this came into my recommendation

[yashvardhan]

Finally found a real software engineer on YouTube who builds cool stuff.

[Karim-lndw]

It's been 4 weeks since you released your video. Man, it is unfair that you have only 1.8k views. If I stumbled upon this video, then you will find your viewers. Keep doing this!!

[joshpauline]

huge respect here, I am a rust engineer and its hard to find someone on YT actually building cool shit and not click-baiting just to create a todo application

[_jfsanchez_]

It is so sad that the Spanish community mostly talks about HTML, CSS, JS and PHP. So happy to see your new channel and such a great content. Subscribed.

[yerpery]

Junior SWE here. This was such a fun watch. Your channel is severely underrated!

[ivgadev]

Gracias Antonio! Estupenda iniciativa, algunos estábamos aburridos ya del nivel de habla hispana donde solo se habla de cosas muy básicas. Me quedo por aquí para seguirlo de cerca. Un abrazo.

[chturne]

This is fantastic. Great production quality, no nonsense clear explanations. A rare gem to find in programmer YouTube. Congratulations on completing your project. I hope you'll continue to do it and be well rewarded for it!

[la-dev]

Probably at 0:15 I have paused the video, opened your channel in new tab. Subscribed, and back here. No one is there to touch these topics. I salute you for sharing your knowledge.

[mikejohneviota]

very rare to see a youtuber that use low level programming for content ❤🎉

im your new subscriber

[agusaris]

Now this is real software engineering content that I'm looking for. Database engineers are on different level

[melajodwan]

i've been wandering around youtube for a while but wait, ....how did i fall into this hidden treasure !?

[Mirino]

Not only is this actual, real software engineering, as other comments have made it clear, but your teaching method is incredible! You've managed to make me follow you on an incredibly high level technical conversation even though my current technical understanding is that of a measly junior web dev.

I want you to know that every single second you spent creating all these visualizations, explanations and teaching patterns was well used and highly, highly appreciated. I expect many of those seconds were painful, as I have some video editing experience myself, but for every one of your sweat drops, the quality of this video sky rocketed! Thank you!

[JuanGM]

Te sigo desde que empezaste la carrera, en el otro canal, but this is another level, ni si quiera me di cuenta que te lo habias creado. La explicación y el tema top, como siempre!

[flodderr]

random recommended video but it explains perfectly each concept and reasons behind decisions. Hidden gem!

[favoring-quiet-goatskin]

Wow, this is really something. You’re incredibly talented, Tony. Look forward to seeing what else you create.

[AlanDarkworld]

Good job! I did my PhD in this field so I know how hard this journey is. Today I don't even bother with B-Trees anymore. The scattershot of updates you get from rotations kill write performance and they're very poor for compression as data may change everywhere without warning. Pretty much every major DB developed since 2012 uses LSM (Log Structured Merge) Trees. They have their own issues of course but no more stupid rotations! I invite you to take a closer look at them, they're a really cool data structure.