Vectors in SQLite! (with libSQL)

In this video, I explore how to improve related article recommendations on a website by leveraging SQLite with experimental vector support in libSQL, a package created by Turso. Follow along as we use OpenAI's API to generate embeddings, store them in a database, and calculate nearest neighbors for more relevant content suggestions.

Mentioned Links:

Follow me:

Chapters:
00:00 - Introduction and Website Overview
00:26 - SQLite Database for the Website
00:39 - LibSQL Vector Support Introduction
01:21 - Creating Article Embeddings with OpenAI API
02:32 - Cleaning Article Contents
03:06 - Generating Embeddings from Article Data
04:55 - Storing Embeddings in libSQL
06:00 - Setting up Database for Embeddings
07:00 - Indexing and Storing Embeddings in SQLite
08:45 - Writing SQL Query for Related Articles
10:15 - Querying Article Embeddings for Nearest Neighbors
12:00 - Testing and Fetching Related Articles
14:10 - Improved Related Articles Display
16:00 - Conclusion and Future Plans

[ylynfatt]

I'm interested in learning more about the Laravel Driver for Turso LibSQL.

[uquantum]

Fantastic, thanks for sharing! Love the code-a-long with useful AI-related use of vectors in sqLite!

[OnlinePseudonym]

Great.... Now to learn what an embedding is!

[goaserer]

Wait, this AI stuff can be used for actual useful things. Nice video btw

[nullzeon]

man these vids are so good, ever since you got me into Laravel I've been hooked on this channel (a Turso driver for Laravel sounds sooo cool, hopefully we get a vid 👀)

[stefanadams]

This was great! I'm interested in learning more from you about what vectors are and what that has to do with a database. In general, I think you explained your objective very well and this was a fantastic subject matter using OpenAI. I just struggled to keep up with what OpenAI was used for and what vectors are.
Would you run your OpenAI query and create your indexes on every page load, or would you have a separate scheduled job to do that as needed / on a schedule?

[vedovelli]

I am super eager to start using Turso to power my SaaS but I simply can't because the PHP extension won't work with Herd. That's a bummer. Great video, thank you!

[EinLinuus]

Awesome! Really interested in the driver, definitely want to try this out. Great video as always 🎉

[xryan]

Yes, I've been waiting for a Turso Eloquent driver!

[parfitt]

Fantastic video. Can’t wait to try something like this.

[drantunes]

For a new article, you need re-generate all embeddings? Curiosity: how much did it cost to generate these embeddings?

[curtisgmchale]

I just tried to get the RSS feed from your site and not only is the content old, it isn't even all your content. Is there an RSS feed available that gives me everything?

[ktwalri]

Very interested in a good Laravel Turso driver. The current driver doesn’t work for multiple libsql database connections. The app I’m paying Tighten to develop for me uses a mix of SQLite, Turso and MySQL databases. Turso databases use embedded replicas. Turso databases are per user while SQLite databases are local to the app instance. Laravel really needs a first class driver ready for production in real multi-tenant applications. Looking forward to trying your Turso driver for Laravel.

[karlismillers]

Hi! Fantastic content! I'm definitely interested in the Laravel LibSQL driver.

[villainhd]

Awesome video! 🔥 It's amazing, I've been wanting to use Turso for my project but can't because of the missing driver 😢 Super interested in the driver you're working on! Keep up the great work! 🙌

[maxicms]

Thanks very interested in Turso and your new libsql laravel driver.

[thomas]

Definitely interested in the Laravel driver for libSQL!

[Defqon]

Why would Eloquents' updateOrCreate not work for the upsert in SQLite?

[LenWoodward]

Very interested in the Laravel driver. Shipping some new things soon, and we're already on sqlite, but we want all these libsql benefits in a managed way.

[bobby_builds]

What code editor are you using? It looks really nice and minimal