Pinecone vs FAISS vs pgvector + OpenAI Embeddings

Comparing 3 vector databases - Pinecone, FAISS and pgvector in combination with OpenAI Embeddings for the semantic search.

Please find the corresponding Google Colabs:

[olcaybuyan]

MongoDB and Qdrant would be great additions to this test :) Great video! Thanks

[_timestamp]

🎯 Key Takeaways for quick navigation:

00:00 🌐 *Overview of Semantic Search with OpenAI Embeddings*
- Introduction to three setups for semantic search using OpenAI embeddings: Pinecone, FAISS, and pgvector.
- Overview of the project "dry" aimed at solving repetitive questions in chatbots using semantic search.
- Explanation of the schema, involving generating vectors, searching the vector database, and storing metadata for incoming messages.
02:26 🧠 *Understanding Semantic Search and Embeddings*
- Definition of embeddings models, with a focus on OpenAI's text embeddings ada002.
- Explanation of how embeddings models generate dense vectors representing words, sentences, or paragraphs.
- Clarification of how the multi-dimensional space in the model reflects the semantic meaning of different strings.
05:24 🍍 *Pinecone Integration*
- Introduction to Pinecone as a fully managed Vector database.
- Demonstration of setting up a project, specifying environment variables, and creating a Pinecone index.
- Implementation of Pinecone integration in a Google Colab notebook, including generating vectors, saving them, and performing similarity searches.
16:24 💲 *Evaluation of Pinecone*
- Efficiency assessment of Pinecone as a managed service for vector storage.
- Low effort needed for integration with Pinecone due to available Python and TypeScript examples.
- Consideration of costs, with the expectation of a $70/month plan being sufficient for a pet project but potentially expensive for larger-scale commercial use.
18:23 📚 *FAISS Integration*
- Introduction to FAISS as a library for efficient similarity search of dense vectors.
- Noting nuances of FAISS, including the need to manage large indexes and separate metadata storage.
- Implementation of FAISS integration in a Google Colab notebook, demonstrating vector generation, saving, and similarity searches.
28:23 💲 *Evaluation of FAISS*
- Recognition of FAISS's efficiency in similarity search with proper usage and fine-tuning.
- Intermediate effort level due to additional considerations for metadata storage and potential index management.
- Acknowledgment of potentially lower costs as an open-source package but with considerations for high RAM requirements.
29:18 🐘 *pgvector Integration*
- Introduction to pgvector as an extension for PostgreSQL, allowing searching by vectors.
- Setting environment variables and connecting to a PostgreSQL database using Superbase.
- Creation of a table with columns for vector ID, text, embeddings, and metadata, including dimensionality.
34:00 💲 *Evaluation of pgvector*
- Recognition of pgvector's efficiency with potential for fine-tuning in usage.
- Intermediate effort level, considering the need for an additional database for metadata storage.
- Acknowledgment of potential costs, including high RAM requirements and potential additional expenses for hosting metadata.
34:15 📊 *pgvector Database Operations*
- Introduction to functions for counting rows and deleting rows by ID in pgvector.
- Demonstration of adding test messages to the table, generating embeddings, and saving vectors with metadata.
- Usage of functions to search for top relevant messages based on cosine distance and manual filtering.
35:58 🧾 *Testing Similarity Search with pgvector*
- Implementation and explanation of the get top relevant messages function in pgvector.
- Testing similarity search on three test messages with varying relevance scores.
- Display of results, showcasing the closest match with a similarity score of 0.106.
36:41 🔄 *Searching and Storing Similar Messages in pgvector*
- Introduction to the gets similar and store function, aiming to search for similar messages and store new messages.
- Execution of the function on a list of 91 messages from a Telegram chat.
- Confirmation of matches found and the addition of new messages to the pgvector database.
38:13 💼 *Evaluation of pgvector*
- Efficiency rating of pgvector with three stars, indicating its capability for the intended purpose.
- Effort assessment, noting a moderate complexity in integration, especially for users familiar with SQL queries.
- Consideration of potential costs, suggesting minimal impact if an existing PostgreSQL database is already part of the architecture.

Made with HARPA AI

[ashishbhutada]

Thanks for a very concise and informative video!

[PizzaLord]

Yes I was specifically thinking about chatbot response speed

[Anton_Bond]

Добрий день! Дякую за дуже корисне та своєчасне відео. Маю питання, чи доцільно зберігати ембедінги в pgvector, а для similarity пошуку використовувати faiss?

[christofferbjorkwall]

Quite useful and good comparison. It would have been easier to see using English sentences though. Well done!

[ZERATELZP]

🎯 Key Takeaways for quick navigation:

00:00 📋 *Introduction to Semantic Search and Comparison of Vector Stores*
- Introduction to the video's topic: Semantic search and comparison of vector stores.
- Explanation of the project "dry" aimed at creating a chatbot for repetitive questions.
- Overview of how semantic search can solve the problem of repetitive queries.
02:00 🤖 *Understanding Semantic Search and Embeddings Models*
- Explanation of semantic search and embeddings models.
- Description of the OpenAI embeddings model (text embeddings ada002).
- How embeddings models generate dense vectors for text strings.
05:24 🌲 *Pinecone: Fully Managed Vector Database*
- Introduction to Pinecone as a fully managed vector database.
- Setting up a Pinecone project and environment variables.
- Demonstrating the integration with Pinecone for semantic search.
09:18 🌟 *Pinecone Efficiency, Effort, and Cost Evaluation*
- Evaluation of Pinecone's efficiency for semantic search.
- Discussion of the low effort required for integration with Pinecone.
- Consideration of Pinecone's cost, especially for commercial projects.
20:06 🧊 *Facebook Faiss: Efficient Similarity Search*
- Introduction to Facebook Faiss as an efficient similarity search library.
- Setting up environment variables for Faiss and pidgey vector.
- Integration of Faiss with the ability to store metadata.
26:30 ⚙️ *Faiss Efficiency, Effort, and Cost Evaluation*
- Evaluation of Faiss's efficiency for semantic search.
- Discussion of the initial integration effort and the need for metadata storage.
- Consideration of costs associated with Faiss, including RAM requirements.
29:32 🐘 *PGVector: PostgreSQL Extension for Vector Search*
- Introduction to PGVector as a PostgreSQL extension for vector search.
- Overview of using PGVector to search by vectors in PostgreSQL.
- Setting up PostgreSQL for PGVector integration.
29:44 🧩 *Setting Up Environment Variables and Database Connection*
- Setting environment variables for API keys, connection string, similarity limit, and top K records.
- Explanation of options to obtain a PostgreSQL database.
- Introduction to using Superbase as an online service for PostgreSQL.
31:16 🧬 *Integration with OpenAI API and PostgreSQL Database*
- Installation of the OpenAI package and confirmation of interaction with the OpenAI API.
- Establishing a connection to the PostgreSQL database and creating a test table.
- Installing the PGVector extension for PostgreSQL.
34:28 🛠️ *Generating Embeddings, Storing Data, and Testing Similarity Search*
- Explanation of functions for generating embeddings, creating vector IDs, and storing data in the table.
- Testing similarity search using cosine distance and filtering results.
- Applying the "get similar and store" function to a list of test messages.
38:13 💼 *PGVector Efficiency, Effort, and Cost Evaluation*
- Evaluation of PGVector's efficiency in the context of the project.
- Discussion of the complexity of integration compared to Pinecone.
- Consideration of cost implications, including using existing PostgreSQL databases and managed services.

Made with HARPA AI

[migunovich]

5:14 Pinecone
18:24 FIASS
29:17 pgvector

[rahulguptargrg]

Thank you for the great explanation. How does MongoDB fare in this regard?

[enricrypto]

Can you do a video with FAISS and Any DB (Postgres)?

[jma]

When using a hosted pgvector postgre service, would the cost be more like 2.5 stars?

[Chuukwudi]

Nice one mate. Just wish you did not use the russian text. As I do not understand russian, the results do not make any sense to me. But I am just believing... rather than knowing and seeing with my own eyes.