Vector Similarity Search at Scale // Dave Bergstein // MLOps Coffee Sessions #52

Показать описание

//Coffee Sessions #52 with Dave Bergstein, Vector Similarity Search at Scale.

//Abstract
Ever wonder how Facebook and Spotify now seem to know you better than your friends? Or why the search feature in some products really “gets” you while in other products it feels stuck in the '90s? The difference is vector search— a method of indexing and searching through large volumes of vector embeddings to find more relevant search results and recommendations.

Dave Bergstein, the Director of Product at Pinecone, joins us to describe how vector search is used by companies today, what are the challenges of deploying vector search to production applications, and how teams can overcome those challenges even without the engineering resources of Facebook or Spotify.

// Bio
Dave Bergstein is Director of Product at Pinecone. Dave previously held senior product roles at Tesseract Health and MathWorks where he was deeply involved with productionalizing AI. Dave holds a Ph.D. in Electrical Engineering from Boston University studying photonics. When not helping customers solve their AI challenges, Dave enjoys walking his dog Zeus and CrossFit.

// Relevant links

--------------- ✌️Connect With Us ✌️ -------------
Follow us on Twitter: @mlopscommunity

Timestamps:
[00:00] Introduction to Dave Bergstein
[00:55] Dave's background in tech
[04:33] Approach to building software products
[06:05] "There are parallels and analogies to what it means to build something that's robust, reliable, and at scale."
[07:58] "In some ways, when you think about the systems, it's not so similar. We have to consider that it becomes incredibly complex and all the different ways that a system fails become incredibly numerous and complex and then you have to think about the ways that you test."
[08:30] Pinecone introduction
[10:47] Vector Search breakdown
[11:38] Example - Zeus
[14:14] Examples of Vector Search use
[16:55] Help with translation
[17:52] Notion about Vector Search
[19:13] Characterizing a common scenario
[20:38] Engineering challenges
[25:05] "Updates is a big one especially when you have a system that's running live and you wanted to be able to update it live as it's running."
[26:03] "When you talk about billions of vectors, cost of the compute in the clod gets to be a significant factor and often cases, it's one of the biggest factors."
[26:35] Challenge comprehension
[28:00] Security as a big challenge
[30:47] "Security is paramount. We absolutely spend a lot of time on security."
[31:40] Switching from building an affordable portable imaging device to building/enabling the use of ML
[33:08] "I think I saw how painful it can be to try and build some things on your own and sometimes the opportunity cost that gets missed."
[33:38] Building modern ML tooling
[37:12] Not catching up to the greater MLOps audience
[39:10] Lessons to support diverse professionals
[41:44] "Openness is a great way to achieve building platforms."
[41:51] "Bringing a lot in-house can help you improve performance and can help you do things even better."
[42:19] "There are advantages to enabling a whole ecosystem around you and ways to operate with other platforms and there is something to be said for that interoperability."
[43:04] Interoperability
[45:10] "We didn't put all our eggs in the basket of our own proprietary and engine, we want to leverage the things that are going on in the ecosystem."
[45:40] Evolution of Vector ecosystem
[47:40] "I would be surprised to see that there are more companies like pinecone that kind of rise to fit this need."