Poor Man's BingChat - Building an Internet-connected Search Assistant from scratch - Global AI Notes

Have you seen BingChat's/ChatGPT's ability to reason about current events, even though the underlying models have been trained with data only up to September 2021? Have you wondered how it works behind the scenes, what did they do to make it work?

During this talk Vlad will be talking about the ways to achieve this, including possible options such as fine-tuning and retrieval augmented generation, but also limitations such as models' slow inference times and limited context window sizes. He will demo a an app that can talk about current events, and you will learn how to build your own, from scratch*.

*from scratch = with nothing but Python, pandas, and the Azure OpenAI APIs

By Vlad Iliescu - Microsoft MVP on AI
Software architect, manager, and Microsoft Most Valuable Professional on AI, with more years of professional experience than I’d like to remember. Apart from developing software and playing the ukulele, I enjoy exploring the capabilities of AI, its strengths and weaknesses, and sharing my knowledge and experience with others.

[prepthenoodles]

🎯 Key Takeaways for quick navigation:

[00:17] 🤖 *Introduction to building a web-connected search assistant*
- Overview of creating a "poor man's Bing chat" with GPT
- Walk through reasoning process and considerations
- Goal is retrieving recent real-world information
[08:50] 🧠 *How LLMs work - predicting next words from context*
- Models trained on huge datasets to predict next tokens
- Fine-tuning and reinforcement learning improves quality
- But doesn't provide real-time up-to-date information
[16:13] 📚 *Large context window sizes enable more understanding *
- GPT-3.5 Turbo increased from 4K tokens to 128K in one year
- Allows summarizing longer texts like books and code
- But still not enough to fit all of internet's data
[18:58] 🤝 *Retrieve-Augment-Generate (RAG) architecture*
- Retrieval finds relevant info, augmentation prepares it
- Clean, relevant docs passed to LLM for generation
- Performance depends heavily on retrieval quality
[25:41] 🚚 *Build a chatbot with GPT + search engine results*
- Optimize user query for search engine input
- Get top search result, process to extract key content
- Dialogue by appending messages to shared context
[29:59] ☑️ *Capabilities demonstrated:*
- Answering questions with latest info
- Rendering results in custom formats (tables)
- Basic manipulation of time and dates

Made with HARPA AI