Ep 15. Should You Use Open Source LLMs or GPT-4?

-----------------------------------------------------------------------------------------------------
Welcome to Episode 15.

In the past year OpenAI released GPT-3.5 and GPT-4, two breakthrough, proprietary, large-language models.

Most people have used them with ChatGPT, but you can also use OpenAI’s API to build your own applications with these models. Google and Amazon are creating competing models, and there has been an explosion of free, open source models supported by companies like Meta.

With so many model choices, you’re wondering …

👉 Are open source LLMs as good as proprietary ones like GPT-4?

👉 What are the pros and cons of open source LLMs?

👉 How should I get started?

In this video I explore these topics by comparing an application running with both GPT-4 and an open source LLM.

Afterward I’m going to give you some specific advice for navigating these complex choices.

Enjoy!

-----------------------------------------------------------------------------------------------------
PROLEGO GITHUB REPO
-----------------------------------------------------------------------------------------------------
ABOUT PROLEGO

[HarpaAI]

🎯 Key Takeaways for quick navigation:

00:00 🎙️ *Introduction and Overview*
- Introduction to the topic of open-source large language models (LLMs) and their effectiveness.
- Mention of OpenAI's GPT 3.5 and GPT 4, as well as competing models from Google and Amazon.
01:10 💼 *Challenges with GPT-4 for Natural Language Query*
- Discussion of using GPT-4 for natural language queries in applications.
- Challenges related to legal and data security policies, cost, and speed.
02:18 🏦 *Introducing the Open-Source Find Model*
- Introduction to an alternative open-source model called "Find" for code generation.
- Explanation of an application scenario involving a bank's CRM data.
03:23 📊 *Evaluation Framework and Model Comparison*
- Creation of an evaluation framework to compare the performance of GPT-4 and Find.
- Comparison of API calls, completion time, and model accuracy.
05:00 📈 *Improving Find's Accuracy*
- Discussion of errors in both GPT-4 and Find responses to specific questions.
- Mention of quick fixes to improve Find's accuracy and speed.
06:26 🔄 *Pros and Cons of Open-Source LLMs*
- Explanation of the pros of open-source LLMs, including data control and flexibility.
- Mention of the cons, such as requiring more expertise and work.
06:52 🚀 *Getting Started with Open-Source LLMs*
- Advice on starting with a prototype using GPT-4 to evaluate feasibility.
- Suggestions for optimizing for speed, cost, competitive edge, or security with open-source alternatives.
07:46 🤫 *Focus on Meaningful Progress*
- Encouragement to focus on meaningful advancements rather than incremental improvements in AI models.
- Emphasis on activities like defining problems and creating evaluation frameworks.

Made with HARPA AI

[rasterize]

Great comparison! I'm honestly impressed that Phind does so well in a real world scenario.

[Martin-krnx]

Just came across your channel. Like your style and content. Have you considered doing a video of autogen in combination with proprietary and open source LLMs?

[SerhiiTolstoi]

Great video, thanks for sharing! I'm curious about how you track the response time or latency in these scenarios. Are you measuring from the moment you hit 'Submit' in the OpenAI playground until the final output is generated? Or is there a different method you're using to gauge the response times?