API For Open-Source Models 🔥 Easily Build With ANY Open-Source LLM

In this video, we review OpenLLM, and I show you how to install and use it. OpenLLM makes building on top of open-source models (llama, vicuna, falcon, opt, etc) as easy as it is to build on top of ChatGPT's API. This allows developers to create incredible apps on top of open-source LLMs while having first-class support for tools (langchain, hugging face agents bentoML), and one-click deployment. Also, fine-tuning is coming soon!

Enjoy :)

Join My Newsletter for Regular AI Updates 👇🏼

Need AI Consulting? ✅

Rent a GPU (MassedCompute) 🚀
USE CODE "MatthewBerman" for 50% discount

My Links 🔗

Media/Sponsorship Inquiries 📈

Links:

[shotelco]

Yet another piece to the democratization of AI! Very valuable.

[daithi]

Incredible content, and he doesn't waffle either!!! just to the point, good pace, great voice, great cadence, and perfect audio levels. This channel is gonna be big.

[VastIllumination]

You are becoming my favorite AI channel! This is literally exactly what I've needed. I've been looking for an open llm alternative to openai API for querying PDFs with Langchain. I haven't been able to test the largest LLMs using Langflow because it always times out from Huggingface.

[wendellvonemet]

When you cut out all the dead space, your sentences run together without the natural pause that would allow beginners to digest each new concept before being bombarded with the next five new concepts that are rattled off at the speed of light. Tutorials work best when the newbies have time to let new concepts sink in. I'll be stuck trying to wrap my head around what you just said, and I continually have to pause and rewind to catch what you said while I was still chewing on the first bite. You also run your words together, within the sentences, so I have to continually rewind to make sure that I heard you correctly. Many of us are complete newbs to all of this. The info you provide is great. I watch a ton of your videos. I just wish you'd go a hair slower and dumb it down for those of us who are brand new and have to look up the definition of each piece of new tech jargon used (had to ask AI what the hell a bento was, and it thought I was interested in Japanese cuisine).

[Tenly]

It would be a lot easier for us to follow along and be successful if you did these demos starting with a brand new machine with just python and conda pre-installed. That way our experience would be more likely to match the one in your video *exactly* and we wouldn’t struggle at the points where you say “the first time I tried this, I got an error” or “I already have this installed”. Just a suggestion.

[paulbishop]

stop it I cant keep up anymore :) everyday I am pivoting around your content, gimme a break already! What an exciting time to be alive!

[pancakeflux]

This is exactly what I’ve recently been looking for! Thanks for showing it off :)

[maxamad]

First time man. To the point and straight forward, Thank

[ajith_e]

Had'nt heard of OpenLLM before but now I can't hold my excitement to test it out. Well paced, Well executed tutorial that touches on the important aspects of deployment. Please follow this space closely because we'll be following you !!
Thank you for this great tutorial

[tiredlocke]

This is awesome. I've played with some different open-source models in Runpod(which is great, btw). And I looked into installing the Text Generator WebUI locally... but I don't have a suitable GPU yet. Ultimately, I want a self-hosted (preferably in a container) API that can run various models and hit from a web browser, or from a console app, or from a game. This looks like exactly what I want.

Now I just need to find a GPU to toss into my server...

[nikdog]

I'm gonna need a cardboard box server again. Time to start a 24/7 AI stream. 😂

[antonioveloy]

I prefer the Oobabooba Web UI, which basically runs an API locally and has a nice button to "import" any hugging face model.. But this is interesting too

[eck]

Will be waiting for their fine-tuning feature. Should be interesting.

[MeinDeutschkurs]

I‘m excited! Yeah! I‘m interested in custom/not listed models, also NLLB-200… And what about Mac? There is no xformers available.

[williammixson]

My last computer was a gaming rig. My newest build this week will be specifically for ML and I cannot wait!!! Easy sub.

[brianv]

Thanks for the video. This is getting close to something i'm looking for, but this still requires a permanent system set up with some decent hardware. Would be interesting to see this combined into a single google colab that could be run as needed, for those of us looking to utilize this on an occasional basis.

[vinylrebellion]

Looking forward to the mosaic 33b. Loving the videos

[jmanhype]

Please explain if this is hosted locally as a server or if we need runpod or chainlit

[khandakerrahin]

Are these models running locally? If yes, what are the hardware requirements?

[eck]

It is like personal computers in era of Steve Jobs, when they were still not so available to anyone. I guess soon this will become even more open with projects like that.