Use AutoGen with ANY Open-Source Model! (RunPod + TextGen WebUI)

I might be obsessed with AutoGen...

In this video, I show you how to use AutoGen powered by TextGen WebUI and RunPod, which means you can use literally any open-source large language model with it, even Falcon 180b or Code LLaMA.

Enjoy :)

Join My Newsletter for Regular AI Updates 👇🏼

Need AI Consulting? ✅

Rent a GPU (MassedCompute) 🚀
USE CODE "MatthewBerman" for 50% discount

My Links 🔗

Media/Sponsorship Inquiries 📈

Links:

[matthew_berman]

Should I make a video testing different open-source models to see which one powers AutoGen best?

[simkjels]

I work at a law firm, and I have set up AutoGen group chat to simulate a legal team to solve tasks. The team gather legal information and argue legal matters between agents to come up with a multiple scenarios that a virtual judge finally rate each of the suggested solutions. I tried it on previous exams from law school and compared AutoGen's output to the exam evaluation, and it is staggering how well it performs.

[marcfruchtman]

Your ability to parse these install instructions and organize them into a video that we can actually follow is amazing.
Thank you for making these videos!

[IvanGabriele]

Thank you so much for the shout-out Matthew 😊! Amazing video and well-explained tutorial as usually! As I told you in private, even as a software engineer, you were the first one I watched and you helped me learn so much during my first steps into the AI & LLM world. Hopefully we'll have more amazing discoveries to share 😉.

[koliux]

things that I think are a must for Autogen to take off are :
1) how well if at all it can push to github
2) iterate on the github
3) Embeddings and Vector DB like Supabase
to store all prompts so it does not deviate too much from the development of the coding project :/ ( but maybe I missed that part )

[luisortega]

Is anyone else continuously getting 502 gateway errors when they finish configuring the pod in the web UI? I've tried it on two different machines while using both Mistral 7b and Dolphin Mistral 7b

[OriginalRaveParty]

I never quite figured out how to get multiple agents set up in VS Code, running mistral 7b locally with autogen. I configured assistant as name "Coder", and then assistant2 as name "Checker" and tried to get Coder to pass all his work to Checker to verify his work, but instead it all came back to me as User Proxy. Would be great to see a 5 agent example, like a little dev team with a CEO, concept designer, user interface guy, coder and code checker or something similar 👍

[Qi]

This will be INSANE! Can't wait to see what all people make from this.

[zyxwvutsrqponmlkh]

2:23 It is not completely uncensored, however an effort was made and with proper instructions you can mostly avoid censorship that it still tries to do to it's output. This was a cencored model which fine tuning efforts were made to reverse the censorship, it was not 100% successful but it was a good effort and it is substantially more useful.

[frankismartinez]

Worked sweet on my older Mac M1; was able to create a POC for a healthcare project… immediate industry value

[kigas]

I've discovered autogen + langchain can work with Excel sheets. Autogen can read the columns and calculate financial ratios (I use it for finance). Really looking forward to the advanced autogen video.

[fuba]

Runpod also offer like a "LLM as a service", where you pay as you go. You think you could cover that in a video sometime?

[xbsox_]

Thx Matthew for this incredible work, I tried it before many times with many models and Mistral was the best and light option, i faced one issue with context length limit and i hope they have a good technique to solve it

[GoldenDragonFromHills]

Waiting for the advanced code generation tutorial by autogen

[JonathanStory]

Really confused about actual pricing for running on RunPod. The posted prices ($/hr) don't mean anything to me because I'm clueless about how much cpu time would be used in the real world. Is it likely to be multiples of Chat-GPT4's $20/mo? If you spend a day coding with mistral, what does that set you back?

[VidarBrekke]

A few questions:
1 - My scripts always fail because they generate more than the 8K token limit. is there a way to avoid this from happening? Can c-tags or other method be implemented?
2 - Will Autogen work with existing (large) codebases (I have a Django project i'm working on), if so how?

[austinpatrick]

A helpful usecase I’ve found was with finding ongoing clinical trials that a particular patient could be a good candidate for.

This implementation was technically with autogpt (not done it with autogen yet)

[JonathanPohlner]

keep up the good work! loving the autogen series! a wizard an an assistant and a completer walk into a bar

[OpenAITutor]

Just got it to work locally on my Windows, box. Thank you for the video. Um, suggestion to folks make sure you tell the bot in your system message the OS you are using. It likes to default to Linux. :) TextGen WebUI is a beast. The lm studio is too new.

[mikewhite]

Can't wait for the Autogen advanced tutorial!