AutoGen Studio with 100% Local LLMs (LM Studio)

Learn how to run AutoGen Studio UI with local LLMs as agents.

LINKS:

TIMESTAMPS:
[00:00] Introduction
[00:26] Setting Up LM Studio for Local LLMs
[00:54] Downloading and Running an Instance of LM Studio
[01:50] Serving the Model through an API Endpoint
[02:48] Starting an Instance of AutoGen Studio
[03:26] Creating Agents and Workflows in AutoGen Studio
[06:30] Testing the Workflow in the Playground
[07:35] Analyzing the Results and Issues

[haroldasraz]

This is so amazing how much the tech progressed in just a couple of months. Thank you for making this video.

[luigitech]

Finally, thanks. I see a bright future for local LLM and agents

[USBEN.]

Great first step, now i would like to see practical workflows that can be used for actual tasks. Hope to see videos on that.

[engineerprompt]

If you run into issues with API key, just add: "not-needed" as the API key.

[CelesteOnYoutube]

OMG lm studio is just great. This makes thing so simple. thanks for the great video.

[prestonmccauley]

For those having issues, make sure you create each item from scratch again and add not-needed to the API key on all local calls; this should fix you up. I tried to modify it after and it failed,

[Dr_Tripper]

I was missing one point in getting this working right yesterday, thanks.

[deciopenna]

Can't get past the The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable" error. Tried NULL, EMPTY, "", not-needed... nothing works.

[alx]

So after playing for several days with it I can firmly say - this whole shit (both autogen and it's UI) is long miles away from being useful for anything rather than very small things. Don't fool yourself guys, try it out with your own hands. Probably with gpt-4 it can do bit more, but with Mixtral-8x7B it's very limited in what it can do and it's very buggy.

[danielshurman]

Very useful video on an important improvement in how we can use Open Source LLMs. More good progress your moves to document how to create a winning Local GPT

[Mystery-nn]

I am getting this error: Error occurred while processing message: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

[dipteshbosedb]

Awesome video, exciting indeed. Looking forward to many more use cases.

[remco]

Always the function calling.. but this is such an important fundamental building block.

[rgm]

I get this error "title":"'messages' array must only contain objects with a 'content' field that is not empty". I have tried several different models.

[thegooddoctor]

Brilliant as usual. One major thing about using LM studio - Its Painfully slow using it as an agent (Verbose Logging seems one of the bottle necks), even on my i9 13900K/64gb/4090 system. Even though AutoGen Studio seems more polished, I'm thinking that Crew with Olama is more performant when using local

[henrychien]

[2024-01-17 13:55:47.798] [ERROR] [Server Error] {"title":"'messages' array must only contain objects with a 'content' field that is not empty"}

how to solve this?
is it because the system message of user is empty?

[thecooleststufficouldfind]

When I run local llm i get 2 word responses from both the user and the other agent.

Like

here is

the script

for the

here is

with each one talking back and forth like this? ANY IDEA? LMAO

[cashmoneybanks]

I tried mistral and noticed some strange behavior. By default, the system message tells it to type TERMINATE if the task is completed. This caused it to terminate early before it successfully saved the image. If I removed this line from the system prompt, it worked, but it then continued to just print("Task completed successfully") over and over until it hit the message cap.

[AmanBansil]

I tried Phi3 and Llama3...both of them generate no output. Agents keep saying gibberish.

[techblock]

Nice video.
Be aware of the LM Studio licence.