Generative UI Computer Use Agent with LangGraph.js

Walkthrough of a computer use agent (CUA) powering a generative UI chat application.

[kurosawa]

Keep it up with the front end tutorials!!! They are so good and thank you!

[petarvukovic]

Great tutorial! Thank you for publishing daily 🎉

[animelover]

This tutorial came into a perfect time as I looking through the manula of openAi wondering how it works

[michaelou]

I believe you should have included a disclaimer stating that this use case requires access to the model, which is only available to OpenAI Tier 3 and above users.

[CRedfearn]

Sadly to use the open ai computer use model you have to be tier 3 or above and at the time of writing, it’s only a subset of tier 3. To be tier 3 you have to have spent $100 or more on api credits.

[amzraptor]

Is there any hope for running this for local browser use? Tons of internal websites that cannot be exposed to public internet just for computer use features.

[WeltenbuergerOrg]

These videos are fantastic—perfect timing! I'm following your approach instead of using the AI SDK. However are you planning to create a Next.js version with the generative UI examples included? I'd love to hear your thoughts on how such a project might look, or how we could extend your posted project (npx create-agent-chat-app) to add generative UI components. Specifically, I'm curious about how to leverage the strenghts of your approach to generative ui, and next.js strengths while maintaining a scalable separation of concerns. Newbie here ;)

[leodang]

thanks, it would be awesome if we can do next step after took screenshot such as tell the AI to analyze the screenshot and giving report with knowledge base and guideline from data we trained it before. Hope you guys create tutorial teach us implement these features, thank you in advanced!

[darkbelg]

Why is the repo on a personal GitHub and not the Lang chain one?

[aehtajazahmed]

Can we use the agent that is designed in Python?

[abdulrehmanbaber]

So in a way you guys just provided support for the open ai computer use. It would have been more awesome if you have given the support or an orchestration that user can use computeruse on thier pc's. Or servers, (like create computer use for themselves, all it will require vision agent with tools to scrol, type, click etc)

Why would i use langgraph when openai agent sdk can do the same.

[AdandKidda]

hi,
so is this can be used for Record and Playback?
i mean perform some actions (ie. Record where clicked), then playback it back

[SecurityFirm]

why use this-cua when so many better and free tools exist there? this is wrong idea for a teach/learn video dude.

[mikestaub]

the experimental_loadshare method is clunky. Look at how webpack does module federation