ChatGPT in Unity is INSANE!

This is a proof of concept from a Unity developer about integrating ChatGPT into the Unity game engine. Will simple text commands work in Unity, or will it fail hard?

The answer... yes... and yes. ;)


Honestly, I think this video is a glimpse into the very future of artificial intelligence powered/assisted game development tools.

Links
-----------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------

[Ares]

Most requested task: "Make multiplayer"

[bhBlacky]

games like sims could really benefit from chat-gpt.. random people with cool conversations and when they add ai voiceover that doenst sound robotic.. will really create a lifelike ambience.. great times comes up

[Cola-]

ChatGPT makes your life easier for now. Five papers down the line, ChatGPT doesn't need your input anymore.😛

[ymi_yugy]

getting this production ready, seems like a huge amount of work and I'm not sure if a company like Unity is able to commit the amount of resources that would take.
Here is a list of requirements I think it would need to be actually useful.
1. Work at interactive speeds.
2. Be very reliable.
3. Cover the vast amount of functionality.
4. Be able to understand abstract problem descriptions and come up with how to solve them using Unity.
I think 4. in particular is still way beyond the capabilities of LLMs of today.

[judgegroovyman]

That’s wonderful. There’s nothing there I couldn’t do before but it makes those things way way easier and I’m looking forward to abusing it to see what it can do. Thanks for showing us this one

[wendten]

the method it actually quite simple, you just ask the bot for code that would do what you want, and then you run the code that it provides. if it provides code that doesnt work, it will give an error and doesnt run. If you sign up for the gpt4 you will probably get access fast, for me it took 1½ days from signup to get access. gpt4 is far more consistent with more complex commands. the price is 15x that of chat gpt though, but at 0.03$/750 words you can still make it write an entire book for 1.5$

[WeenieWalkerGames]

It could be interesting. I know there are a lot of tools I've bought on the Asset Store that performs some of this (grab all objects with this name, mass renaming, etc.) but I'm not sure where this would necessarily help me right now. We'll see how much it takes off and honestly, it is the stuff I can't think of that are the most exciting part of this. What COULD it do that I just don't think is possible right now?

[VesuviasV]

so this was the prompt according to the github:"Write a Unity Editor script.\n" +
" - It provides its functionality as a menu item placed \"Edit\" > \"Do Task\".\n" +
" - It doesn’t provide any editor window. It immediately does the task when the menu item is invoked.\n" +
" - Don’t use +
" - There is no selected object. Find game objects manually.\n" +
" - I only need the script body. Don’t add any explanation.\n" +
"The task is described as follows:\n" + input; nothing particular ground breaking, you could probably play with this some more to make it signifigantly better and fail less often.

[aeriusblindsight]

Seriously love how this stuff is made, released, then, the community improves upon it, and it grows. Imagine using this with chatgpt memory plug-in, and telling it to analyze every failure then save it to memory.

[HE]

Chatgpt is a good tool and can be of great assistance and can be helpful with things; especially when we're trying to learn. But, I think things are going to be less fun and less inspiring when it starts replacing human creativity and effort. I'm not sure if I'd want to buy art, music and games made solely by A.I. Because then, I'm just paying a robot. This is just my opinion. It should help us, not altogether replace us.

[svedalawoodcrafts]

To be honest AI will be a complete game changer in game design, development, testing. It's going to be unbelievable. It already has done a lot of changes in the game industry.

[chillenld]

Using GPT-4, I managed to get ChatGPT to generate a full Godot game, including project config, to make Pong from almost no description. Just taking any errors that it produced and pasting them in to get new files. Really powerful! GPT-4 will eventually accept image inputs, so being able to show off a design document with map sketches, or showing it spritesheets to have it automatically set up animations, may all be ways to make use of it in small ways to reduce manual labour.

It's also quite good at language translation, so I imagine a first pass of translation spreadsheets would be fantastic at getting your game to a larger market as long as you can still validate its output.

Very cool stuff! Between this and 3D generative art, prototyping is likely going to be much higher fidelity and much faster. Game jams in 2025 are going to be exciting.

[deathscythe]

Ok 2 question.
What Version of unity is this?
What did you download besides unity and visual studio to get that ai settings?

[reeboothemad]

Aw, man... the improvement steps this tech has made in the last two years are frightingly big. I wouldn't be surprised if five more years down the road you could interact with any creation software more or less the same way people in ST:TNG interact with the holodeck: You just tell it what you want and it gives you a draft and options to refine things to your needs.

[aceace]

having written some chatgpt api code and looked at his code im surprised how well it works (not critisizing just observing). he is using a very basic single line prompt with no setup/example data for chatGPT to learn from. i would bet you could reduce the code errors significantly if he just added more data to the API request.

for example i did similar and asked it to create c# scripts for a AI charecter and response system as a POC on using it to create AI characters and i found it was unreliable at first similar to this, until i gave it a grounding and example.

so instead of just "basic prompt + {user input}" which is what he is doing you do

system "basic setup prompt similar to his"
user "create 10 random cubes"
assistant "{{input perfect answer code here}}"
user "{user input}"

he also could possibly tweak some of the api variables as chatgpt api lets you define how deterministic (reproducible) its output is so it is less random

[gokudomatic]

Impressive. I'm not a unity user and I'm more interested to see such plugin in Godot. But overall, yes, it's very impressive.

[hartvenus]

Love this! But to be fair it kinda looks like it’s easier to just make the script that generates all of these things using actual code instead of only natural language which is not as debuggable.

[crummymudd]

It's just like Star Trek, on the Holodeck: "Computer, make a forest." "Make the tree branches higher." "Add 3 paths through the forest." etc, etc. Computer could do it and it passed for real, but they still needed a human to have a vision to create a scene and interaction. What worries me the most about this AI stuff and how easy it is -- am I going to become a Barclay, seeking the Goddess of Empathy ???

[JC-jzrx]

Does anyone have any idea how this actually works tho? I looked through source and still can’t figure it out.

Like if chat gpt recieves a command. I get it can give a response and even understand it, but how do chat gpt responses get converted into unity actions

[Lout-edof]

I'm novice on Unreal Engine, where do i have to place folders "packages" "assets" and "projectsettings" ?