Minecraft AI - NVIDIA uses GPT-4 to create a SELF-IMPROVING 🤯 autonomous agent.

🔥 Get my A.I. + Business Newsletter (free):

#minecraft #chatgpt #nvidia

TIMELINE:
[00:00] - Intro
[00:48] - Voyager AI
[01:58] - Minecraft
[02:53] - Mineflayer API
[04:11] - GPT-4
[05:36] - Self-Improving
[07:51] - The Full Prompt Used
[11:18] - The Prompt for Creating Code
[17:58] - GPT-4 Decision Making
[19:00] - The Results
[22:16] - MineDojo (open source)
[23:35] - Implications for coding

The Paper:

Other Links:

[WesRoth]

I just started a gaming channel:

If you enjoy gaming, I hope you check it out. I'm VERY early in that progress, so don't expect pro level gaming :)

But, the goal is to eventually add various AI concepts and showcase AI games and progress. It's a long road ahead, but check it out if it sounds interesting.

[meloz]

I can already feel the effects of the 20+ Intelligence Buff

[apache]

I want like a hour video of the bot actually trying to win the game

[metacob]

If they stopped working on AI entirely (no GPT-5, no improvements to the context window etc) right now, and all we had was GPT-4 in its current state, just coming up with systems like this would be enough to completely revolutionize the world.

It feels a bit like being in the 1950s and being handed a modern desktop PC. First, people would be in awe how mind-bendingly fast it can compute numbers. But it would take a long time to figure out the full spectrum of things you can do with computers like that. That's what these LLMs feel like to me - incredibly advanced tools that we are just beginning to understand how to use.

[williamdewitt]

Imagine giving an AI control of the minecraft world, making every creature act like its real world counterpart. The Villagers and Illagers would actively try to accomplish their goals and will seek to improve their civilisations. The Illagers would become an actual threat that is attempting to become stronger...

[IceMetalPunk]

It's interesting to note that, at a high level, this is quite similar in architecture to the original Generative Agents paper. They both use GPT-based LLMs for their core processing; provide "sensory" information in the form of localized world descriptions to the LLM prompt; use the LLM's text output to trigger embodied actions in the simulated world; store memories in a semantic (vector) database for later retrieval; and use self-reflection to build upon memories and improve future decisions.

The main differences are simply what state info is given to the LLM, what actions are available to the LLM, and what kinds of memories are stored. But overall, this kind of "semantic memory stream with self-reflection" approach seems to be quite powerful in many different situations.

[Squiffel]

Would be interesting to see the Starcraft AI deepmind created combined with a language model so it can reason higher level thinking about the current game state.

[ourtwenties]

This is a *very* interesting experiment and result. hard to know where we are at in the S curve but this paper definitely seems like an important milestone.

[garretthiggins]

I would also like to see a person "play" minecraft in the same way. Would be interesting to watch how this looks from GPT4's perspective

[TrekkingThePlanetNet]

Great video. You didn't skip any steps and I appreciate that. You even review the Twitter post that you originally found. That was helpful to see the context.

[pbl]

6:44 Just noticed that Wes Roth accidentally did what ChatGpt did to correct it self, he pronounced Acacia wrong then corrected himself. Quite Impressive for an Ai to similarly correct it self in that example.

[mbrochh]

Here's a ChatGPT summary:

- Nvidia has created an AI to play Minecraft called Nito
- Implications of this paper are bigger than initially understood
- AI is writing code that improves itself
- GPT-4 is used as a reasoning engine, but it is blind and cannot see the screen
- Paper is called Voyager, an open-ended embodied agent with large language models
- MindFlare API is used to interact with Minecraft
- GPT-4 perceives the world of Minecraft through prompts
- GPT-4 creates tasks for itself and codes skills to complete those tasks
- Skills are saved in a skill library
- Iterative prompting mechanism is used to create code
- GPT-4 is given a goal to discover and unlock as many things as possible

- A study was conducted to test the effectiveness of using GPT-3.5 to create code for an AI agent to interact with the game Minecraft.
- The AI agent was given a skill library of functions created with the Mind Flayer API.
- The AI was given instructions on how to complete tasks such as collecting blocks, crafting items, killing mobs, and fishing.
- The AI was asked to reason why code from the last round failed and generate code to complete the task.
- The AI was also asked to plan step by step plans to finish the task and generate the code.
- The AI was asked to act as a mentor and guide the agent to the next task based on its current learning progress.
- The AI was asked to look for rare resources, upgrade equipment, and discover new things.
- The AI was tested against other AI agents and was found to be the best, fastest, and most advanced.
- The AI was also able to efficiently generalize unseen tasks.
- The AI was able to use the skill library as a versatile tool to enhance its performance.

- GPT-3 is capable of creating its own code and improving its own ability in a Minecraft world.
- Code may no longer be required in the future, as GPT-3 can directly communicate with the computer using natural language.
- Matrix is a good example of this, as Neo was able to see the code of the matrix and do stuff without actually seeing it.
- GPT-3 is able to pick up on text and manipulate it in the environment.
- It is unclear how far away we are from being able to directly communicate with the computer using natural language and having GPT-3 write the code without us seeing it.
- Code comments may become archaic in the next 5-10 years.

[jmattoxriskpro]

This is amazing and a little scary. I’d love to see them run something similar with no pre training but rather just the goal to survive. Then let it struggle to grasp its world and succeed

[miracle_grrrl_mira]

outside of playing the game, just imagine the updates an AI could make to the Minecraft source code and assets, in terms of both content and performance, that would have otherwise taken much longer if unassisted, or may not have ever been done by Mojang/humans. and then scale that to pretty much all software.... lol

[alertbri]

Imagine using GPT-4 to be the thinking brain of a robot... We're much closer than we realise.

[alertbri]

9:48 bingo! I've felt this for some considerable time... GPT-4 is like Einstein before he could walk or talk, the potential is incredible. OpenAI has given the world fire for the first time... We don't need to develop plasma yet, just start cooking things!

[AaronMorrisTheSteamFox]

10:20 42 is *, also known as "wildcard." The answer was meant to be interpreted as "everything."

[almiraj-x]

15:05 - 15:55 I resonate with your tangent, good point.

I'm incredibly annoyed when the full scope of this great technological ingenuity is downplayed by blunt reductionism, nevermind that it totally dismisses Sutskever's thoughts on the matter - that at some point (eg with enough NVIDIA GPUs) the easiest way for a transformer to increasingly best predict the next token is to "model" (abstract away) the underlying processes that generated it. So, an "internal model of reality" based on statistical regularities in data, in this case text, arises.

Somewhere in the inscrutable matrices several concepts are encoded as mathematical abstractions, such as the ability (or abilities?) to reason about world states and create subgoals. The quality of the abstraction that is revealed to us also depends on how good humans are at activating these "regions" of the neural nets through prompting which I think is another aspect of alignment (eg, we want you to reason!). Many people believe what you see is what you get, but that's not true with GPTs. There are unexplored depths in these models, it's truly fascinating.

Thank you for covering this, Wes.

[GabrielLima-ghwe]

I imagine now how amazing can Minecraft become if there was an update with many Agents NPC's, with this level of thinking from GPT-4 it would be a much richer experience to play the game, especially if the characters are allowed to talk.

I can already see this coming in the next months.

[neilangelomolleda]

I think an added parameter would be all of the "placed blocks" or "places of recognised structure" i think placed blocks would be too big. But recognising structures and remembering where they are are important. For example a base, or a storage space. The agent would have a skill to build it but also need a skill to remember where they are. Or what they look like. How to travel to it etc.