LLMs as Operating System (Memory) CRAZY START?

From Abstract:

MemGPT (Memory-GPT), a system that intelligently manages different memory tiers in order to effectively provide extended context within the LLM’s limited context window, and utilizes
interrupts to manage control flow between itself and the user. We evaluate our OS-inspired design in two domains where the limited context windows of modern LLMs severely handicaps their performance: document analysis, where MemGPT is able to analyze large documents that far exceed the underlying LLM’s context window, and multi-session chat, where MemGPT can create conversational agents that remember, reflect, and evolve dynamically through long-term interactions with their users.

🔗 Links 🔗

❤️ If you want to support the channel ❤️
Support here:

🧭 Follow me on 🧭

[olafge]

This is the future. Combine this with dynamic agents and you have agents with an individual memory that remember their activities and the manger agent remembers the dialog with the user. Just brilliant.

[acdstore]

Your are most underrated tech guy in India. You are the one of the best among few indian who study new tech and bring to techies.

[ameygujre]

The hell… I can’t believe, this is the exact thing which I was thinking and imagining about just few days back!!! This new GenAI wave it’s pretty paced

[farrael]

This is not a new idea, but I guess they are good at marketing it. Their approach to creating a memory system for an LLM chatbot can work to a certain extent, but since they are not reprocessing the memories to summarize events, cluster topics, and reflect on these to create new memories, I'd say you will see it falling short at various moments where it can't really find the information it needs to.

I won't be very excited until someone implements those things I mentioned.

[Anonymous-lwzy]

Great job - very clear summary of paper! Thanks!

[ChaoticNeutralMatt]

I haven't seen anyone else cover this yet, but this is pretty amazing, ngl

[anatolydyatlov]

Holy hell, I've just tested it and I have to admit, it's beyond impressive! I can't wait for tomorrow to start modifying it for the purpose of my project.

[lehattori]

Keep going with this amazing job! :) In fact, this idea is excellent, especially when it comes to accessibility. Elderly people, for example, often need assistance when using technological devices, and having an interface to interact more naturally with the computer can be a great facilitator. Who has never heard something like "Boy, help me move this thing please". hehe

[electric]

🎯 Key Takeaways for quick navigation:

00:00 🚀 Introduction to MemGPT
- The video introduces MemGPT, a system that treats large language models (LLMs) as operating systems to manage memory effectively.
00:14 💡 Understanding Computer Operating Systems
- Explanation of computer operating systems, the concept of primary and secondary memory, and how data is swapped between them to ensure proper functioning.
01:22 🔍 MemGPT: Treating LLMs as Operating Systems
- Introducing the MemGPT paper and its authors, highlighting the connection with the Gorilla paper. Discussing the concept of LLMs managing their memory with different tiers.
02:03 💬 Context Window and Memory Management
- Exploring the concept of the context window in LLMs and how it affects their knowledge and response capabilities. Mentioning the use of retrieval augmented generation and vector databases.
03:12 🗂 The Virtual Context System
- Explaining the virtual context management system, including main context (analogous to RAM) and external context (analogous to external storage). Main context has a limited context window, while external context offers a potentially unlimited context.
05:33 🧩 MemGPT's Core Architecture
- Describing the core architecture of MemGPT, including the parser, virtual context, LLM processor, and parser outputs. Discussing how MemGPT manages memory and user interactions.
07:54 🔄 Working and External Contexts
- Detailed explanation of working context, system context, conversational context, and recall storage in the main context. How MemGPT updates and retrieves information for user interactions.
10:12 📚 External Memory and Deep Memory Retrieval
- Discussing the recall storage and archival storage in the external context, including their roles in storing and retrieving information. Introduction to the deep memory retrieval benchmark.
12:58 ⚠️ Limitation: Dependency on GPT-4
- Highlighting the limitation of MemGPT, which is its dependence on GPT-4 for fine-tuning and function call recognition, and issues with other LLMs like GPT-3.5.

Made with HARPA AI

[anonanon]

guy blew my mind with the title alone

wake up, build OS for purpose, work, finish repeat would be a crazy work cycle

[tanikam]

This is wiiildd. Thank you for putting this on my radar & for the resources to dig deeper. Critical piece of architecture for building intelligent assistants. Watching all of these pieces develop is really exciting. Great job covering this.

[fontenbleau]

I'm ready for this. My motherboard Asrock X99 Extreme which can support 256Gb Ram is become a standard on Ali for 2011 socket $150 and with latest never released outside enterprise Xeon 22 cores also in China costs 150. 😅 need to raise from 128 to 256Gb now, although they already reported of price hikes on memory modules worldwide

[BrianMosleyUK]

Nice paper, lots of potential here! Thanks for the breakdown. 🙏👍

[project-asgard]

Thanks for the video, great and exciting find!

[thexnr]

with the paper of the max tokens limit of 1mil (for short term memory)& this for long term memory this will make great AGIs

[Entropy]

Maybe a more link based memory storage approach might make sense, when it stores stuff it might be better to seperate the content into different actors vs the environmemt and create links between them? Then memgpt can follow the links to gain relavent context 🤔

[Jim]

I do feel everything is moving really fast these days. It will be interesting to see tech /AI in the last 2030s

[StefanReich]

Will we ever get LLMs to be actually reliable though? They are basically still a crazy kludge, just super high powered

[KevinKreger]

OS? It got my attention but it didn't connect well with existing new AI terminology/technology like RAG, which this definitely is. It created quite the lively discussion, so it's really a timely topic!

[Plystire]

I can easily see OS's adopting this idea. Cortana on Windows is lame and doesn't feel like a real AI, it's just a search bot. What worries me though, is that the AI will still be remotely called, meaning it will need to send your personal information over the Internet if it's necessary for a response (highly likely if the bot is designed to be attuned to you).
I hope AI tech moves to local execution before something that relies this heavily on personal info becomes big.