Google's NEW TITANS: Transformer w/ RNN Memory

Google's new transformer architecture, called TITANS, explained.

All rights with authors:
Titans: Learning to Memorize at Test Time
Ali Behrouz, Peilin Zhong and Vahab Mirrokni
by Google Research

#airesearch
#google
#science
#aiagents

[ptkjuwb]

Thanks for the video. Don’t listen to the guy asking for shorter videos

[irbsurfer]

It seems everyone is talking about Titans on YouTube right now but this video stands out as one of the few that not only thoroughly explains it's architecture but also provides a deep, insightful dive into its mechanisms and applications. It strikes a perfect balance between tecnhical depth and accessibility, making it an invaluable resource for both newcomers and those looking to deepen their understanding. Excellent work on breaking down such a complex topic into something comprehensible yet intellectually engaging!

[truliapro]

Sir you are doing great. And do not shorten length of videos. Your videos are outstanding. Love it and alot of appreciation.

[Wobbotherd]

I always knew RNNs would make a comeback. The human brain itself is a RNN, not a convolutional NN or a transformer.

[oursbrun]

12:10 Aahah; thank you ! I was about to ask that question 😂

[EnGmA]

The field seems fast if you hopped on at GPT3 release date

[HeyFaheem]

Cool, but many came and gone. (Mamba, Jamba, ...) Let's see if it sticks along. Even so, adaptability is a concern too. Coz you know, the OS is too much integrated with the transformer architecture. It has to do very very well to start getting adapted.

Btw, your videos are too good. Your presentation is amazing. Keep up the work. But one thing that is little down with your videos is they are little much longer. It would be great, if you could publish written notes like an article or a blog. That would be very helpful to read, refer and learn.

[maertscisum]

I think this is it. This will lead to the right direction.

[dudicrous]

Looking forward to the publication of the code! Meanwhile: awesome content, thanks!

[fdavis]

A step closer to self improvementing models and shortly after that, ASI!

[CharlotteLopez-ni]

Great insight into TITANS! Could this mean a shift in handling model evaluation? Any thoughts on the impact on existing AI systems?

[timothywcrane]

Every day some new way to train on the data, infer the data, reinterpret the data, compress the data, etc... are you working on the packaging and presentation, or if not in commerce, other use cases for your organic or synthetic data product yet?

[shaneoseasnain]

The Titan architecture’s modular approach to short- and long-term memory differs from Kahneman’s model, where immediate recall (System 1) is fast and intuitive, while higher-level reasoning (System 2) is slower and effortful. A hidden layer in both systems determines which mechanism to use, resembling Weick’s concept of sensemaking through storytelling and surprise. This suggests AI could benefit from similar dynamic prioritization. While Titan shows progress, we are still at the early stages of developing architectures that fully mimic human memory and reasoning.

[shaneoseasnain]

The Titan architecture’s modular approach to short- and long-term memory differs from Kahneman’s model, where immediate recall (System 1) is fast and intuitive, while higher-level reasoning (System 2) is slower and effortful. A hidden layer in both systems determines which mechanism to use, resembling Weick’s concept of sensemaking through storytelling and surprise. This suggests AI could benefit from similar dynamic prioritization. While Titan shows progress, we are still at the early stages of developing architectures that fully mimic human memory and reasoning.