How are memories stored in neural networks? | The Hopfield Network #SoME2

Can we measure memories in networks of neurons in bytes? Or should we think of our memory differently?

Time stamps:
0:00 - Where is your memory?
1:41 - Computer memory in a nutshell
2:58 - Modeling neural networks
4:42 - Memories in dynamical systems
9:54 - Learning
13:36 - Memory capacity and conclusion

Animations largely made using the manim community edition:

Original Paper on Hopfield Networks:
Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences, 79(8), 2554-2558.

Neuron image by Santiago Ramón y Cajal, The pyramidal neuron of the cerebral cortex, 1904 Ink and pencil on paper, 8 5/8 x 6 7/8 in. Credit: Cajal Institute (CSIC), Madrid

Music: Aakash Gandhi - "Dreamland"

[ginqus]

bro made a channel dedicated to neural networks, posted one video, gained 24k subs, decided this was enough and left

[aegontargaryen]

As a neural network, I confirm that this is how we store memories

[murilospineli]

is this a first video? it's surprisingly good

[minh]

Guy, it's 1 year already. Cmon comeback bro

[piggy]

This guy explained such a complex topic so well, great job.

[luke.perkin.inventor]

You nailed the 3blue1brown feeling whilst being ligther on the maths. I wish you the best of luck with future videos! 🙂

[keithmiller]

This is remarkable. How the heck does this channel only have 1 video?!

[Sazazezer]

I've been idly musing about the brain's memory in terms of binary and bytes for years and always thought it was such a limited way of looking at the subject but didn't know any better. I asked around in the past about alternative ways of looking at the subject but never came away with a decent answer. This video has been such an eye opener and i think i'm going to have to take a deeper dive into the subject. Thanks!

[mmarrotte]

The amount of effort required to not only curate such complex information in an intuitive way, but also to animate the concepts in the way you've done, is clearly tremendous - thank you for taking the time to produce this! I'm inspired by your work, really looking forward to more!

[sciencelab]

This is one of the most important pieces of media I have yet consumed to better understand this world. It is incredible to realize how one's brain works in principle. Thank you so much!

[frenk]

I studied Hopfield networks last year in the framework of spin glasses and all assumptions were dropped seemingly out of nowhere in a big confusing mess (as usually happens in lectures). This video really tied up all together and now makes so much sense beyond memorising proofs. Excellent video and brilliant demonstration.

[gianlucazinarof]

You cannot fathom my disappointment when I looked at your channel and discovered that this is your only video. What do you mean there aren't any others? I need more! Seriously, interesting topic, crystal clear presentation, keep up the great work!

[NajibElMokhtari]

This is amazing! I finished the video and immediately went looking for the next one where you'd explain the more accurate models. Then realized this is the first and only video on this channel.
Can't wait for the upcoming stuff. Congrats for the launch!

[macchiato_]

It's really amazing how something as simple as a neuron has highly emergent properties that can do very complex tasks.

[LearnwithAnnaN]

Dude made one video and then stops uploading.

You have a talent.

[HansLemurson]

Different "seed" states will call up different memories. I wonder if this could be analogous to how certain ideas link together, and maybe some of the underlying information of related ideas is actually a correlated section of their memory patterns.

[martingisser]

Brilliant exposition for both mathematicians and nonmathematicians. It took a few decades to get me (mathematician) interested in neural networks. Now I'm hooked. Musing about connections to Markov processes and PDE...

[Wonders_of_Reality]

Looking forward to watching the second part! And many thanks for choosing dark text on a light background. It looks like a calm creamy ocean of fascination!

[sebastianmuller]

"Contrast this with what we believe about the brain..." I like how careful you are with how you make your statements.

[tfconfirmedbuthv]

Love it! Hopefully this video will get big, this question is wonderful and really makes you think outside the box, and I think you explained it perfeclty.