Memristors for Analog AI Chips

Links:

[dmtree]

Hey jon, me and my father (an electrical engineer and i am in IT) watch your videos all the time. He loves your content and reminisces on his long history in EE. If you're ever in Toronto let us know

[flygonbreloom]

I really appreciate this. I'd always hear about Memristors online, but not quite get what they actually were. This helps a tonne, and I am thankful for it.

The concept of analog technology in a surprising field does remind me of something - have you ever been interested in what the life and death of the Scanimate machines were? Those analog CGI machines that produced a lot of CGI graphics for TV and film from the late 60s to the mid-80s.
People seem to completely forget they existed when it comes to discussing CGI, and I'd argue they ended up being an essential and important stepping stone for the eventual incredible success of digital CGI.
The way the machines work is both incredibly familiar to how digital CGI worked, but also completely alien. But it's fine if this sort of thing doesn't interest you.

I'm really happy I got to watch this video, it taught me a lot in something I was curious of, but was too afraid to ask. Keep it up, these videos are a highlight for my day, and do genuinely cheer me up.

[rayoflight]

One memristor replace a digital macro component made of:
1) An A/D converter
2) An 8 or 12 bit Flash memory
3) A D/A converter.
The applications of memristor arrays as computing memory really depends on the voltage retention qualities of the device.
Thanks for the video,
Anthony

[YY-lvfg]

The synthesizers we make with these are gonna sound crazy.

[jayglookr]

You had me at " these 'bad-boys' ". My chem profs back at university pointed that out - everything was a 'bad-boy' to me. Glassware, reactants, terms in equations. Bad-boys all. It was used so frequently that a few of them started using it too even. Anyways, sick vid, sick channel. Liked, subd.

[leonardonakatanimoretti]

During my undergrad my research orientator said that this would be very useful in the future. That was 2018. Here we are five years later and slowly but surely progress is being made. I doubt it will ever reach peak commercial application, mainly due to manufactoring constraints, but sure is a cool niche piece of tech.

[Kasy]

A memristor array looks oddly similar to the Apollo 11 memory modules

[Hmework]

Your voice sounds much clearer than usual not sure why. Great content as always I have all your notifications on.

[yurcchello]

maybe memristors could be used in super speed camera sensors or fusion of neural network and sensor

[SimEon-jtsr]

This is cool for me because I used to be in a group BEAM memristor group where guys were working on making them at home!
Recently I've learned more about electronics and now there's all this legit research on the topic and it's great stuff.

[kuhascoat]

OMG I HAVE BEEN WAITING FOR A VIDEO ABOUT EXACTLY THIS!!!! Ever since Sixty Symbols made their video on memristors 😊

[dewashyadubey]

I wonder how you learned all that. You make it sound so simple. But if I go out searching for all this I might just end up scratching my head. Amazing videos and amazing work.❤

[MatthewSmith-uqmx]

Intel made an analog NN chip in 1990 based on flash memory cells (aka isolated gate FET). It wasn't a commercial success but I believe it's likely the best approach. Flash cells can be made in any normal CMOS process with very few extra steps. Analog NN really are many orders of magnitude more efficient because you're not shuffling data through your limited number of compute units. Digital NN will also benefit from having compute local to every memory row, but the chip real estate would get enormous.

[chengong]

Current computation based neural networks are just stupidly inefficient, to determine whether an artificial neuron should turn on you have to read a bunch of values from memory, which is not in the same chip so you gotta go through a bunch of memory management crap, wasting a lot of power and time, then you gotta do the math which is basically unnecessary because neurons don't need precise numbers anyway. Then you take your result, put it back out into memory, so it could then be used by another round of simulation of another neuron down the line.
Where as for a real neuron, this whole process is basically just baked into the wire, you send information into the wire, it automatically gets transformed and transferred over to the next neuron, there's no unnecessary nonsense like math or memory.
A big reason why current deep learning based AI is so stupidly inefficient, but if we could somehow have hardware that does the same thing as neurons, without using math or external memory, it would instantly be orders of magnitudes more efficient.

[ahchoooo]

Qinghua University in China built a memristor chip several months ago, and conducted test in AI, and found to be much better than the regular approach. I personally think that by building a Field Trainable Memristor Array (FTMA), similar to the Field Programmable Gate Array, one can achieve AI function very similar to our brain. There is no need to worry about the imprecise nature of the memristor as our brain is also imprecise in nature. Jeff Hawkins of Numemta explained it very well, and has come with highly redundant models similar to our brain that can overcome this shortcoming. Using this approach, AGI can be implemented very quickly, with efficiency and cost very similar to our biological brain. Yes, AGI is finally coming faster than we think possible.

[MikeGaruccio]

I wonder if the defectivity may be a smaller problem than it looks on the surface.

First, current research into quantized models has shown good-enough performance at inference time all the way down to 4 bits, which would at least seem to suggest a good bit of tolerance for poor precision as long as the overall design of the system takes it into account.

[EvDelen]

Got thinking about Memristors again after your SRAM video, was thinking about if they could be used in place of SRAM too.

[ronlipsius]

(1:40) Actually current (I) is not limited by a resistance in it's path.
That happens when the path or is divided or otherwise diverted.
Voltage (E) however, drops increasingly across an increasing resistance and the available power is reduced (because IE, or current times voltage = Watts).
So a series resistance regulates power or ability to do work.
If you make a ladder, as it were, of many parallel "rungs" of resistors, the voltage between the two rails will remain constant,
but the current will be divided between all of the resistors.
Greater power will be consumed in this, or in any circuit or part, by less resistance and maximally by zero resistance (e.g. short circuit).

[joseeduardobolisfortes]

Thank you for show the Doctor Chua's papers on video; I could find it by the name and now I will use it on my work!

[justingarretson]

I don't understand how you can create such amazing content week over week!