What Can We Learn From Deep Learning Programs? | Two Minute Papers #75

The paper "Model Compression" is available here:

There is also a talk on it here:

Discussions on this issue:

Recommended for you:

WE WOULD LIKE TO THANK OUR GENEROUS PATREON SUPPORTERS WHO MAKE TWO MINUTE PAPERS POSSIBLE:
David Jaenisch, Sunil Kim, Julian Josephs.

Károly Zsolnai-Fehér's links:

[RC-]

Hahaha, I like the idea of "we have no idea how this learned behavior works, let's make it learn how to explain itself".

[selfreference]

Honestly, you make one of the best shows on YouTube. Every time I watch your show, I learn about something amazing--or even sometimes many of these things. It's wonderful!

[DasButterBoat]

Love these shows. Keep up the great work!

[robosergTV]

Very interesting, thanks. I think 5 minutes videos are way better than 2 minutes ones. Of course if you have the time recording longer videos.

[complexobjects]

I just discovered and immediately subscribed to your channel. Such amazing content+descriptions! Im so glad this exists.

[estranhosidade]

It doesn't has much to do with the main subject of this video, but you talk briefly about languages and rules. And this made me think that, maybe, you could make a video talking about this new google research paper that has come out a few weeks ago, about how they have improved their machine translation, "Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation".

Also, thank you very much! I'm loving your videos. You explain things very well and in very didactic way :)

[TonyDiCroce]

we do understand these networks. they are essentially giant math problems. asking what one neuron is doing makes about as much sense as asking what one number in an equation is doing. the sum is greater than the parts.

[AlexanderBollbach]

what if you had one network that identifies discrete objects/structure in the world. the second network simply learns to optimize gameplay, say. Then, the first NN can segment the second NN's weights into subnetworks where each subnetwork corresponds to a discrete task. Finally, a meta-network could train using all of these subnetworks, but the abstraction level has been moved up because the pieces are all functional sub-units, and so it might be more understandable to us.

[attilakun]

I think part of the confusion comes from the fact that people have similar expectations toward deep learning as to more traditional AI algorithms. The key difference in my view is that training neural networks with gradient descent is not an explicit algorithm: it is a meta-algorithm whose job is to output a specific algorithm. The output algorithm's "code" is the concrete assignment of neural connection weights. This is why we do not understand how neural networks actually work: we only understand how to evolve them.

This brings deep learning into a similar ballpark as evolution. Evolutionary algorithms and deep learning appear to do similar things with the important distinction that evolutionary algorithms require explicit sampling during training, whereas deep learning has a more efficient, gradient-driven method of identifying the direction where the optimal algorithm might lie.

Considering this prompts me to assign a great scientific importance to the understanding of these kinds of meta-algorithms: it might have taken 3 billion years, but produced humans after all, so the potential of these methods is already proven by natural processes (which might not be replicable in the human world due to computational limitations, but at least we have a promising direction for now).

[timjrgh]

"Knowledge or Efficiency" problem is pretty close to so-called "Neats vs scruffies" problem in machine learning. You might want to see Neats vs scruffies wikipage.

Also, great video, it always interesting to hear your thoughts. Hope, there will be more videos like this one in the future

[michaelemouse]

I suppose part of the disagreement is based on how thoroughly one wishes to know the processes. There's plenty of knowledge where we have a good idea of the starting conditions and the end conditions but we're fuzzy on what happens in-between. Think of, say, metallurgy before the advent of modern chemistry. They might have known that adding this bit of alloy and that much heat for this long would strengthen steel but not really known exactly what was happening. It can still be quite useful to publish that kind of information.

[Kram]

What papers do and don't get published seems to be kind of arbitrary - or, well, it's actually predictable but the metrics that predict what is published are kind of arbitrary - in general.
For instance, there is a huge bias towards "positive" results - i.e. results where a new experiment has been shown to result in what ever the author predicted it would. Seemingly nobody cares about "negative" results (predictions failed to hold water) even though very often, from failure, we can learn a whole lot more than from success: Just ask "why does it fail" and do a followup experiment to confirm your failure hypothesis.
Though I agree that, at least for now, neural networks have a bit of an insight problem. We need to (and some groups already do) develop techniques that really help us analyze why a particular configuration worked well when a slightly different one did not, or what, exactly*, the NN does to accomplish its tasks.

*careful here: "exactly" can have multiple meanings in this context. For instance, we do exactly know the algorithm that drives the network towards its behavior. However, this is exact in the same way how knowing assembly code of a complex program is exact. We want to understand the behavior exactly although on a significantly higher level of abstraction. For instance, what particular subset of neurons do what particular task? - This is where "Deep Dreaming" and derivative techniques come into play. They try to visualize just that.
Could we separate out a couple neurons so they can still fulfill their task? Could we build a, uh, "FrankenBrain" out of thousands of such sub units but from different training runs, which would almost solve a task at a similar performance?
Could we build such a brain to fulfill multiple different tasks? As long as we can't, at least approximately, do that, I'm not sure that we really understand what's going on, in an abstract way. (Though I'm also unsure as to whether that could even work. Most of these NNs are fully laterally connected so it's hard to imagine that you could actually section out small parts without significantly altering outcomes, unless we are somehow directly encouraging to make regions separable, which does happen in some evolutionary NN approaches, but those are less powerful than the hand-built ones.

Often quite different networks end up with almost the same output. Can we somehow reach optimality in some sense? Is there a single NN (or a small set or perhaps a Lie-ish group thereof) which is able to optimally classify a given task with a given base architecture? If there is and if we can successfully reach it, then that's, in a sense, very tangible new highlevel knowledge.

[bernardfinucane]

A lot of work is currently being done on different neural networks to reverse engineer them.

[thomasstarzynski]

am i the only one who got distracted by this amazing gameplay?

[jojojorisjhjosef]

"Research" papers being rejected from what? They're still a great source if they can be found anywhere.

[NeoShameMan]

Micro singularity lol

Anyway:
knowledge is for academics
efficiency is for business

Resources will be reallocated accordingly, it's happening

Now on the stupid thought:
I wonder if we can have neural network that are train to recognize and describe function by looking at neural data, like we do for photo annotation :) train the network to annotate itself! Now can we ahev anough network we have some insight, does this exist? So we could use insight to discover new insight and iterate until we get more and more insight as training data lol

I'm writing science fiction lol

[testok]

why not have AI explain what it is doing? then we hand PhD's to AI's instead of humans. ...or Nobel Prizes