Neural networks [7.9] : Deep learning - DBN pre-training

Показать описание

Hugo Larochelle

Рекомендации по теме

Комментарии

Thank you Hugo for all the time you put into making these edifying set of videos. You saved me a lot of time pin-pointing where to apply my efforts. Merry Christmas :)

williamkyburz

hi Hugo, if you don't mind, could you share the ppt(slides) that you used please?

hendriktampubolon

To make sure I understand correctly:
1. At each new hidden layer i+1, we are effectively training an RBM on q(h_i|x)
2. In the first new hidden layer (i=2), we happen to see a good choice of q(h_1|x) = p(h_1|x) by tying weights
3. Once we train this first new layer (i=2), q(h_1|x) =/= p(h_1|x)
4. In each following new hidden layer (i>2), is there no good choice of q(h_i|x)? Due to non-linearities, we cannot choose perfect weights, but what do we tend choose in practice?

aSeaofTroubles

Dear Hugo, can you tell me some real life application of deep belief network? what are the advantages of using DBNs instead of CNN? I read some papers about this DBN but I can't really understand how supervised DBNs work

minh

Please Could you help me by giving the slides(PPT) of this video? I need it so badly. Ofcourse you shared the link for slides which are pdf form. If you don't mind and if you have powerpoint I will be really happy. Thanks a lot

ninanashew

@16:00: Using probabilities instead of sampling: if I'm not mistaken, the vector of probabilities is just the mean vector in this case, so here we do exactly what we do, say, in dropout, where we sample during training and use the mean during testing/prediction. Basically we approximate E[f(z)] with f(E[z]) which is exact when f is linear (and gives inequalities when f is convex/concave).
In our case, if log p(h) is "linear enough" in a region of high probability wrt q(h|x), we should be fine. I guess that's a lot of if's... and my intuition is possibly wrong :(
BTW, aren't the sigmoid and the softmax functions the mean functions (inverse of the link function) of the bernoulli and multinoulli (categorical) distributions? So it seems we're using this approximation all over the place.

kiuhnmmnhuik

Hi Hugo, could you explain why do we only optimize P(h) but not P(v|h) when pre-training a deep belief network?

bomb

It is hard for me to understand your explanation because of the mathematics, but after reading many paper and stuck in the direction of h^1 to v, now it seems it is related to Gibbs sampling and thank you for resolving this problem. However, I wonder where is the output layer that provides the result?

rezarawassizadeh

Hi Hugo, at 7:46, can you please elaborate on how the expression at the bottom is equivalent to training an RBM on data generated from q(h1|x), please?

saurabhmehta

Since RBM is being used for pretraining, so hidden units can take only binary values?

sameermalik

Neural networks [7.9] : Deep learning - DBN pre-training

But what is a neural network? | Chapter 1, Deep learning

Neural Network In 5 Minutes | What Is A Neural Network? | How Neural Networks Work | Simplilearn

Explained In A Minute: Neural Networks

Recurrent Neural Networks - Ep. 9 (Deep Learning SIMPLIFIED)

Deep Learning | What is Deep Learning? | Deep Learning Tutorial For Beginners | 2023 | Simplilearn

Why Do Tree Based-Models Outperform Neural Nets on Tabular Data?

Simple explanation of convolutional neural network | Deep Learning Tutorial 23 (Tensorflow & Pyt...

Neural Networks Pt. 2: Backpropagation Main Ideas

Deep Learning chapter 2 part 3: Training Deep Neural Networks | Pretraining, Momentum, Adagrad Etc

Recurrent Neural Network | Lecture 9 | Deep Learning

How Does a Neural Network Work in 60 seconds? The BRAIN of an AI

Tutorial 9- Drop Out Layers in Multi Neural Network

Deep Dream Video- Unconventional Neural Networks p.9

Introduction to Deep Learning (I2DL 2023) - 9. Convolutional Neural Networks

An Introduction to Machine Learning: Deep Learning for Sequential Inputs (7/9)

Artificial neural networks (ANN) - explained super simple

Lecture 7 | Training Neural Networks II

Convolutional Neural Networks | CNN | Kernel | Stride | Padding | Pooling | Flatten | Formula

What's a Neural Network?

Neural Network For Handwritten Digits Classification | Deep Learning Tutorial 7 (Tensorflow2.0)

Deep Dream - Unconventional Neural Networks p.7

[Neural Network 7] Backpropagation Demystified: A Step-by-Step Guide to the Heart of Neural Networks

What happens *inside* a neural network?

Forward Propagation and Backward Propagation | Neural Networks | How to train Neural Networks

What happens inside a neural network?