Neural networks [6.7] : Autoencoder - contractive autoencoder

Показать описание

Hugo Larochelle

Рекомендации по теме

Комментарии

I just want to say you, 'Thank u; in person.An awesome representation!!

NikhilYadala

Hi Hugo! Great video..I really appreciate that you have taken some time out of your busy schedule for educating people like us :-)

Thanks for this video, it's very clear with you representation!

junjieli

@4:30 I don't understand why you say that the regularizer throws away all information in x_k. It throws away only the *noise* in x_k and it does that by flattening out h near x_k, AFAICT.
To clarify, your explanation is 90% aligned with my intuition, but there's that "throws away all information" part which I don't get.
On second thought, I think I get what you mean. You mean that if the regularizer is not contrasted by the reconstruction term then it will throw away all information, right?
But that doesn't need to be the case as it's enough that the jacobian is 0 at the samples (edit: but we're using continuous functions...). But maybe SGD is good at finding degenerate/uninteresting solutions, which is why we needed the regularizer in the first place (i.e. to avoid the "copying").

kiuhnmmnhuik

Hİ Hugo, Are the contractive autoencoders related with contraction mappings? Since the regularization term could be thought as a squared Lİpschitz constant of the h which may give some degree of control on the learned manifold? If not, why? Thanks!

emirceyani

Thank you so much for this video. Love the visualisations!!

laimeilin

Thank you Mr. Larochelle for this really helpful presentation.

behradtajalli

I have a question -- Can auto encoders be used for building good word embeddings (like word2vec, GloVe etc.)? I have a TREC style text corpus. My target is to find contextually similar words
using cosine similarity between their embedding vectors. For example, I have a word "emergency" and I want to find words like "crisis", disaster" etc. which are supposed to have high contextual similarity
(hence high cosine similarity) with "emergency". Thanks in advance!

Great explanation thank you! Which book do you recommend (apart from Deep Learning Goodfellow book) to deep dive into this concepts?

jamespaz

thank you, this explains the intuition behinds the CAE quite well..I finally get it!

planktonscollective

So for those inputs (x_ks) which dont change much you do not want significant weights. But that partial derivate of h(x_t) w.r.t x_k is just weight isnt it? And if so this sounds a lot like regular L2 norm for the purposes of regularization. Am I missing something?

kimishpatel

Hi Hugo, what happens when you combine contractive AE and DAE? So you perturb your input, compute the loss with respect to the untouched input, but have the jacobian in the loss function as well?

Herrgolani

But isn't it violating the purpose of dimensionality reduction?

TechFinTv

Great video ! Intuition explained to the point. Thanks a lot

MohitTare

Neural networks [6.7] : Autoencoder - contractive autoencoder

Neural networks [7.6] : Deep learning - deep autoencoder

Autoencoders - EXPLAINED

Simple Explanation of AutoEncoders

MIT 6.S191 (2023): Deep Generative Modeling

Neural networks [6.2] : Autoencoder - loss function

Tutorial 7 - Autoencoders and GANs | Deep Learning on Computational Accelerators

What are AutoEncoders in deep learning? - explained

Variational Autoencoders

Autoencoders for Data Compression and Reconstruction - Neural Networks

Autoencoders in Deep Learning

Neural networks [6.5] : Autoencoder - undercomplete vs. overcomplete hidden layer

Build an Autoencoder in 5 Min - Fresh Machine Learning #5

Autoencoders in Keras and TensorFlow for Data Compression and Reconstruction - Neural Networks

MIT 6.S191: Deep Generative Modeling

Deep Learning(CS7015): Lec 7.1 Introduction to Autoncoders

Autoencoders Tutorial | Autoencoders In Deep Learning | Tensorflow Training | Edureka

Autoencoder Explained

Neural Networks and Clustering (Autoencoders)

Variational Autoencoders - EXPLAINED!

Types of Neural Networks | FNN, ANN, RNN, LSTM, Autoencoder, GAN, RBFN, Transformer, SOM, BM

L16.4 A Convolutional Autoencoder in PyTorch -- Code Example

What are Convolutional Neural Networks (CNNs)?

4. Autoencoders

Auto Encoders - lecture 46/ machine learning