#86 - Prof. YANN LECUN and Dr. RANDALL BALESTRIERO - SSL, Data Augmentation [NEURIPS2022]

Yann LeCun is a French computer scientist known for his pioneering work on convolutional neural networks, optical character recognition and computer vision. He is a Silver Professor at New York University and Vice President, Chief AI Scientist at Meta. Along with Yoshua Bengio and Geoffrey Hinton, he was awarded the 2018 Turing Award for their work on deep learning, earning them the nickname of the "Godfathers of Deep Learning".

Dr. Randall Balestriero has been researching learnable signal processing since 2013, with a focus on learnable parametrized wavelets and deep wavelet transforms. His research has been used by NASA, leading to applications such as Marsquake detection. During his PhD at Rice University, Randall explored deep networks from a theoretical perspective and improved state-of-the-art methods such as batch-normalization and generative networks. Later, when joining Meta AI Research (FAIR) as a postdoc with Prof. Yann LeCun, Randall further broadened his research interests to include self-supervised learning and the biases emerging from data-augmentation and regularization, resulting in numerous publications.

Note: We have another full interview with Randall, which we will release soon as part of a show focussed on Spline Theory of NNs.

TOC:

[00:00:00] LeCun interview
[00:18:25] Randall Balestriero interview (mostly on spectral SSL paper, first ref)

References:

[Randall Balestriero, Yann LeCun] Contrastive and Non-Contravention Self-Supervised Learning Recover Global and Local Spectral Embedding Methods

[Randall Balestriero, Ishan Misra, Yann LeCun] A Data-Augmentation Is Worth A Thousand Samples: Exact Quantification From Analytical Augmented Sample Moments

[Bobak Kiani, Randall Balestriero, Yann LeCun, Seth Lloyd] projUNN: efficient method for training deep networks with unitary matrices

[Randall Balestriero, Richard G. Baraniuk]A Spline Theory of Deep Networks

Learning in High Dimension Always Amounts to Extrapolation [Randall Balestriero, Jerome Pesenti, Yann LeCun]

[Mathilde Caron et al] DINO - Emerging Properties in Self-Supervised Vision Transformers

[Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton] A Simple Framework for Contrastive Learning of Visual Representations (SIMCLR)

[Yann LeCun] A Path Towards Autonomous Machine Intelligence Version

[Patrice Y. Simard, Yann A. LeCun et al]
Transformation Invariance in Pattern Recognition – Tangent Distance and Tangent Propagation

[Kaiming He et al] Masked Autoencoders Are Scalable Vision Learners

[Radford et al] Whisper - Robust Speech Recognition via Large-Scale Weak Supervision

RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank [Quentin Garrido, Randall Balestriero, Laurent Najman, Yann Lecun]

[David Silver, Satinder Baveja, Doina Precup, Richard Sutton] Reward is Enough

[nokar]

I'm glad we're starting to see so many previous interviewees again, especially when they are such interesting thinkers. Good to hear iteration on previous conversation and their new ideas.

[AICoffeeBreak]

Just wow. I love it how you let everybody get a taste of the best bits at NeurIPS. ❤

[bissbort]

Great content and production quality once again! Keep up the great work.

[paxdriver]

Yan is so so funny sometimes lol

"the purpose of reinforcement learning should be to minimize the use of reinforcement learning, because reinforcement learning is so damn inefficient; *pardon my French*" 😂

[simonstrandgaard]

Thank you for covering this conference.

[grape]

Very lucky to interview someone who is never, ever wrong!

[johntanchongmin]

Really inspired by Yann Lecun! Hope we can find a better way to do RL in the future!

[nafizabdoulcarime]

Does anyone know from where the clip at 17:17 is taken ?

[xXKMUXx]

This interview is the definition of 'ML street talk' aha. With engineering, biology etc its right or wrong. When it comes to A.I it is a debate.

[FergalByrne]

The duality is due to the duality between normals to the manifold and vectors in the tangent space. The latter is more efficient as any other data point (or an augmented data point) will approximate geodesics on the manifold. Most efficient (as Yann mentions) is use of previous or next frames which actually are on geodesics. Both the neocortex and the Feynman Machine use this succession and are thus orders of magnitude more efficient than these methods.

[bytesizedbraincog]

Is reinforcement learning really inefficient? I really liked the architecture where reinforcement learning is brought in the scope of text generation, especially chatGPT model, it looked like it had some plausible reasoning because of reinforcement learning.

[fyodorminakov]

Nearest neighbor does extrapolate... that's what the k value is for. The problem with KNN is the k is just not all that predictive, because it's fixed for all, and even worse, each N is unweighted in a given context, so there is massive information loss. Converting KNN to a neural net fixes the problem.

[TheReferrer]

When are you guys going to have a play with ChatGPT?

Would love if you include Dr. Walid Saba if you do.

[charlesb.]

Come on Yann !
Even the French accent
is distortioned . 😂
Tell us instead what is the real purpose of all this IA predictions.I
n real life IA will be sadly used in order to control private life of people around the world against their will or real consentement .

[michaelhartjen]

He is copying from jeff hawkins with his HTM's. Where the key part is the Temporal Adjancency Matrix.