[ML/DL] Over-Parameterization: A Phenomenon Emerging from Large Models

Показать описание

In this video we discuss the phenomenon of over-parameterization. This phenomenon usually happens when a model has extremely huge number of parameters. In this case, the model does not usually overfit to the training data but gaining higher performance than expected. This phenomenon contradict to the conventional theory of machine learning. Hence, it is very important to know what is over-parameterization.

0:00:29 Here is the outline
0:00:59 Over-fitting in traditional machine learning
0:09:22 Over-parameterization
0:12:29 Why VC Bound could not work in over-parameterization?
0:13:08 Take away

Mark Chang

Рекомендации по теме

[ML/DL] Over-Parameterization: A Phenomenon Emerging from Large Models

[ML/DL] Over-Parameterization: A Phenomenon Emerging from Large Models

Optimization for Deep Learning (Momentum, RMSprop, AdaGrad, Adam)

Machine Learning-Bias And Variance In Depth Intuition| Overfitting Underfitting

Design Methods and Processes for ML/DL models

LLM Explained | What is LLM

Gradient Descent in 3 minutes

Suriya Gunasekar - Kernel and rich regimes in overparameterized linear models

MIT 6.S191 (2018): Deep Learning Limitations and New Frontiers

Curse of Dimensionality Easily explained| Machine Learning

DL with Python: Fundamentals of machine learning (Chapter 5)

This is why Deep Learning is really weird.

Why Deep Neural Networks Work So Well? | Deep Learning

ICLR 2021 Keynote - 'Geometric Deep Learning: The Erlangen Programme of ML' - M Bronstein

Deep Learning Foundations: Misha Belkin's Talk on deep learning through the prism of interpolat...

Overparameterization in machine learning: insights from linear models – Andrea Montanari

Regularization in machine learning | L1 and L2 Regularization | Lasso and Ridge Regression

'Optimization theories of neural networks with its statistical perspective'Prof. Taiji Suz...

Mikhail Belkin: From classical statistics to modern deep learning

Nadav Cohen - Analyzing Optimization and Generalization in DL via Dynamics of Gradient Descent

ECE 695E Data Analysis, Design of Experiment, ML Lecture 3: Physical and Empirical Distributions

Designing Machine Learning Systems | book summary | Read a book with me

At What Blood Sugar Level Does the Damage Begin?

Analyzing Optimization and Generalization in Deep Learning via Dynamics of Gradient Descent

Cosplay by b.tech final year at IIT Kharagpur