Understanding self supervised learning dynamics without contrastive | Outstanding Paper | ICML 2021

preview_player
Показать описание

0:00 Understanding self-supervised Learn Dynamics without Contrastive Pair
0:27 Non-contrastive SSL (BYOL/SimSiam)
1:37 A simple model
2:45 The Dynamics of Training Procedure
4:26 Part II Assumptions
5:56 Symmetrized Dynamics Under the three assumptions, the dynamics becomes
7:13 Why non-contrastive SSL doesn't collapse?
9:31 Part III The Effect of Weight Decay n
10:06 The Benefit of Weight Decay
11:20 Exponential Moving Average rateſ
11:52 Part IV Direct Pred
14:33 Performance of DirectPred on ImageNet
15:53 Conclusion

Award-Winning | Outstanding Paper | Best Honorable Mention Award Papers of ICLR and ICML 2021 Conferences
Momentum Residual Neural Networks | ICML 2021
Score-Based Generative Modeling through Stochastic Differential Equations | Best Paper | ICLR 2021
Rethinking Architecture Selection in Differentiable NAS | Best Paper Award | ICLR 2021
Optimal Rates for Averaged Stochastic Gradient Descent under Neural Tangent Kernel Regim | ICLR 2021
Neural Synthesis of Binaural Speech From Mono Audio | Best Paper Award | ICLR 2021
Learning Mesh-Based Simulation with Graph Networks | Best Paper Award | ICLR 2021
EigenGame: PCA as a Nash Equilibrium | Outstanding Paper Award | ICLR 2021
Solving high-dimensional parabolic PDE using the tensor train format | Outstanding Paper | ICML 2021
Understanding self-supervised learning dynamics without contrastive | Outstanding Paper | ICML 2021
Oops I Took A Gradient: Scalable Sampling for Discrete | Outstanding Honorable Mention | ICML 2021
Optimal Complexity in Decentralized Training | Outstanding Paper Honorable Mention | ICML 2021
Unbiased Gradient Estimation in Unrolled Computation Graphs | Outstanding Paper Award | ICML 2021
H / T : Khawar Islam
#computervision #machinelearning #artificialintelligence #AI
  1. Artificial Intelligence
  2. computer vision
  3. deep learning
  4. machine learning
  5. research
  6. ai labs
Рекомендации по теме
Understanding self supervised 0:17:08

Understanding self supervised learning dynamics without contrastive | Outstanding Paper | ICML 2021

A quick review 0:01:01

A quick review of Self-supervised Representation Learning Across Sequential and Tabular Features

Self Supervised Learning 0:01:20

Self Supervised Learning

Self-Supervised Learning to 0:03:10

Self-Supervised Learning to Reconstruct Dynamic Scenarios at Scale - NVIDIA DRIVE Labs Ep. 33

On the Learning 0:03:18

On the Learning Dynamics of Semi-Supervised Training for ASR - (3 minutes introduction)

Using self-supervised and 0:19:50

Using self-supervised and active learning for data curation

Andrew Ng's Secret 0:00:48

Andrew Ng's Secret to Mastering Machine Learning - Part 1 #shorts

Self-Supervised Learning | 1:04:13

Self-Supervised Learning | Self-supervision for Learning from the Bottom Up by Alexei A. Efros

Inductive Learning: The 0:08:42

Inductive Learning: The Future of AI #ai #python #coding #neuralnetwork #youtube #tutorial

Self-Supervised Learning - 0:31:15

Self-Supervised Learning - Towards Autonomously Learning Machines—Simon Stiebellehner

Time-Contrastive Networks: Self-Supervised 0:03:00

Time-Contrastive Networks: Self-Supervised Learning from Video

2022.04 Self supervised 1:20:06

2022.04 Self supervised Learning - Miguel Sarabia, Jason Ramapuram, Dan Busbridge

TerraPN: Unstructured Terrain 0:02:41

TerraPN: Unstructured Terrain Navigation Through Online Self-Supervised Learning

An Embedding-Dynamic Approach 0:04:02

An Embedding-Dynamic Approach to Self-Supervised Learning

Self-Supervised Learning, 2022 0:49:14

Self-Supervised Learning, 2022

[CVPR 2020 Tutorial] 0:52:18

[CVPR 2020 Tutorial] Talk #5 Self-supervised Learning by Licheng Yu, Yen-Chun Chen and Linjie Li

[Deep Learning] Self-supervised 0:54:43

[Deep Learning] Self-supervised Learning

Self-Supervised Learning by 1:24:49

Self-Supervised Learning by Yann LeCun

DINO: Emerging Properties 0:31:54

DINO: Emerging Properties in Self-Supervised Vision Transformers | Paper Explained!

Self-Supervised Learning 0:28:43

Self-Supervised Learning

Self-Supervised Correspondence in 0:05:00

Self-Supervised Correspondence in Visuomotor Policy Learning

Neural Networks explained 0:01:00

Neural Networks explained in 60 seconds!

Self-Supervised Learning of 0:01:01

Self-Supervised Learning of Video-Induced Visual Invariances

Self-Supervised Learning of 0:01:47

Self-Supervised Learning of Appliance Usage - ICLR 2020