Sharpness-Aware Minimization (SAM) in 7 minutes

Thank you for checking out my video notes on the Sharpness-Aware Minimization (SAM) in 7 mins! I would love to share my ML learning journey with you.

Paper information:
- Foret, P., Kleiner, A., Mobahi, H., & Neyshabur, B. (2020). Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412.

Please let me know in the comment section regarding any questions, points of discussion, or anything you would like see next. See you in the next video!

[franciscobarragancastro]

I applied this technique a while back for a BERT like encoder for a SSL task and got much improved results. In your experience, what kind of tasks usually have a noise loss functions that benefit from applying this technique?

[santiagocalvo]

I could harldy understand anything you just said, but love your channel, you are awesome! What should I learn if I want to fully understand this? I have some math background, not a lot, thanks!

[yasaswijesekara]

Thanks for the awesome explanation! A quick question, how do you choose the perturbation values? Do you just sample the epsilon vector from a normal distribution?