Mikhail Belkin: From classical statistics to modern deep learning

Recent empirical successes of deep learning have exposed significant gaps in our fundamental understanding of learning and optimization mechanisms. Modern best practices for model selection are in direct contradiction to the methodologies suggested by classical analyses. Similarly, the efficiency of SGD-based local methods used in training modern models, appeared at odds with the standard intuitions on optimization. In this talk, Mikhail Belkin (UC San Diego) presents evidence, empirical and mathematical, that necessitates revisiting classical statistical notions, such as over-fitting. He discusses the emerging understanding of generalization, and, in particular, the "double descent" risk curve, which extends the classical U-shaped generalization curve beyond the point of interpolation.

[aydaynil]

I am curious about the paper he references at 29:30 which gives bounds for linear regression case. I would appreciate it, if someone can share