Karolina Dziugaite on Nonvacuous Generalization Bounds for Deep Neural Networks via PAC-Bayes

Показать описание

Abstract: Karolina presents her recent work constructing generalization bounds in order to understand existing learning algorithms and propose new ones. Generalization bounds relate empirical performance to future expected performance. The tightness of these bounds vary widely, and depends on the complexity of the learning task and the amount of data available, but also on how much information the bounds take into consideration. Her work is particularly concerned with data and algorithm-dependent bounds that are quantitatively nonvacuous. She presents bounds built from solutions obtained by stochastic gradient descent (SGD) on MNIST. By formalizing the notion of flat minima using PAC-Bayes generalization bounds, we obtain nonvacuous generalization bounds for stochastic classifiers built by randomly perturbing SGD solutions.

Lecture given in on June 2018 in Toronto, ON at the Borealis AI lab.

Рекомендации по теме

Karolina Dziugaite on Nonvacuous Generalization Bounds for Deep Neural Networks via PAC-Bayes

Karolina Dziugaite on Nonvacuous Generalization Bounds for Deep Neural Networks via PAC-Bayes

Gintare Karolina Dziugaite - Distribution-dependent generalization bounds

Progress towards understanding generalization in Deep Learning | Gintarė Karolina Džiugaitė

PAC-Bayesian approaches to understanding generalization in deep learning - Gintare Dziugaite

Gintarė Karolina Džiugaitė, Towards explaining generalization for interpolating predictors

IMG_0706

Lecture 8: Progress Towards Understanding Generalization in Deep Learning (English)

IMG_0711

Studying Generalization in Deep Learning via PAC-Bayes

IMG_0707

Size-free Generalization Bounds for Convolutional Neural Networks

Generalization Bounds for Uniformly Stable Algorithms

Understanding Generalization in Deep Learning

A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks (Talk)

PAC-Bayesian Contrastive Unsupervised Representation Learning

Uniform convergence may be unable to explain generalization in deep learning (NeurIPS19 oral paper)

Generalization Error Bounds for Extreme Multi-class Classification

Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks

Tightening information-theoretic generalization bounds with data-dependent estimate... - Daniel Roy

NeurIPS 2019:Generalization Bounds of stochastic Gradient Descent for Wide and Deep Neural Networks

MLSS 2021 Taipei- An intro to Statistical Learning Theory and PAC-Bayes Analysis (John Shawe-Taylor)

Statistical Learning Theory Part 6: Generalization Bound

Elad Granot - Generalization Bounds for Neural Networks via Approximate Description Length

PAC-Bayes control for obstacle avoidance