AutoML20: A Modern Guide to Hyperparameter Optimization

Modern deep learning model performance is very dependent on the choice of model hyperparameters, and the tuning process is a major bottleneck in the machine learning pipeline. The talk will first motivate the need for advancements in hyperparameter tuning methods. The talk will then overview standard methods for hyperparameter tuning: grid search, random search, and bayesian optimization. Then, we will motivate and discuss cutting edge methods for hyperparameter tuning: multi-fidelity bayesian optimization, successive halving algorithms (HyperBand), and population-based training. The talk will then present an overview of Tune, a scalable hyperparameter tuning system from the UC Berkeley RISELab, and demonstrate how users can leverage cutting edge hyperparameter tuning methods implemented in Tune to quickly improve the performance of standard deep learning models

Speaker: Richard Liaw, Anyscale