Practical Approaches for Efficient Hyperparameter Optimization

Hyperparameters, the tuning knobs of machine learning algorithms, are instrumental for the generation of high-performing models. The tedious task of hyperparameter optimization (HPO) is nonetheless often reduced to manual optimization, humorously called ‘graduate student descent’, or unsophisticated grid search and random search [1], a situation often leading to results that are highly sensitive to hyperparameters.
In this talk, we will walk machine learning practitioners through guidelines for efficient hyperparameter optimization based on Oríon, an open source HPO framework. We will start by presenting practical approaches for the design of the search space, then provide guidelines to select hyperparameter optimization algorithms, and finally demonstrate how to leverage the pioneering Experiment Version Control provided by Oríon for more efficiency.

Speakers:
Yonggang Hu
IBM Distinguished Engineer, Applied HPC, AI and Quantum. He has been working on distributed computing, HPC, grid, cloud and big data analytics for the past 20 years. He is Chief Architect of Spectrum Computing products and IBM high performance AI runtime - Watson Machine Learning Accelerator.
Xavier Bouthillier

Resources: