Making the Perfect Cup of Joe - Quan Nguyen | PyData Global 2021

Making the Perfect Cup of Joe: Active Preference Learning and Optimization Under Uncertainty
Speaker: Quan Nguyen

Summary
Modeling user preferences over a feature space and efficiently identifying the highest-valued regions is a common problem in machine learning. This talk discusses a solution to this problem that leverages Bayesian optimization. In doing so, we offer a way to model users’ preferences when data is expensive to obtain and comes in the form of pairwise comparisons.

Description

Optimizing a user’s valuation of a product is a common problem in machine learning (ML), and an attractive solution to this kind of blackbox optimization problem is Bayesian optimization (BO). BO offers a principled way to quickly identify a user’s highest valuation by designing queries that balance the trade-off between exploration and exploitation. However, a limitation of existing solutions in this framework is that they only work with data on real-valued ratings (e.g., a score of 4 on a 5-point scale asking users to rate a product), and yet in many situations we only have access to information about the user’s preference for a product in one setting over another setting, i.e., pairwise comparison data.

To learn users’ valuation using these pairwise comparisions, we can leverage a technique called preference learning (PL). This talk presents how BO may be extended to PL, including using a Gaussian Process (GP) for modeling and designing optimization strategies. We will learn how to build an optimization loop of this kind and gain practical skills to do so , such as how to decide which strategy is effective in low- vs. high-dimensional spaces. This talk will benefit ML practitioners who work on human-in-the-loop, A/B testing, product recommendation, and user preference optimization problems. While the talk will cover most background knowledge necessary to implement this optimization technique, the audience should be familiar with common concepts in ML such as training data, predictive models, and multivariate normal distributions.

Quan Nguyen's Bio
Quan is a Python programmer and machine learning enthusiast. He is interested in solving decision-making problems under uncertainty. Quan has authored several books on Python programming and scientific computing. He is currently pursuing a Ph.D. degree in computer science at Washington University in St. Louis, researching Bayesian methods in machine learning.

PyData Global 2021

PyData is an educational program of NumFOCUS, a 501(c)3 non-profit organization in the United States. PyData provides a forum for the international community of users and developers of data analysis tools to share ideas and learn from each other. The global PyData network promotes discussion of best practices, new approaches, and emerging technologies for data management, processing, analytics, and visualization. PyData communities approach data science using many languages, including (but not limited to) Python, Julia, and R.

PyData conferences aim to be accessible and community-driven, with novice to advanced level presentations. PyData tutorials and talks bring attendees the latest project features along with cutting-edge use cases.

00:00 Welcome!
00:10 Help us add time stamps or captions to this video! See the description for details.