3. Introduction to Statistical Learning Theory

This is where our "deep study" of machine learning begins. We introduce some of the core building blocks and concepts that we use in this course: input space, action space, outcome space, prediction functions, loss functions, and hypothesis spaces. We also present empirical risk minimization, our first machine learning method. We highlight the issue of overfitting, which will occur when we find the empirical risk minimizer over too large a hypothesis space.