04: Real Learning Is Feasible (100min)

Machine Learning From Data, Rensselaer Fall 2020.

Professor Malik Magdon-Ismail talks about real learning, not verification. We discuss how choosing a hypothesis from a hypothesis set that is fixed before you see the data poses problems for the Hoeffding bound. We can modify the Hoeffding bound to account for the ability to select a hypothesis from H, and the price that must be paid is exactly the size of the hypothesis set. This brings us to the two step process for real learninig: (1) Get Eout=Ein and (2) Get Ein=0. The theory holds for any target function but we can now argue that more complex target functions are harder to learn because they need more data. Lastly we bring our learning setup closer to practical settings by discussing noisy target functions and choice of error/risk measures.

This is the fourth lecture in a "theory" course focusing on the foundations of learning, as well as some of the more advanced techniques like support vector machines and neural (deep) networks that are used in practice.

Level of the course: Advanced undergraduate, beginning graduate. Knowledge of probability, linear algebra, and calculus is helpful.

[tusarswain]

Thanks Professor for making it public. Appreciate you energy and the clarity in which you are explaining things. Finally I am "Learning From Data" :)

[howLucaLB]

Can't believe how good these classes are. Thank you very much

[jllyfh]

This videos are gold, I can only thank you !!