Optimization Techniques - W2023 - Lecture 2 (Preliminaries)

The course "Optimization Techniques" (ENGG*6140, section 2) at the School of Engineering at the University of Guelph. Instructor: Benyamin Ghojogh
Lecture 2 includes some preliminaries on optimization, including norm, cone, convex function, Lipschitz continuity (smoothness), local and global minimizers, stationary points, first-order optimality condition, arguments of optimization, converting minimization to maximization and vice versa, gradient, Jacobian, and Hessian.

Chapters:

0:00 - Discussions about previous session
21:56 - Norm
25:48 - Important norms for vectors
32:22 - Important norms for matrices
47:08 - Quadratic forms
50:35 - Unit balls
55:02 - Dual norm
1:02:35 - Cone and dual cone
1:09:22 - Generalized inequality
1:15:20 - Break in the session
1:20:12 - Convex function
1:33:57 - Strongly convex function
1:36:14 - Lipschitz continuity
1:43:50 - local and global minimizers
1:50:48 - Stationary, extremum, and saddle points
1:53:58 - First-order optimality condition
1:54:45 - Arguments of optimization
1:56:57 - Converting minimization to maximization and vice versa
2:08:12 - Dimensionality of derivative
2:21:48 - Gradient, Jacobian, and Hessian
2:25:34 - Technique for calculating derivative
2:29:29 - Review of next session's topics