Optimization Techniques - W2023- Lecture 11 (Non-Convex Optimization, Sequential Convex Programming)

The course "Optimization Techniques" (ENGG*6140, section 2) at the School of Engineering at the University of Guelph. Instructor: Benyamin Ghojogh
Lecture 11 covers non-convex optimization for solving optimization problems in which the cost is a non-convex function and/or the feasibility set of constraints is a non-convex set. We cover both local and global optimization methods where local methods are faster but do not guarantee finding the global optimum solution while global methods are slow but guarantee finding the global optimum solution.

Chapters:

0:00 - Explaining the ambiguity in the previous session
3:41 - Where to stop in branch and bound for integer linear programming
11:55 - Non-convex function
15:11 - Non-convex optimization problem
19:47 - Optimization landscape of neural networks is highly non-convex
33:41 - Non-convex optimization categories
37:47 - SCP: convex approximation
43:30 - SCP: convex approximation by Taylor series expansion
48:19 - SCP: convex approximation by particle method
51:48 - SCP: convex approximation by quasi-linearization
56:05 - SCP: formulation of trust region
1:00:54 - SCP: updating trust region
1:22:00 - Branch and bound method
1:41:46 - Important references in non-convex optimization

[TheCrmagic]

This lecture series is an extremely helpful resource, along with your new book.

[andresariaslondono]

Thank you for the video!!. Really useful to understand SCP. I have one question: When I have to update the trust region using phi and phi_hat, I must solve the mathematical models in equations (9) and (10)? or only substitute my solutions xk and xk_hat into the expressions for phi and phi_hat ?