Converting Constrained Optimization to Unconstrained Optimization Using the Penalty Method

In this video we show how to convert a constrained optimization problem into an approximately equivalent unconstrained optimization problem using the penalty method.

Topics and timestamps:
0:00 – Introduction
3:00 – Equality constrained only problem
12:50 – Reformulate as approximate unconstrained problem
34:59 – Penalty functions for inequality constraints

References:
-Fiacco, Anothny V. and McCormick, Garth P., “Nonlinear Programming: Sequential Unconstrained Minimization Techniques,” Wiley, 1968.

-Jensen, Paul A. and Bard, Jonathan F., “Operations Research Models and Methods,” John Wiley & Sons, 2003

#optimization

[edwardmau]

[AE 512] Thanks for going in depth and defining every variable, makes it easier and much more clear to follow. I also now understand the explicit differences between constrained and unconstrained optimization, you showed how to use both in order to utilize the efficiencies of both.

[timproby]

[AE 512] The clear distinction and purpose between unconstrained and constrained optimization is excellent

[darylfishback-duran]

This was a fantastic video. I worked within MATLAB alongside the video and it was great to see all the ideas come together into the final plot showing the two constraints and the numerical minimum. The explanations are always clear and concise. Looking forward to the next ones!

[Gholdoian]

AE 512: Wow such a powerful yet simple way to reframe optimization routines to use basic optimization schemes.

[koshiroyamaguchi]

AA516: I have had vaguly understood constraind optimization ideas before, but this video cleared my understanding so much better. Thank you Prof. Lum!

[akshaymishra]

One of the BEST videos to understand the topic.

[AlexandraSurprise]

AA516: Allie S, THIS IS SO COOL! This type of mathematical manipulation is exactly what enticed me to go into math and engineering in the first place. I'm so excited to see the following videos!!

[Kumky]

AA516: I have gone over optimization several times in my education and struggled through it at times. This video helped clear up a lot of confusion

[milesrobertroane]

AA516: All of the Matlab visualizations were so helpful in understanding how the distance from the constraint impacts how much it is attracted in that direction!

[rowellcastro]

AA516: These penalty functions are very nice and simple to implement in matlab using fminsearch. Thanks for the lecture Professor.

[yaffetbedru]

AA516: The visuals helped tons in my understanding of the constraints and their solutions.

[chayweaver.]

AE512: This is a very cool visualization and new way of looking at constrained vs. unconstrained optimization.

[mayfu]

this is just amazing, can't express how grateful i am.

[justinhendrick]

Thanks for the lecture, Professor! One bit of constructive feedback. The audio is much louder while you're at the board than at the computer. Maybe doing some balancing of the loudness while editing the video together would help.

[manitaregmi]

AA 516 - Another great lecture, I like how you use both Mathematica and Matlab together along with your lecture to explain the material.

[davidtelgen]

AE 512: Great explanation, excited to use this on RCAM

[disturbed_singer]

thank you for the lecture. the video was very helpful. keep up the good work. thanks again

[bingxinyan]

It is beneficial for me to understand how to covert the constrained opt problem into un-constrained ones. And very helpful with those implementations on Matlab. In applying cubic spline regression in engineering, I found a lot of papers using the penalty function to avoid overfitting or using the integrated square second derivative cubic spline penalty. I am confused about adding the "avoid overfitting" penalty and why I chose that form penalty. Would it be possible to give us a video about those? Plus, would it be possible to provide us with a video about the implementations using python? Whether it is possible or not, I've learned a lot from this video. Again, thank you very much.

[WalkingDeaDJ]

Jason-AE512: This video appears to be a useful resource for understanding how to transform optimization problems, potentially valuable for students and professionals in fields like operations research or applied mathematics.

[jia-hueiju]

great video!
it's really really clear!
Thanks, hope to see more great optimization lectures