Gradient descent, constrained
KKT in first-order problems.
2017-08-07 — 2017-08-07
Wherein Constrained Gradient Descent Is Examined by Lagrange Multipliers and Karush–Kuhn–Tucker Conditions, Saddle Points Are Sought, and Primal–dual Formulations With L_p Norms Are Considered.
Placeholder; I need to update this with real info, given how often I need to know it.
1 Lagrange multipliers
Constrained optimisation using Lagrange’s one weird trick, and the Karush—Kuhn—Tucker conditions. The search for saddle points and roots.
2 Duals
The types of optimisation problems you can create from a given set of constraints and objectives, based on primal and dual formulations.
There are several different duals and I don’t know their relations. Legendre-Fenchel dual, Lagrange dual, Wolfe dual… however these all work.
🏗 Discuss role of \(L_p\) norms.