Gradient descent, constrained
KKT in first-order problems.
2017-08-07 — 2017-08-07
Wherein constrained gradient descent is examined by Lagrange multipliers and Karush–Kuhn–Tucker conditions, saddle points are sought, and primal–dual formulations with L_p norms are considered.
Placeholder; I need to update this with real info, given how often I need to know it.
1 Lagrange multipliers
Constrained optimisation using Lagrange’s one weird trick, and the Karush—Kuhn—Tucker conditions. The search for saddle points and roots.
2 Duals
The types of optimisation problems you can create from a given set of constraints and objectives, based on primal and dual formulations.
There are several different duals and I don’t know their relations. Legendre-Fenchel dual, Lagrange dual, Wolfe dual… however these all work.
🏗 Discuss role of \(L_p\) norms.