Fun tricks in non-convex optimisation

October 4, 2014 — July 14, 2022

functional analysis

optimization

statmech

Figure 1: In non-convex optimisations our ultimate destination depends upon the starting point.

Zeyuan ALLEN-ZHU: Recent Advances in Stochastic Convex and Non-Convex Optimization. Clear, has good pointers.

1 With symmetries

Zhang, Qu, and Wright (2022)

2 In phase retrieval

See phase retrieval.

3 References

Choromanska, Henaff, Mathieu, et al. 2015. “The Loss Surfaces of Multilayer Networks.” In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics.

Jain, and Kar. 2017. Non-Convex Optimization for Machine Learning.

Soltanolkotabi, Javanmard, and Lee. 2019. “Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks.” IEEE Transactions on Information Theory.

Wright, and Ma. 2022. High-dimensional data analysis with low-dimensional models: Principles, computation, and applications.

Zhang, Qu, and Wright. 2022. “From Symmetry to Geometry: Tractable Nonconvex Problems.”