Fun tricks in non-convex optimisation

2014-10-04 — 2022-07-14

Suspiciously similar content

Figure 1: In non-convex optimisations our ultimate destination depends upon the starting point.

Zeyuan ALLEN-ZHU: Recent Advances in Stochastic Convex and Non-Convex Optimization. Clear, has good pointers.

1 With symmetries

Zhang, Qu, and Wright (2022)

2 In phase retrieval

See phase retrieval.

3 References

Azizian, Iutzeler, Malick, et al. 2025. “The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations.”

Choromanska, Henaff, Mathieu, et al. 2015. “The Loss Surfaces of Multilayer Networks.” In Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics.

Jain, and Kar. 2017. Non-Convex Optimization for Machine Learning.

Soltanolkotabi, Javanmard, and Lee. 2019. “Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks.” IEEE Transactions on Information Theory.

Wright, and Ma. 2022. High-dimensional data analysis with low-dimensional models: Principles, computation, and applications.

Zhang, Qu, and Wright. 2022. “From Symmetry to Geometry: Tractable Nonconvex Problems.”