Multi-objective optimisation

Optimising for an objective defined as weighted sum of multiple objectives of unknown weights can be difficult. Useful in multi task learning, for example, or in weighting regularisation in regression including neural nets.

Jonas Degrave and Ira Korshunova describe a method in How we can make machine learning algorithms tunable based on Platt and Barr (1987), which (I think?) introduced the field of Pareto optimisation.


Das, Indraneel, and John E. Dennis. 1997. “A Closer Look at Drawbacks of Minimizing Weighted Sums of Objectives for Pareto Set Generation in Multicriteria Optimization Problems.” Structural Optimization 14 (1): 63–69.
Jakob, Wilfried, and Christian Blume. 2014. Pareto Optimization or Cascaded Weighted Sum: A Comparison of Concepts.” Algorithms 7 (1): 166–85.
Kim, Il Yong, and O. L. De Weck. 2006. “Adaptive Weighted Sum Method for Multiobjective Optimization: A New Method for Pareto Front Generation.” Structural and Multidisciplinary Optimization 31 (2): 105–16.
Kim, Il Yong, and Oliver L. De Weck. 2005. “Adaptive Weighted-Sum Method for Bi-Objective Optimization: Pareto Front Generation.” Structural and Multidisciplinary Optimization 29 (2): 149–58.
Marler, R., and Jasbir Arora. 2010. The Weighted Sum Method for Multi-Objective Optimization: New Insights.” Structural and Multidisciplinary Optimization 41 (6): 853–62.
Platt, John C., and Alan H. Barr. 1987. Constrained Differential Optimization.” In Proceedings of the 1987 International Conference on Neural Information Processing Systems, 612–21. NIPS’87. Cambridge, MA, USA: MIT Press.
Ryu, Jong-hyun, Sujin Kim, and Hong Wan. 2009. “Pareto Front Approximation with Adaptive Weighted Sum Method in Multiobjective Simulation Optimization.” In Proceedings of the 2009 Winter Simulation Conference (WSC), 623–33. IEEE.

No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.