Bengio, Yoshua. 2000. “Gradient-Based Optimization of Hyperparameters.” Neural Computation
12 (8): 1889–1900.
Bergstra, James S., Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. “Algorithms for Hyper-Parameter Optimization.”
In Advances in Neural Information Processing Systems
, 2546–54. Curran Associates, Inc.
Bergstra, J, D Yamins, and D D Cox. 2013. “Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures.” In ICML, 9.
Domke, Justin. 2012. “Generic Methods for Optimization-Based Modeling.”
In International Conference on Artificial Intelligence and Statistics
Eggensperger, Katharina, Matthias Feurer, Frank Hutter, James Bergstra, Jasper Snoek, Holger H. Hoos, and Kevin Leyton-Brown. n.d. “Towards an Empirical Foundation for Assessing Bayesian Optimization of Hyperparameters.”
Eigenmann, R., and J. A. Nossek. 1999. “Gradient Based Adaptive Regularization.”
In Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468)
Elsken, Thomas, Jan Hendrik Metzen, and Frank Hutter. 2019. “Neural Architecture Search: A Survey.” arXiv:1808.05377 [Cs, Stat]
Feurer, Matthias, Aaron Klein, Katharina Eggensperger, Jost Springenberg, Manuel Blum, and Frank Hutter. 2015. “Efficient and Robust Automated Machine Learning.”
In Advances in Neural Information Processing Systems 28
, edited by C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, 2962–70. Curran Associates, Inc.
Foo, Chuan-sheng, Chuong B. Do, and Andrew Y. Ng. 2008. “Efficient Multiple Hyperparameter Learning for Log-Linear Models.”
In Advances in Neural Information Processing Systems 20
, edited by J. C. Platt, D. Koller, Y. Singer, and S. T. Roweis, 377–84. Curran Associates, Inc.
Fu, Jie, Hongyin Luo, Jiashi Feng, Kian Hsiang Low, and Tat-Seng Chua. 2016. “DrMAD: Distilling Reverse-Mode Automatic Differentiation for Optimizing Hyperparameters of Deep Neural Networks.”
In PRoceedings of IJCAI, 2016
Gelbart, Michael A., Jasper Snoek, and Ryan P. Adams. 2014. “Bayesian Optimization with Unknown Constraints.”
In Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence
, 250–59. UAI’14. Arlington, Virginia, United States: AUAI Press.
Grünewälder, Steffen, Jean-Yves Audibert, Manfred Opper, and John Shawe-Taylor. 2010. “Regret Bounds for Gaussian Process Bandit Problems.”
Hutter, Frank, Holger H. Hoos, and Kevin Leyton-Brown. 2011. “Sequential Model-Based Optimization for General Algorithm Configuration.”
In Learning and Intelligent Optimization
, 6683:507–23. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer, Berlin, Heidelberg.
Hutter, Frank, Holger Hoos, and Kevin Leyton-Brown. 2013. “An Evaluation of Sequential Model-Based Optimization for Expensive Blackbox Functions.”
In Proceedings of the 15th Annual Conference Companion on Genetic and Evolutionary Computation
, 1209–16. GECCO ’13 Companion. New York, NY, USA: ACM.
Li, Lisha, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. 2017. “Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization.” The Journal of Machine Learning Research
18 (1): 6765–6816.
Liu, Hanxiao, Karen Simonyan, and Yiming Yang. 2019. “DARTS: Differentiable Architecture Search.” arXiv:1806.09055 [Cs, Stat]
Maclaurin, Dougal, David Duvenaud, and Ryan Adams. 2015. “Gradient-Based Hyperparameter Optimization Through Reversible Learning.”
In Proceedings of the 32nd International Conference on Machine Learning
, 2113–22. PMLR.
Močkus, J. 1975. “On Bayesian Methods for Seeking the Extremum.”
In Optimization Techniques IFIP Technical Conference: Novosibirsk, July 1–7, 1974
, edited by G. I. Marchuk, 400–404. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer.
Real, Esteban, Chen Liang, David R. So, and Quoc V. Le. 2020. “AutoML-Zero: Evolving Machine Learning Algorithms From Scratch,”
Salimans, Tim, Diederik Kingma, and Max Welling. 2015. “Markov Chain Monte Carlo and Variational Inference: Bridging the Gap.”
In Proceedings of the 32nd International Conference on Machine Learning (ICML-15)
, 1218–26. ICML’15. Lille, France: JMLR.org.
Snoek, Jasper, Hugo Larochelle, and Ryan P. Adams. 2012. “Practical Bayesian Optimization of Machine Learning Algorithms.”
In Advances in Neural Information Processing Systems
, 2951–59. Curran Associates, Inc.
Snoek, Jasper, Kevin Swersky, Rich Zemel, and Ryan Adams. 2014. “Input Warping for Bayesian Optimization of Non-Stationary Functions.”
In Proceedings of the 31st International Conference on Machine Learning (ICML-14)
Srinivas, Niranjan, Andreas Krause, Sham M. Kakade, and Matthias Seeger. 2012. “Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design.” IEEE Transactions on Information Theory
58 (5): 3250–65.
Swersky, Kevin, Jasper Snoek, and Ryan P Adams. 2013. “Multi-Task Bayesian Optimization.”
In Advances in Neural Information Processing Systems 26
, edited by C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, 2004–12. Curran Associates, Inc.
Thornton, Chris, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. 2013. “Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms.”
In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
, 847–55. KDD ’13. New York, NY, USA: ACM.