Wherein Multi-Task Models Are Presented as Means to Produce Multivariate Predictions From Univariate Losses, and Gaussian Process Formulations Are Described as a Natural Multivariate Extension.
On training a model which predicts several things at once.
This is a very ML way of phrasing things. In classical statistics, if we have fit some multivariate regressions by a likelihood-based procedure, then it produces multivariate output. No problem. However, in machine learning, we frequently fit based on univariate predictive loss, and it is not clear, or at least affordable, to translate these univariate predictions to multivariate ones without starting over and simply training lots of univariate prediction models. In that context, it is not foolish to ask about multivariate predictions and think of them as the task of developing a “multi-task model” as some kind of new thing.
Bonilla, Chai, and Williams. 2007. “Multi-Task Gaussian Process Prediction.” In Proceedings of the 20th International Conference on Neural Information Processing Systems. NIPS’07.
Evgeniou, and Pontil. 2004. “Regularized Multi-Task Learning.” In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’04.
Platt, and Barr. 1987. “Constrained Differential Optimization.” In Proceedings of the 1987 International Conference on Neural Information Processing Systems. NIPS’87.
Radford, Wu, Child, et al. 2019. “Language Models Are Unsupervised Multitask Learners.”