# Functional regression

January 5, 2016 — May 28, 2020

calculus
dynamical systems
functional analysis
Hilbert space
nonparametric
sparser than thou
time series

Statistics where the samples are not just data but whole curves and manifolds, or subsamples from them. Function approximation meets statisticsm, especially in Karhunen-Loève expansion

## 1 Regression using curves

Functional data analysis, […] is about the analysis of information on curves or functions. For example, these twenty traces of the writing of “fda” are curves in two ways: first, as static traces on the page that you see after the writing is finished, and second, as two sets functions of time, one for the horizontal “X” coordinate, and the other for the vertical “Y” coordinate.

FDA is a collection statistical techniques for answering questions like, “What are the main ways in which the curves vary from one writing to another?” In fact, most of the questions and problems associated with the usual multivariate data analyzed by statistical packages like SAS and SPSS have their functional counterparts.

But what is unique about functional data is the possibility of also using information on the rates of change or derivatives of the curves. We use slopes, curvatures, and other characteristics made available because these curves are intrinsically smooth, and we can use this information in many useful ways. For example, our high school physics tells us that force = mass times acceleration, and that suggests that we look at the acceleration or second derivative of the pen’s position as a function of time. What we see in the plot of the magnitudes of the acceleration vector is that acceleration hits nearly ten meters/second/second. That’s a lot of energy! Equally remarkable is the stability of these acceleration records from one trial to the next. Also, note that where the acceleration magnitudes are near zero, both the X and Y accelerations must simultaneously be zero. The brain seems to know what it’s doing!

Regression upon the shapes of curves entire. A stylishly nonparametric thing to do. Can be simpler than you’d think — just doing typical statistics on a functional basis, Hilbert-space-style. You can try to infer the differential operator that defines continuous dynamics. Apropos that, see the kernel trick. Many other nonparametric methods of function approximation, such as spline bases and density estimation, mixture models, and so on are generalised by functional data analysis representation.

See for the foundational spline-smoothing work, and check the big names textbooks the modern framing.

An interesting related question is how you align the curves that are your objects of study. That is a problem of warping.

## 2 Functional autoregression

I’m interested in functional autoregressive models. In these we are concerned with a curve evolving in time. AFAICT this idea originates from but has been generalised since then.

## 3 References

Arribas-Gil, and Romo. 2012. Biostatistics (Oxford, England).
Bathia, Yao, and Ziegelmann. 2010. Annals of Statistics.
Battey, Fan, Liu, et al. 2015. arXiv:1509.05457 [Math, Stat].
Battey, and Linton. 2014. Journal of Multivariate Analysis.
Battey, and Liu. 2013. arXiv:1308.3968 [Stat].
Battey, and Sancetta. 2013. Journal of Multivariate Analysis.
Bosq. 1998. Nonparametric Statistics for Stochastic Processes: Estimation and Prediction. Lecture Notes in Statistics 110.
Dupont, Kim, Eslami, et al. 2022. In Proceedings of the 39th International Conference on Machine Learning.
Eilers, and Marx. 1996. Statistical Science.
Ferraty, Laksaci, Tadj, et al. 2011. Electronic Journal of Statistics.
Ferraty, and Vieu, eds. 2006a. In Nonparametric Functional Data Analysis: Theory and Practice. Springer Series in Statistics.
———. 2006b. Nonparametric Functional Data Analysis: Theory and Practice. Springer Series in Statistics.
Han, and Shin. n.d.
Heinonen, and d’Alché-Buc. 2014. arXiv:1411.5172 [Cs, Stat].
Horváth, Hušková, and Kokoszka. 2010. Journal of Multivariate Analysis, Statistical Methods and Problems in Infinite-dimensional Spaces,.
Horváth, and Kokoszka. 2012a. In Inference for Functional Data with Applications. Springer Series in Statistics.
———. 2012b. Inference for functional data with applications. Springer series in statistics.
Hsing, and Eubank. 2015. Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators. Wiley Series in Probability and Statistics.
Kadri, Duflos, Preux, et al. 2016. The Journal of Machine Learning Research.
Koner, and Staicu. 2023. Annual Review of Statistics and Its Application.
Lian. 2007. Canadian Journal of Statistics.
Liu, Ray, and Hooker. 2014. arXiv:1411.4681 [Math, Stat].
Mirzargar, Whitaker, and Kirby. 2014. IEEE Transactions on Visualization and Computer Graphics.
Morris. 2015. Annual Review of Statistics and Its Application.
Paparoditis, and Sapatinas. 2014. arXiv:1409.4317 [Math, Stat].
Pham, and Panaretos. 2016. arXiv:1612.07197 [Math, Stat].
Ramsay, Hooker, and Graves. 2009. Functional Data Analysis with R and MATLAB.
Ramsay, and Silverman. 2005. Functional Data Analysis. Springer Series in Statistics.
Saha, and Balamurugan. 2020. In Advances in Neural Information Processing Systems.
Shang. 2014. AStA Advances in Statistical Analysis.
Sun, and Genton. 2011. Journal of Computational and Graphical Statistics.
Tavakoli, and Panaretos. 2016. Journal of the American Statistical Association.
Wahba. 1990. Spline Models for Observational Data.