Stochastic signal sampling

Discrete sample representation of continuous stochastic processes

May 30, 2017 — June 24, 2022

dynamical systems
Hilbert space
signal processing
statistics
time series
Figure 1

Signal sampling is the study of approximating continuous signals with discrete ones and vice versa. What if the signal you are trying to recover is random, but you have a model for that randomness, and can thus assign likelihoods (posterior probabilities even) to some sample paths?

This naturally arises in functional Bayes inverse problems. In the presence of observation noise, they might also frame it in terms of state filtering/smoothing. There is a brief summary of that framing in Draščić (2016).

I am especially interested in this in the context of non-Gaussian-process models, because everything more or less works already for stationary Gaussian processes. If you consider Lévy noise driving a linear SDE, there is some work done under the heading of sparse stochastic processes.

TBC.

1 References

Borcea, Druskin, and Knizhnerman. 2005. “On the Continuum Limit of a Discrete Inverse Spectral Problem on Optimal Finite Difference Grids.” Communications on Pure and Applied Mathematics.
Bostan, Kamilov, Nilchian, et al. 2013. “Sparse Stochastic Processes and Discretization of Linear Inverse Problems.” IEEE Transactions on Image Processing.
Broersen, Piet M. T. 2005. “Time Series Analysis for Irregularly Sampled Data.” IFAC Proceedings Volumes, 16th IFAC World Congress,.
Broersen, P. M. T., and Bos. 2006. “Estimating Time-Series Models from Irregularly Spaced Data.” In IEEE Transactions on Instrumentation and Measurement.
Bui-Thanh, and Nguyen. 2016. “FEM-Based Discretization-Invariant MCMC Methods for PDE-Constrained Bayesian Inverse Problems.” Inverse Problems & Imaging.
Chen, Zheng, Al Kontar, et al. 2020. “Stochastic Gradient Descent in Correlated Settings: A Study on Gaussian Processes.” In Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS’20.
Coulaud, and Richard. 2018. “A Consistent Framework for a Statistical Analysis of Surfaces Based on Generalized Stochastic Processes.”
D’Ambrogi, Mäenpää, and Markkanen. 1999. “Discretization Independent Retrieval of Atmospheric Ozone Profile.” Geophysica.
Draščić. 2016. “Sampling Reconstruction of Stochastic Signals– The Roots in the Fifties.” Austrian Journal of Statistics.
Glynn, and Sigman. 1998. “Independent Sampling of a Stochastic Process.” Stochastic Processes and Their Applications.
Jones. 1981. “Fitting a Continuous Time Autoregression to Discrete Data.” In Applied Time Series Analysis II.
———. 1984. “Fitting Multivariate Models to Unequally Spaced Data.” In Time Series Analysis of Irregularly Observed Data.
Lahalle, Fleury, and Rivoira. 2004. “Continuous ARMA Spectral Estimation from Irregularly Sampled Observations.” In Proceedings of the 21st IEEE Instrumentation and Measurement Technology Conference, 2004. IMTC 04.
Larsson, and Söderström. 2002. “Identification of Continuous-Time AR Processes from Unevenly Sampled Data.” Automatica.
Lasanen. 2002. “Discretizations of Generalized Random Variables with Applications to Inverse Problems.”
Lassas, Saksman, and Siltanen. 2009. “Discretization-Invariant Bayesian Inversion and Besov Space Priors.” Inverse Problems and Imaging.
Lii, and Masry. 1992. “Model Fitting for Continuous-Time Stationary Processes from Discrete-Time Data.” Journal of Multivariate Analysis.
Luschgy. 1996. “Linear Estimators and Radonifying Operators.” Theory of Probability & Its Applications.
Mandelbaum. 1984. “Linear Estimators and Measurable Linear Transformations on a Hilbert Space.” Zeitschrift Für Wahrscheinlichkeitstheorie Und Verwandte Gebiete.
Maravic, and Vetterli. 2005. “Sampling and Reconstruction of Signals with Finite Rate of Innovation in the Presence of Noise.” IEEE Transactions on Signal Processing.
Marziliano, Vetterli, and Blu. 2006. “Sampling and Exact Reconstruction of Bandlimited Signals with Additive Shot Noise.” IEEE Transactions on Information Theory.
Matheron. 1973. “The Intrinsic Random Functions and Their Applications.” Advances in Applied Probability.
Murray-Smith, and Pearlmutter. 2005. “Transformations of Gaussian Process Priors.” In Deterministic and Statistical Methods in Machine Learning. Lecture Notes in Computer Science.
Niinimäki, Siltanen, and Kolehmainen. 2007. “Bayesian multiresolution method for local tomography in dental x-ray imaging.” Physics in Medicine and Biology.
O’Callaghan, and Ramos. 2011. “Continuous Occupancy Mapping with Integral Kernels.” In Twenty-Fifth AAAI Conference on Artificial Intelligence.
Petra, Martin, Stadler, et al. 2014. “A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems, Part II: Stochastic Newton MCMC with Application to Ice Sheet Flow Inverse Problems.” SIAM Journal on Scientific Computing.
Pikkarainen. 2006. “State Estimation Approach to Nonstationary Inverse Problems: Discretization Error and Filtering Problem.” Inverse Problems.
Scargle. 1981. “Studies in Astronomical Time Series Analysis. I-Modeling Random Processes in the Time Domain.” The Astrophysical Journal Supplement Series.
Söderström, and Mossberg. 2000. “Performance evaluation of methods for identifying continuous-time autoregressive processes.” Automatica.
Sun, and Unser. 2012. “Left-Inverses of Fractional Laplacian and Sparse Stochastic Processes.” Advances in Computational Mathematics.
Tan, and Goyal. 2008. “Estimating Signals With Finite Rate of Innovation From Noisy Samples: A Stochastic Algorithm.” IEEE Transactions on Signal Processing.
Tobar. 2019. “Band-Limited Gaussian Processes: The Sinc Kernel.” Advances in Neural Information Processing Systems.
Unser, M. 2015. “Sampling and (Sparse) Stochastic Processes: A Tale of Splines and Innovation.” In 2015 International Conference on Sampling Theory and Applications (SampTA).
Unser, Michael A., and Tafti. 2014. An Introduction to Sparse Stochastic Processes.
Unser, M., Tafti, Amini, et al. 2014. “A Unified Formulation of Gaussian Vs Sparse Stochastic Processes - Part II: Discrete-Domain Theory.” IEEE Transactions on Information Theory.
Unser, M., Tafti, and Sun. 2014. “A Unified Formulation of Gaussian Vs Sparse Stochastic Processes—Part I: Continuous-Domain Theory.” IEEE Transactions on Information Theory.
Wolfe. 1982. “On a Continuous Analogue of the Stochastic Difference Equation Xn=[rho]Xn-1+Bn.” Stochastic Processes and Their Applications.
Yadrenko. 1983. Spectral theory of random fields. Translation series in mathematics and engineering.
Yaglom. 1987. Correlation Theory of Stationary and Related Random Functions. Volume II: Supplementary Notes and References. Springer Series in Statistics.