# Signal sampling

Discrete representation of continuous signals and converse

May 30, 2017 — June 24, 2022

dynamical systems
functional analysis
Hilbert space
signal processing
statistics
time series

When we can approximate discrete systems with continuous ones and vice versa. Sampling theorems. Nyquist rates, Compressive sampling, nonuniform signal sampling, signatures of rough paths, etc.

TODO: just write this as a basis decomp.

There are a few ways to frame this. Traditionally in electrical engineering applications we talk about Shannon sampling theorems, Nyquist rates and so on. The received-wisdom version of the Shannon theorem is that you can reconstruct a signal if you know it has frequencies in it that are “too high”. Specifically, if you sample a continuous time signal at intervals of $$T$$ seconds, then you had better have no frequencies of period shorter than $$2T$$.1 If you do much non-trivial signal processing, (in my case I constantly need to do things like multiplying signals) it rapidly becomes impossible to maintain bounds on the support of the spectrogram (TODO explain this with diagrams).

This doesn’t tell us much about more bizarre non-uniform sampling regimes, mild violations of frequency constraints, or whether other sets of (perhaps more domain-appropriate) constraints on our signals will lead to a sensible reconstruction theory.

More abstractly there is a Hilbert-space framing of this problem This way is general, and based on projections between Hilbert spaces. Nice works in this tradition are, e.g. that observes that you don’t care about Fourier spectrogram support, but rather the rate of degrees of freedom to construct a coherent sampling theory. Also accessible is , which constructs the problem of discretising signals as a minimal-loss projection/reconstruction problem.

More recently you have fancy persons such as Adcock and Hansen unifying compressed sensing and signal sampling with more or less the same framework. Looks interesting.

Most of the above use some variant of minimum $$L_2$$ norm error when reconstructing the signal. However, there are more reconstruction errors; for example I might wish to find some representation of a signal which is best with respect to some kind of transformation, e.g. in an inverse problem.

Or I might wish to sample a random signal, which is especially useful in functional Bayes inverse problems.

## 1 References

Adcock, and Hansen. 2016. Foundations of Computational Mathematics.
Adcock, Hansen, Roman, et al. 2014. In Advances in Imaging and Electron Physics.
Adcock, Hansen, and Roman. 2015. In Compressed Sensing and Its Applications: MATHEON Workshop 2013. Applied and Numerical Harmonic Analysis.
Aldroubi, and Gröchenig. 2001. SIAM Review.
Amini, and Marvasti. 2008. Sampling Theory in Signal & Image Processing.
Babu, and Stoica. 2010. Digital Signal Processing.
Baisch, and Bokelmann. 1999. Computers & Geosciences.
Bartlett. 1946. Supplement to the Journal of the Royal Statistical Society.
Borcea, Druskin, and Knizhnerman. 2005. Communications on Pure and Applied Mathematics.
Bostan, Kamilov, Nilchian, et al. 2013. IEEE Transactions on Image Processing.
Brémaud, Massoulié, and Ridolfi. 2005. Advances in Applied Probability.
Bretó, He, Ionides, et al. 2009. The Annals of Applied Statistics.
Broersen, Piet M. T. 2005. IFAC Proceedings Volumes, 16th IFAC World Congress,.
Broersen, Petrus MT. 2006. Automatic Autocorrelation and Spectral Analysis.
Broersen, P. M. T., and Bos. 2006. In IEEE Transactions on Instrumentation and Measurement.
Broersen, Piet M. T., de Waele, and Bos. 2004. Automatica.
Bui-Thanh, and Nguyen. 2016. Inverse Problems & Imaging.
Cauchemez, and Ferguson. 2008. Journal of The Royal Society Interface.
Cochran, Cooley, Favin, et al. 1967. Proceedings of the IEEE.
Coulaud, and Richard. 2018.
D’Ambrogi, Mäenpää, and Markkanen. 1999. Geophysica.
Dumitrescu. 2017. Positive trigonometric polynomials and signal processing applications. Signals and communication technology.
Eldar, and Oppenheim. 2000. IEEE Transactions on Signal Processing.
Feichtinger, and Gröchenig. 1989. In Multivariate Approximation Theory IV. International Series of Numerical Mathematics / Internationale Schriftenreihe Zur Numerischen Mathematik / Série Internationale d’Analyse Numérique.
———. 1992. SIAM Journal on Mathematical Analysis.
———. 1994. Wavelets: Mathematics and Applications.
Feichtinger, Gröchenig, and Strohmer. 1995. Numerische Mathematik.
Feichtinger, and Strohmer. 1992. In SpringerLink.
Feichtinger, and Werther. 2000. In IEEE International Conference on Acoustics, Speech, and Signal Processing, 2000. ICASSP ’00. Proceedings.
Fessler, and Sutton. 2003. IEEE Transactions on Signal Processing.
Finzi, Bondesan, and Welling. 2020. arXiv:2010.10876 [Cs].
García. 2002. In Advances in Imaging and Electron Physics.
Gray. 1984. IEEE ASSP Magazine.
Greengard, and Lee. 2004. SIAM Review.
Gröchenig. 1992. Mathematics of Computation.
———. 1993. Linear Algebra and Its Applications.
Jones. 1981. “Fitting a Continuous Time Autoregression to Discrete Data.” In Applied Time Series Analysis II.
———. 1984. In Time Series Analysis of Irregularly Observed Data.
Kazhdan, Bolitho, and Hoppe. 2006. In SGP06: Eurographics Symposium on Geometry Processing.
Lahalle, Fleury, and Rivoira. 2004. In Proceedings of the 21st IEEE Instrumentation and Measurement Technology Conference, 2004. IMTC 04.
Landau. 1967. Acta Mathematica.
Lasanen. 2002. “Discretizations of Generalized Random Variables with Applications to Inverse Problems.”
Lassas, Saksman, and Siltanen. 2009. Inverse Problems and Imaging.
Lii, and Masry. 1992. Journal of Multivariate Analysis.
Luschgy. 1996. Theory of Probability & Its Applications.
Mandelbaum. 1984. Zeitschrift Für Wahrscheinlichkeitstheorie Und Verwandte Gebiete.
Maravic, and Vetterli. 2005. IEEE Transactions on Signal Processing.
Margolis, and Eldar. 2008. IEEE Transactions on Signal Processing.
Marple. 1987. Digital Spectral Analysis with Applications.
Martin. 1998. Signal Processing.
———. 1999. Signal Processing.
Marvasti, Farokh. 2012. Nonuniform Sampling: Theory and Practice.
Marvasti, F., Analoui, and Gamshadzahi. 1991. IEEE Transactions on Signal Processing.
Marvasti, F. A., and Chuande. 1990. IEEE Transactions on Acoustics, Speech, and Signal Processing.
Marziliano, Vetterli, and Blu. 2006. IEEE Transactions on Information Theory.
Matheron. 1973. Advances in Applied Probability.
McCrorie. 2002. Statistical Inference for Stochastic Processes.
Mishali, M., and Eldar. 2009. IEEE Transactions on Signal Processing.
Mishali, Moshe, and Eldar. 2010. IEEE Journal of Selected Topics in Signal Processing.
Mobli, and Hoch. 2014. Progress in Nuclear Magnetic Resonance Spectroscopy.
Murray-Smith, and Pearlmutter. 2005. In Deterministic and Statistical Methods in Machine Learning. Lecture Notes in Computer Science.
Niinimäki, Siltanen, and Kolehmainen. 2007. Physics in Medicine and Biology.
O’Callaghan, and Ramos. 2011. In Twenty-Fifth AAAI Conference on Artificial Intelligence.
Papavasiliou, and Taylor. 2016. arXiv:1612.02536 [Math, Stat].
Petra, Martin, Stadler, et al. 2014. SIAM Journal on Scientific Computing.
Pikkarainen. 2006. Inverse Problems.
Piroddi, and Petrou. 2004. In Advances in Imaging and Electron Physics. Advances in Imaging and Electron Physics.
Särkkä. 2007. IEEE Transactions on Automatic Control.
Scargle. 1981. “Studies in Astronomical Time Series Analysis. I-Modeling Random Processes in the Time Domain.” The Astrophysical Journal Supplement Series.
Smith. 2018.
Söderström, and Mossberg. 2000. Automatica.
Stark. 2001. In Nonlinear Dynamics and Statistics.
Stoica, and Sandgren. 2006. Digit. Signal Process.
Strohmer. 1997. IEEE Transactions on Image Processing.
Sun, and Unser. 2012. Advances in Computational Mathematics.
Tan, and Goyal. 2008. IEEE Transactions on Signal Processing.
Tarczynski, and Allay. 2004. IEEE Transactions on Signal Processing.
Tobar. 2019. Advances in Neural Information Processing Systems.
Tropp, Laska, Duarte, et al. 2010. “Beyond Nyquist: Efficient Sampling of Bandlimited Signals.” IEEE Transactions on Information Theory.
Unser, Michael A. 1995. “General Hilbert Space Framework for the Discretization of Continuous Signal Processing Operators.” In Wavelet Applications in Signal and Image Processing III.
Unser, M. 1999. IEEE Signal Processing Magazine.
———. 2000. Proceedings of the IEEE.
———. 2015. In 2015 International Conference on Sampling Theory and Applications (SampTA).
Unser, Michael, and Aldroubi. 1992. In Wavelets. Wavelet Analysis and Its Applications.
———. 1994. “A General Sampling Theory for Nonideal Acquisition Devices.” IEEE Transactions on Signal Processing.
Unser, M., Aldroubi, and Eden. 1992. IEEE Transactions on Information Theory.
Unser, Michael A., and Tafti. 2014. An Introduction to Sparse Stochastic Processes.
Unser, M., Tafti, Amini, et al. 2014. IEEE Transactions on Information Theory.
Unser, M., Tafti, and Sun. 2014. IEEE Transactions on Information Theory.
Venkataramani, and Bresler. 2000. IEEE Transactions on Information Theory.
Vetterli, Marziliano, and Blu. 2002. IEEE Transactions on Signal Processing.
Wolfe. 1982. Stochastic Processes and Their Applications.
Yadrenko. 1983. Spectral theory of random fields. Translation series in mathematics and engineering.
Yaglom. 1987. Correlation Theory of Stationary and Related Random Functions. Volume II: Supplementary Notes and References. Springer Series in Statistics.
Yaroslavsky, Shabat, Salomon, et al. 2009. Journal of the Optical Society of America A.
Yen. 1956. IRE Transactions on Circuit Theory.

## Footnotes

1. I’m playing fast-and-loose with definitions here — the spectrum in this context is the continuous Fourier spectrogram.↩︎