# Signal sampling

## Discrete representation of continuous signals and converse

When we can approximate discrete systems with continuous ones and vice versa. Sampling theorems. Nyquist rates, Compressive sampling, nonuniform signal sampling, signatures of rough paths, etc.

TODO: just write this as a basis decomp.

There are a few ways to frame this. Traditionally in electrical engineering applications we talk about Shannon sampling theorems, Nyquist rates and so on. The received-wisdom version of the Shannon theorem is that you can reconstruct a signal if you know it has frequencies in it that are “too high”. Specifically, if you sample a continuous time signal at intervals of $$T$$ seconds, then you had better have no frequencies of period shorter than $$2T$$.1 If you do much non-trivial signal processing, (in my case I constantly need to do things like multiplying signals) it rapidly becomes impossible to maintain bounds on the support of the spectrogram (TODO explain this with diagrams).

This doesn’t tell us much about more bizarre non-uniform sampling regimes, mild violations of frequency constraints, or whether other sets of (perhaps more domain-appropriate) constraints on our signals will lead to a sensible reconstruction theory.

More abstractly there is a Hilbert-space framing of this problem This way is general, and based on projections between Hilbert spaces. Nice works in this tradition are, e.g. that observes that you don’t care about Fourier spectrogram support, but rather the rate of degrees of freedom to construct a coherent sampling theory. Also accessible is , which constructs the problem of discretising signals as a minimal-loss projection/reconstruction problem.

More recently you have fancy persons such as Adcock and Hansen unifying compressed sensing and signal sampling with more or less the same framework. Looks interesting.

Most of the above use some variant of minimum $$L_2$$ norm error when reconstructing the signal. However, there are more reconstruction errors; for example I might wish to find some representation of a signal which is best with respect to some kind of transformation, e.g. in an inverse problem.

Or I might wish to sample a random signal, which is especially useful in functional Bayes inverse problems.

## References

Adcock, Ben, and Anders C. Hansen. 2016. Foundations of Computational Mathematics 16 (5): 1263–323.
Adcock, Ben, Anders C. Hansen, and Bogdan Roman. 2015. In Compressed Sensing and Its Applications: MATHEON Workshop 2013, edited by Holger Boche, Robert Calderbank, Gitta Kutyniok, and Jan Vybíral, 143–67. Applied and Numerical Harmonic Analysis. Cham: Springer International Publishing.
Adcock, Ben, Anders Hansen, Bogdan Roman, and Gerd Teschke. 2014. In Advances in Imaging and Electron Physics, edited by Peter W. Hawkes, 182:187–279. Elsevier.
Aldroubi, Akram, and Karlheinz Gröchenig. 2001. SIAM Review 43 (4): 585–620.
Amini, Arash, and Farokh Marvasti. 2008. Sampling Theory in Signal & Image Processing 7 (2).
Babu, Prabhu, and Petre Stoica. 2010. Digital Signal Processing 20 (2): 359–78.
Baisch, Stefan, and Götz H. R. Bokelmann. 1999. Computers & Geosciences 25 (7): 739–50.
Bartlett, M. S. 1946. Supplement to the Journal of the Royal Statistical Society 8 (1): 27–41.
Borcea, Liliana, Vladimir Druskin, and Leonid Knizhnerman. 2005. Communications on Pure and Applied Mathematics 58 (9): 1231–79.
Bostan, E., U. S. Kamilov, M. Nilchian, and M. Unser. 2013. IEEE Transactions on Image Processing 22 (7): 2699–2710.
Brémaud, Pierre, Laurent Massoulié, and Andrea Ridolfi. 2005. Advances in Applied Probability 37 (4): 1116–46.
Bretó, Carles, Daihai He, Edward L. Ionides, and Aaron A. King. 2009. The Annals of Applied Statistics 3 (1): 319–48.
Broersen, P. M. T., and R. Bos. 2006. In IEEE Transactions on Instrumentation and Measurement, 55:1124–31.
Broersen, Petrus MT. 2006. Automatic Autocorrelation and Spectral Analysis. Secaucus, NJ, USA: Springer.
Broersen, Piet M. T. 2005. IFAC Proceedings Volumes, 16th IFAC World Congress, 38 (1): 154–59.
Broersen, Piet M. T., Stijn de Waele, and Robert Bos. 2004. Automatica 40 (9): 1495–1504.
Bui-Thanh, Tan, and Quoc P. Nguyen. 2016. Inverse Problems & Imaging 10 (4): 943.
Cauchemez, Simon, and Neil M. Ferguson. 2008. Journal of The Royal Society Interface 5 (25): 885–97.
Cochran, W.T., James W. Cooley, D.L. Favin, H.D. Helms, R.A. Kaenel, W.W. Lang, Jr. Maling G.C., D.E. Nelson, C.M. Rader, and Peter D. Welch. 1967. Proceedings of the IEEE 55 (10): 1664–74.
Coulaud, Benjamin, and Frédéric JP Richard. 2018.
D’Ambrogi, Barbara, Sari Mäenpää, and Markku Markkanen. 1999. Geophysica 35 (1-2): 87–99.
Dumitrescu, Bogdan. 2017. Positive trigonometric polynomials and signal processing applications. Second edition. Signals and communication technology. Cham: Springer.
Eldar, Y. C., and A. V. Oppenheim. 2000. IEEE Transactions on Signal Processing 48 (10): 2864–75.
Feichtinger, Hans G., and Karlheinz Gröchenig. 1989. In Multivariate Approximation Theory IV, 135–42. International Series of Numerical Mathematics / Internationale Schriftenreihe Zur Numerischen Mathematik / Série Internationale d’Analyse Numérique. Birkhäuser Basel.
———. 1992. SIAM Journal on Mathematical Analysis 23 (1): 244–61.
———. 1994. Wavelets: Mathematics and Applications 1994: 305–63.
Feichtinger, Hans G., Karlheinz Gröchenig, and Thomas Strohmer. 1995. Numerische Mathematik 69 (4): 423–40.
Feichtinger, Hans G., and Thomas Strohmer. 1992. In SpringerLink, 231:82–89. Springer Berlin Heidelberg.
Feichtinger, Hans G., and Thomas Werther. 2000. In IEEE International Conference on Acoustics, Speech, and Signal Processing, 2000. ICASSP ’00. Proceedings, 6:3834–3837 vol.6.
Fessler, Jeffrey A., and Bradley P. Sutton. 2003. IEEE Transactions on Signal Processing 51 (2).
Finzi, Marc, Roberto Bondesan, and Max Welling. 2020. arXiv:2010.10876 [Cs], October.
García, Antonio G. 2002. In Advances in Imaging and Electron Physics, edited by Peter W. Hawkes, 124:63–137. Elsevier.
Gray, R. 1984. IEEE ASSP Magazine 1 (2): 4–29.
Greengard, L., and J. Lee. 2004. SIAM Review 46 (3): 443–54.
Gröchenig, Karlheinz. 1992. Mathematics of Computation 59 (199): 181–94.
———. 1993. Linear Algebra and Its Applications 193 (November): 129–50.
Jones, Richard H. 1981. “Fitting a Continuous Time Autoregression to Discrete Data.” In Applied Time Series Analysis II, 651–82.
———. 1984. In Time Series Analysis of Irregularly Observed Data, 158–88. Springer.
Kazhdan, Michael, Matthew Bolitho, and Hugues Hoppe. 2006. In SGP06: Eurographics Symposium on Geometry Processing, 1:0. The Eurographics Association.
Lahalle, E., G. Fleury, and A. Rivoira. 2004. In Proceedings of the 21st IEEE Instrumentation and Measurement Technology Conference, 2004. IMTC 04, 2:923–927 Vol.2.
Landau, H. J. 1967. Acta Mathematica 117 (1): 37–52.
Larsson, Erik K., and Torsten Söderström. 2002. Automatica 38 (4): 709–18.
Lasanen, Sari. 2002. “Discretizations of Generalized Random Variables with Applications to Inverse Problems.”
Lassas, Matti, Eero Saksman, and Samuli Siltanen. 2009. Inverse Problems and Imaging 3 (1): 87–122.
Lii, Keh-Shin, and Elias Masry. 1992. Journal of Multivariate Analysis 41 (1): 56–79.
Luschgy, H. 1996. Theory of Probability & Its Applications 40 (1): 167–75.
Mandelbaum, Avi. 1984. Zeitschrift Für Wahrscheinlichkeitstheorie Und Verwandte Gebiete 65 (3): 385–97.
Maravic, I., and M. Vetterli. 2005. IEEE Transactions on Signal Processing 53 (8): 2788–2805.
Margolis, E., and Y.C. Eldar. 2008. IEEE Transactions on Signal Processing 56 (7): 2728–45.
Marple, S. Lawrence, Jr. 1987. Digital Spectral Analysis with Applications.
Martin, R. J. 1998. Signal Processing 69 (3): 229–48.
———. 1999. Signal Processing 77 (2): 139–57.
Marvasti, F. A., and L. Chuande. 1990. IEEE Transactions on Acoustics, Speech, and Signal Processing 38 (6): 1061–63.
Marvasti, F., M. Analoui, and M. Gamshadzahi. 1991. IEEE Transactions on Signal Processing 39 (4): 872–78.
Marvasti, Farokh. 2012. Nonuniform Sampling: Theory and Practice. Springer Science & Business Media.
Marziliano, P., M. Vetterli, and T. Blu. 2006. IEEE Transactions on Information Theory 52 (5): 2230–33.
Matheron, G. 1973. Advances in Applied Probability 5 (3): 439–68.
McCrorie, J. Roderick. 2002. Statistical Inference for Stochastic Processes 5 (3): 273–86.
Mishali, M., and Y. C. Eldar. 2009. IEEE Transactions on Signal Processing 57 (3): 993–1009.
Mishali, Moshe, and Yonina C. Eldar. 2010. IEEE Journal of Selected Topics in Signal Processing 4 (2): 375–91.
Mobli, Mehdi, and Jeffrey C. Hoch. 2014. Progress in Nuclear Magnetic Resonance Spectroscopy 83 (November): 21–41.
Murray-Smith, Roderick, and Barak A. Pearlmutter. 2005. In Deterministic and Statistical Methods in Machine Learning, edited by Joab Winkler, Mahesan Niranjan, and Neil Lawrence, 110–23. Lecture Notes in Computer Science. Springer Berlin Heidelberg.
Niinimäki, K., S. Siltanen, and V. Kolehmainen. 2007. Physics in Medicine and Biology 52 (22): 6663–78.
O’Callaghan, Simon Timothy, and Fabio T. Ramos. 2011. In Twenty-Fifth AAAI Conference on Artificial Intelligence.
Papavasiliou, Anastasia, and Kasia B. Taylor. 2016. arXiv:1612.02536 [Math, Stat], December.
Petra, Noemi, James Martin, Georg Stadler, and Omar Ghattas. 2014. SIAM Journal on Scientific Computing 36 (4): A1525–55.
Pikkarainen, Hanna Katriina. 2006. Inverse Problems 22 (1): 365–79.
Piroddi, Roberta, and Maria Petrou. 2004. In Advances in Imaging and Electron Physics, 132:109–65. Advances in Imaging and Electron Physics. Elsevier.
Särkkä, Simo. 2007. IEEE Transactions on Automatic Control 52 (9): 1631–41.
Scargle, Jeffrey D. 1981. “Studies in Astronomical Time Series Analysis. I-Modeling Random Processes in the Time Domain.” The Astrophysical Journal Supplement Series 45: 1–71.
Smith, Julius O. 2018. Center for Computer Research in Music and Acoustics (CCRMA), Stanford University.
Söderström, T., and M. Mossberg. 2000. Automatica 1 (36): 53–59.
Stark, Jaroslav. 2001. In Nonlinear Dynamics and Statistics, edited by Alistair I. Mees, 81–103. Birkhäuser Boston.
Stoica, Petre, and Niclas Sandgren. 2006. Digit. Signal Process. 16 (6): 712–34.
Strohmer, T. 1997. IEEE Transactions on Image Processing 6 (4): 540–48.
Sun, Qiyu, and Michael Unser. 2012. Advances in Computational Mathematics 36 (3): 399–441.
Tan, V. Y. F., and V. K. Goyal. 2008. IEEE Transactions on Signal Processing 56 (10): 5135–46.
Tarczynski, A., and N. Allay. 2004. IEEE Transactions on Signal Processing 52 (12): 3324–34.
Tobar, Felipe. 2019. Advances in Neural Information Processing Systems 32: 12749–59.
Tropp, J., J.N. Laska, M.F. Duarte, J.K. Romberg, and R.G. Baraniuk. 2010. “Beyond Nyquist: Efficient Sampling of Bandlimited Signals.” IEEE Transactions on Information Theory 56: 1–26.
Unser, M. 1999. IEEE Signal Processing Magazine 16 (6): 22–38.
———. 2000. Proceedings of the IEEE 88 (4): 569–87.
———. 2015. In 2015 International Conference on Sampling Theory and Applications (SampTA), 221–25.
Unser, M., A. Aldroubi, and M. Eden. 1992. IEEE Transactions on Information Theory 38 (1): 95–103.
Unser, Michael A. 1995. “General Hilbert Space Framework for the Discretization of Continuous Signal Processing Operators.” In Wavelet Applications in Signal and Image Processing III, 2569:51–62. International Society for Optics and Photonics.
Unser, Michael A., and Pouya Tafti. 2014. An Introduction to Sparse Stochastic Processes. New York: Cambridge University Press.
Unser, Michael, and Akram Aldroubi. 1992. In Wavelets, edited by Charles K Chui, 2:91–122. Wavelet Analysis and Its Applications. San Diego: Academic Press.
———. 1994. “A General Sampling Theory for Nonideal Acquisition Devices.” IEEE Transactions on Signal Processing 42 (11): 2915–25.
Unser, M., P. D. Tafti, A. Amini, and H. Kirshner. 2014. IEEE Transactions on Information Theory 60 (5): 3036–51.
Unser, M., P. D. Tafti, and Q. Sun. 2014. IEEE Transactions on Information Theory 60 (3): 1945–62.
Venkataramani, R., and Y. Bresler. 2000. IEEE Transactions on Information Theory 46 (6): 2173–83.
Vetterli, M., P. Marziliano, and T. Blu. 2002. IEEE Transactions on Signal Processing 50 (6): 1417–28.
Wolfe, Stephen James. 1982. Stochastic Processes and Their Applications 12 (3): 301–12.
Yadrenko, Mikhail Iosifovich. 1983. Spectral theory of random fields. Translation series in mathematics and engineering. New York, NY: Optimization Software.
Yaglom, A. M. 1987. Correlation Theory of Stationary and Related Random Functions. Volume II: Supplementary Notes and References. Springer Series in Statistics. New York, NY: Springer Science & Business Media.
Yaroslavsky, Leonid P., Gil Shabat, Benny G. Salomon, Ianir A. Ideses, and Barak Fishbain. 2009. Journal of the Optical Society of America A 26 (3): 566.
Yen, J. 1956. IRE Transactions on Circuit Theory 3 (4): 251–57.

1. I’m playing fast-and-loose with definitions here — the spectrum in this context is the continuous Fourier spectrogram.↩︎

### No comments yet. Why not leave one?

GitHub-flavored Markdown & a sane subset of HTML is supported.