Time frequency analysis

Multiplying your exposure to uncertainty principles


The approximation of a non-stationary signal by many locally stationary signals.

Here I care more about the hack where you take a non-localised spectrogram and attempt to localise it over short windows of a long signal. That comes next.

Chromatic derivatives, Welch-style DTFT spectrograms, wavelets sometimes. Wigner distribution (which is sort-of a joint distribution over time and frequency). Constant Q transforms.

Much to learn here, even in the deterministic case.

I am especially interested in the Bayesian approach to this, a.k.a. probabilistic spectral analysis, which treats this as a problem in random functions.

TODO: In the classical setup we might still talk about distributions although these are usually Wigner distributions which quantify something related to time-frequency uncertainty rather than posterior likelihoods. I would like to explain that.


Ackroyd, M. H. 1970. “Instantaneous and Time-Varying Spectra—an Introduction.” Radio and Electronic Engineer 39 (3): 145–52. https://doi.org/10.1049/ree.1970.0022.
Ackroyd, Martin H. 1971. “Short-Time Spectra and Time-Frequency Energy Distributions.” The Journal of the Acoustical Society of America 50: 1229–31.
Claasen, TACM, and W. F. G. Mecklenbrauker. 1980. “The Wigner Distribution—A Tool for Time-Frequency Signal Analysis.” Philips J. Res 35 (3): 217–50.
Cohen, L. 1989. “Time-Frequency Distributions-a Review.” Proceedings of the IEEE 77 (7): 941–81. https://doi.org/10.1109/5.30749.
———. 1993. “The Scale Representation.” IEEE Transactions on Signal Processing 41 (12): 3275–92. https://doi.org/10.1109/78.258073.
Cohen, L., and T. Posch. 1985. “Positive Time-Frequency Distribution Functions.” IEEE Transactions on Acoustics, Speech, and Signal Processing 33 (1): 31–38. https://doi.org/10.1109/TASSP.1985.1164512.
Daubechies, I. 1990. “The Wavelet Transform, Time-Frequency Localization and Signal Analysis.” IEEE Transactions on Information Theory 36 (5): 961–1005. https://doi.org/10.1109/18.57199.
Davis, Geoffrey M., Stephane G. Mallat, and Zhifeng Zhang. 1994a. “Adaptive Time-Frequency Decompositions.” Optical Engineering 33 (7, 7): 2183–91. https://doi.org/10.1117/12.173207.
———. 1994b. “Adaptive Time-Frequency Decompositions with Matching Pursuit.” In Wavelet Applications, 2242:402–14. International Society for Optics and Photonics. https://doi.org/10.1117/12.170041.
Delft, Anne van, and Michael Eichler. 2015. “Data-Adaptive Estimation of Time-Varying Spectral Densities.” December 2, 2015. http://arxiv.org/abs/1512.00825.
Dörfler, Monika, Gino Velasco, Arthur Flexer, and Volkmar Klien. 2010. “Sparse Regression in Time-Frequency Representations of Complex Audio.” In. https://web.archive.org/web/20160803140912/http://smcnetwork.org/files/proceedings/2010/16.pdf.
Driedger, Jonathan, Mathias Muller, and Sebastian Ewert. 2014. “Improving Time-Scale Modification of Music Signals Using Harmonic-Percussive Separation.” IEEE Signal Processing Letters 21 (1, 1): 105–9. https://doi.org/10.1109/LSP.2013.2294023.
Driedger, Jonathan, and Meinard Müller. 2016. “A Review of Time-Scale Modification of Music Signals.” Applied Sciences 6 (2, 2): 57. https://doi.org/10.3390/app6020057.
Elowsson, Anders, and Anders Friberg. 2017. “Long-Term Average Spectrum in Popular Music and Its Relation to the Level of the Percussion.” In Audio Engineering Society Convention 142, 13. Audio Engineering Society.
Fano, R. M. 1950. “Short‐Time Autocorrelation Functions and Power Spectra.” The Journal of the Acoustical Society of America 22 (5): 546–50. https://doi.org/10.1121/1.1906647.
Gardner, Timothy J., and Marcelo O. Magnasco. 2006. “Sparse Time-Frequency Representations.” Proceedings of the National Academy of Sciences 103 (16): 6094–99. https://doi.org/10.1073/pnas.0601707103.
Goodwin, M., and M. Vetterli. 1997. “Atomic Decompositions of Audio Signals.” In 1997 IEEE ASSP Workshop on Applications of Signal Processing to Audio and Acoustics, 1997. https://doi.org/10.1109/ASPAA.1997.625601.
Griffin, D., and Jae Lim. 1984. “Signal Estimation from Modified Short-Time Fourier Transform.” IEEE Transactions on Acoustics, Speech, and Signal Processing 32 (2, 2): 236–43. https://doi.org/10.1109/TASSP.1984.1164317.
Hohmann, V. 2002. “Frequency Analysis and Synthesis Using a Gammatone Filterbank.” Acta Acustica United with Acustica 88 (3, 3): 433–42.
Ignjatovic, A. 2009. “Chromatic Derivatives and Local Approximations.” IEEE Transactions on Signal Processing 57 (8): 2998–3007. https://doi.org/10.1109/TSP.2009.2020749.
Ignjatovic, Aleksandar. 2007. “Local Approximations Based on Orthogonal Differential Operators.” Journal of Fourier Analysis and Applications 13 (3): 309–30. https://doi.org/10.1007/s00041-006-6085-y.
Irizarry, Rafael A. 2001. “Local Harmonic Estimation in Musical Sound Signals.” Journal of the American Statistical Association 96 (454, 454): 357–67. https://doi.org/10.1198/016214501753168082.
Janssen, A. J. 1984. “Gabor Representation and Wigner Distribution of Signals.” In, 9:258–61. Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICASSP.1984.1172739.
Kim, Duk Su, Young Han Lee, Hong Kook Kim, Song Ha Choi, Ji Woon Kim, and Myeong Bo Kim. 2010. “Complexity Reduction of WSOLA-Based Time-Scale Modification Using Signal Period Estimation.” In Communication and Networking, 120:155. Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-17604-3_17.
Krishnan, Sridhar. 2005. “A New Approach for Estimation of Instantaneous Mean Frequency of a Time-Varying Signal.” EURASIP J. Appl. Signal Process. 2005 (January): 2848–55. https://doi.org/10.1155/ASP.2005.2848.
Kronland-Martinet, R., Ph. Guillemain, and S. Ystad. 1997. “Modelling of Natural Sounds by Time–Frequency and Wavelet Representations.” Organised Sound 2 (03): 179–91. https://doi.org/null.
Lewicki, Michael S. 2002. “Efficient Coding of Natural Sounds.” Nature Neuroscience 5 (4): 356–63. https://doi.org/10.1038/nn831.
Mallat, S., and Z. Zhang. 1992. “Adaptive Time-Frequency Decomposition with Matching Pursuits.” In Time-Frequency and Time-Scale Analysis, 1992., Proceedings of the IEEE-SP International Symposium, 7–10. https://doi.org/10.1109/TFTSA.1992.274245.
Mallat, Stéphane G., and Zhifeng Zhang. 1993. “Matching Pursuits with Time-Frequency Dictionaries.” IEEE Transactions on Signal Processing 41 (12): 3397–3415. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=258082.
Masri, Paul, Andrew Bateman, and Nishan Canagarajah. 1997a. “A Review of Time–Frequency Representations, with Application to Sound/Music Analysis–Resynthesis.” Organised Sound 2 (03, 03): 193–205. https://doi.org/10.1017/S1355771898009042.
———. 1997b. “The Importance of the Time–Frequency Representation for Sound/Music Analysis–Resynthesis.” Organised Sound 2 (03, 03): 207–14. https://doi.org/10.1017/S1355771898009054.
Mecklenbräuker, W., and F. Hlawatsch, eds. 1997. The Wigner Distribution: Theory and Applications in Signal Processing. Amsterdam ; New York: Elsevier.
Moussallam, Manuel, Laurent Daudet, and Gaël Richard. 2012. “Matching Pursuits with Random Sequential Subdictionaries.” Signal Processing 92 (10): 2532–44. https://doi.org/10.1016/j.sigpro.2012.03.019.
Müller, M., D. P. W. Ellis, A. Klapuri, and G. Richard. 2011. “Signal Processing for Music Analysis.” IEEE Journal of Selected Topics in Signal Processing 5 (6, 6): 1088–1110. https://doi.org/10.1109/JSTSP.2011.2112333.
Necciari, T., P. Balazs, N. Holighaus, and P. L. Sondergaard. 2013. “The ERBlet Transform: An Auditory-Based Time-Frequency Representation with Perfect Reconstruction.” In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 498–502. https://doi.org/10.1109/ICASSP.2013.6637697.
Noll, A. Michael. 1967. “Cepstrum Pitch Determination.” The Journal of the Acoustical Society of America 41 (2): 293–309. https://doi.org/10.1121/1.1910339.
Preis, Douglas, and Voula Chris Georgopoulos. 1999. “Wigner Distribution Representation and Analysis of Audio Signals: An Illustrated Tutorial Review.” Journal of the Audio Engineering Society 47 (12): 1043–53. http://www.ece.rochester.edu/courses/ECE472/Site/Assignments/Entries/2009/2/9_Unit_2_-_Spectral_Analysis_files/Preis_1999_1.pdf.
Rafii, Z. 2018. “Sliding Discrete Fourier Transform with Kernel Windowing [Lecture Notes].” IEEE Signal Processing Magazine 35 (6, 6): 88–92. https://doi.org/10.1109/MSP.2018.2855727.
Rioul, O., and M. Vetterli. 1991. “Wavelets and Signal Processing.” IEEE Signal Processing Magazine 8 (4): 14–38. https://doi.org/10.1109/79.91217.
Scarpazza, Daniele Paolo. n.d. “A Brief Introduction to the Wigner Distribution,” 5. http://www.scarpaz.com/Attic/Documents/TheWignerDistribution.pdf.
Schroeder, M. R., and B. S. Atal. 1962. “Generalized ShortTime Power Spectra and Autocorrelation Functions.” The Journal of the Acoustical Society of America 34 (11): 1679–83. https://doi.org/10.1121/1.1909090.
Sejdic, Ervin, Igor Djurovic, and Jin Jianga. 2009. “Time–Frequency Feature Representation Using Energy Concentration: An Overview of Recent Advances.” Digital Signal Processing 19 (1): 153–83. https://doi.org/10.1016/j.dsp.2007.12.004.
Shafi, Imran, Jamil Ahmad, Syed Ismail Shah, and F. M. Kashif. 2009. “Techniques to Obtain Good Resolution and Concentrated Time-Frequency Distributions: A Review.” EURASIP Journal on Advances in Signal Processing 2009 (1): 673539. https://doi.org/10.1155/2009/673539.
Stankovic, L. J., and S. Stankovic. 1995. “An Analysis of Instantaneous Frequency Representation Using Time-Frequency Distributions-Generalized Wigner Distribution.” IEEE Transactions on Signal Processing 43 (2): 549–52. https://doi.org/10.1109/78.348139.
Szmajda, M., K. Gorecki, and J. Mroczka. 2010. “Gabor Transform, Gabor-Wigner Transform and SPWVD as a Time-Frequency Analysis of Power Quality.” In Proceedings of 14th International Conference on Harmonics and Quality of Power, 1–8. Bergamo, Italy: IEEE. https://doi.org/10.1109/ICHQP.2010.5625371.
Szmajda, M., and J. Mroczka. 2011. “Comparison of Gabor-Wigner Transform and SPWVD as Tools of Harmonic Computation.” Renewable Energy and Power Quality Journal, May, 386–92. https://doi.org/10.24084/repqj09.343.
Torrence, Christopher, and Gilbert P Compo. 1998. “A Practical Guide to Wavelet Analysis.” Bulletin of the American Meteorological Society 79 (1): 61–78. http://shadow.eas.gatech.edu/ kcobb/seminar/torrence.
Yu, Guoshen, and Jean-Jacques Slotine. 2009. “Audio Classification from Time-Frequency Texture.” In Acoustics, Speech, and Signal Processing, IEEE International Conference on, 0:1677–80. Los Alamitos, CA, USA: IEEE Computer Society. https://doi.org/10.1109/ICASSP.2009.4959924.
Zhao, Y., L. E. Atlas, and R. J. Marks. 1990. “The Use of Cone-Shaped Kernels for Generalized Time-Frequency Representations of Nonstationary Signals.” IEEE Transactions on Acoustics, Speech, and Signal Processing 38 (7): 1084–91. https://doi.org/10.1109/29.57537.