Time frequency analysis

Multiplying your exposure to uncertainty principles

January 6, 2018 — December 21, 2021

functional analysis
Hilbert space
signal processing


The approximation of a non-stationary signal by many locally stationary signals, which is a thing we might do in an analysis or synthesis procedure. One of the many places where uncertainty principles come into play. If we let the window size shrink to a single sample, then we are looking instead at empirical mode decompositions.

Chromatic derivatives, Welch-style DTFT spectrograms, wavelets sometimes. Wigner distribution (which is sort-of a joint distribution over time and frequency). Constant Q transforms.

Much to learn here, even in the deterministic case.

I am especially interested in the Bayesian approach to this, a.k.a. probabilistic spectral analysis, which treats this as a problem in random functions.

TODO: In the classical setup we might still talk about distributions although these are usually Wigner distributions not probability distribution, which quantify something related to time-frequency uncertainty rather than posterior likelihoods. I would like to understand that.

1 Effect of windows

2 Adaptive windows

The Adaptspec methods (Bertolacci et al. 2020; Rosen, Wood, and Stoffer 2012) assign a probabilisty distribution to possible locally stationary windows converting this into a probabilistic spectral problems. Without an explicit spectrogram, so does Saatçi, Turner, and Rasmussen (2010).

3 References

Ackroyd, M. H. 1970. Instantaneous and Time-Varying Spectra—an Introduction.” Radio and Electronic Engineer.
Ackroyd, Martin H. 1971. “Short-Time Spectra and Time-Frequency Energy Distributions.” The Journal of the Acoustical Society of America.
Bertolacci, Rosen, Cripps, et al. 2020. AdaptSPEC-X: Covariate Dependent Spectral Modeling of Multiple Nonstationary Time Series.” arXiv:1908.06622 [Stat].
Claasen, and Mecklenbrauker. 1980. “The Wigner Distribution—A Tool for Time-Frequency Signal Analysis.” Philips J. Res.
Cochran, Cooley, Favin, et al. 1967. What Is the Fast Fourier Transform? Proceedings of the IEEE.
Cohen. 1989. Time-Frequency Distributions-a Review.” Proceedings of the IEEE.
———. 1993. The Scale Representation.” IEEE Transactions on Signal Processing.
Cohen, and Posch. 1985. Positive Time-Frequency Distribution Functions.” IEEE Transactions on Acoustics, Speech, and Signal Processing.
Cooley, Lewis, and Welch. 1970. The Application of the Fast Fourier Transform Algorithm to the Estimation of Spectra and Cross-Spectra.” Journal of Sound and Vibration.
Daubechies. 1990. The Wavelet Transform, Time-Frequency Localization and Signal Analysis.” IEEE Transactions on Information Theory.
Davis, Mallat, and Zhang. 1994a. Adaptive Time-Frequency Decompositions.” Optical Engineering.
———. 1994b. Adaptive Time-Frequency Decompositions with Matching Pursuit.” In Wavelet Applications.
Dörfler, Velasco, Flexer, et al. 2010. Sparse Regression in Time-Frequency Representations of Complex Audio.” In.
Driedger, and Müller. 2016. A Review of Time-Scale Modification of Music Signals.” Applied Sciences.
Driedger, Muller, and Ewert. 2014. Improving Time-Scale Modification of Music Signals Using Harmonic-Percussive Separation.” IEEE Signal Processing Letters.
Elowsson, and Friberg. 2017. “Long-Term Average Spectrum in Popular Music and Its Relation to the Level of the Percussion.” In Audio Engineering Society Convention 142.
Fano. 1950. Short‐Time Autocorrelation Functions and Power Spectra.” The Journal of the Acoustical Society of America.
Gardner, and Magnasco. 2006. Sparse Time-Frequency Representations.” Proceedings of the National Academy of Sciences.
Goodwin, and Vetterli. 1997. Atomic Decompositions of Audio Signals.” In 1997 IEEE ASSP Workshop on Applications of Signal Processing to Audio and Acoustics, 1997.
Griffin, and Lim. 1984. Signal Estimation from Modified Short-Time Fourier Transform.” IEEE Transactions on Acoustics, Speech, and Signal Processing.
Hohmann. 2002. “Frequency Analysis and Synthesis Using a Gammatone Filterbank.” Acta Acustica United with Acustica.
Ignjatovic, Aleksandar. 2007. Local Approximations Based on Orthogonal Differential Operators.” Journal of Fourier Analysis and Applications.
Ignjatovic, A. 2009. Chromatic Derivatives and Local Approximations.” IEEE Transactions on Signal Processing.
Irizarry. 2001. Local Harmonic Estimation in Musical Sound Signals.” Journal of the American Statistical Association.
Janssen. 1984. Gabor Representation and Wigner Distribution of Signals.” In.
Kim, Lee, Kim, et al. 2010. Complexity Reduction of WSOLA-Based Time-Scale Modification Using Signal Period Estimation.” In Communication and Networking.
Krapf, Marinari, Metzler, et al. 2018. Power Spectral Density of a Single Brownian Trajectory: What One Can and Cannot Learn from It.” New Journal of Physics.
Krishnan. 2005. A New Approach for Estimation of Instantaneous Mean Frequency of a Time-Varying Signal.” EURASIP J. Appl. Signal Process.
Kronland-Martinet, Guillemain, and Ystad. 1997. Modelling of Natural Sounds by Time–Frequency and Wavelet Representations.” Organised Sound.
Lewicki. 2002. Efficient Coding of Natural Sounds.” Nature Neuroscience.
Mallat, S., and Zhang. 1992. Adaptive Time-Frequency Decomposition with Matching Pursuits.” In Time-Frequency and Time-Scale Analysis, 1992., Proceedings of the IEEE-SP International Symposium.
Mallat, Stéphane G., and Zhang. 1993. Matching Pursuits with Time-Frequency Dictionaries.” IEEE Transactions on Signal Processing.
Masri, Bateman, and Canagarajah. 1997a. A Review of Time–Frequency Representations, with Application to Sound/Music Analysis–Resynthesis.” Organised Sound.
———. 1997b. The Importance of the Time–Frequency Representation for Sound/Music Analysis–Resynthesis.” Organised Sound.
Mecklenbräuker, and Hlawatsch, eds. 1997. The Wigner Distribution: Theory and Applications in Signal Processing.
Moussallam, Daudet, and Richard. 2012. Matching Pursuits with Random Sequential Subdictionaries.” Signal Processing.
Müller, Ellis, Klapuri, et al. 2011. Signal Processing for Music Analysis.” IEEE Journal of Selected Topics in Signal Processing.
Necciari, Balazs, Holighaus, et al. 2013. The ERBlet Transform: An Auditory-Based Time-Frequency Representation with Perfect Reconstruction.” In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).
Noll. 1967. Cepstrum Pitch Determination.” The Journal of the Acoustical Society of America.
Preis, and Georgopoulos. 1999. Wigner Distribution Representation and Analysis of Audio Signals: An Illustrated Tutorial Review.” Journal of the Audio Engineering Society.
Qian, and Chen. 1994. Signal Representation Using Adaptive Normalized Gaussian Functions.” Signal Processing.
Rafii. 2018. Sliding Discrete Fourier Transform with Kernel Windowing [Lecture Notes].” IEEE Signal Processing Magazine.
Rioul, and Vetterli. 1991. Wavelets and Signal Processing.” IEEE Signal Processing Magazine.
Rosen, Wood, and Stoffer. 2012. AdaptSPEC: Adaptive Spectral Estimation for Nonstationary Time Series.” Journal of the American Statistical Association.
Saatçi, Turner, and Rasmussen. 2010. Gaussian Process Change Point Models.” In Proceedings of the 27th International Conference on International Conference on Machine Learning. ICML’10.
Scarpazza. 2003. A Brief Introduction to the Wigner Distribution.”
Schroeder, and Atal. 1962. Generalized Short‐Time Power Spectra and Autocorrelation Functions.” The Journal of the Acoustical Society of America.
Sejdic, Djurovic, and Jianga. 2009. Time–Frequency Feature Representation Using Energy Concentration: An Overview of Recent Advances.” Digital Signal Processing.
Shafi, Ahmad, Shah, et al. 2009. Techniques to Obtain Good Resolution and Concentrated Time-Frequency Distributions: A Review.” EURASIP Journal on Advances in Signal Processing.
Stankovic, Ljubisa, Daković, and Thayaparan. 2014. Time-Frequency Signal Analysis with Applications.
Stankovic, L. J., and Stankovic. 1995. An Analysis of Instantaneous Frequency Representation Using Time-Frequency Distributions-Generalized Wigner Distribution.” IEEE Transactions on Signal Processing.
Szmajda, Gorecki, and Mroczka. 2010. Gabor Transform, Gabor-Wigner Transform and SPWVD as a Time-Frequency Analysis of Power Quality.” In Proceedings of 14th International Conference on Harmonics and Quality of Power.
Szmajda, and Mroczka. 2011. Comparison of Gabor-Wigner Transform and SPWVD as Tools of Harmonic Computation.” Renewable Energy and Power Quality Journal.
Torrence, and Compo. 1998. A Practical Guide to Wavelet Analysis.” Bulletin of the American Meteorological Society.
van Delft, and Eichler. 2015. Data-Adaptive Estimation of Time-Varying Spectral Densities.” arXiv:1512.00825 [Stat].
Welch. 1967. The Use of Fast Fourier Transform for the Estimation of Power Spectra: A Method Based on Time Averaging over Short, Modified Periodograms.” IEEE Transactions on Audio and Electroacoustics.
Yu, and Slotine. 2009. Audio Classification from Time-Frequency Texture.” In Acoustics, Speech, and Signal Processing, IEEE International Conference on.
Zhao, Atlas, and Marks. 1990. The Use of Cone-Shaped Kernels for Generalized Time-Frequency Representations of Nonstationary Signals.” IEEE Transactions on Acoustics, Speech, and Signal Processing.