The cepstrum of a time series takes the represents the power-spectrogram using a log link function. I haven’t actually read the foundational literature (e.g. Bogert, Healy, and Tukey 1963), merely used some algorithms; but it seems to be mostly a hack for rapid identification of correlation lags where said lags are long.
For a generalized modern version, see Proietti and Luati (2019).
In this chapter we consider a class of parametric spectrum estimators based on a generalized linear model for exponential random variables with power link. The power transformation of the spectrum of a stationary process can be expanded in a Fourier series, with the coefficients representing generalised autocovariances. Direct Whittle estimation of the coefficients is generally unfeasible, as they are subject to constraints (the autocovariances need to be a positive semidefinite sequence). The problem can be overcome by using an ARMA representation for the power transformation of the spectrum. Estimation is carried out by maximising the Whittle likelihood, whereas the selection of a spectral model, as a function of the power transformation parameter and the ARMA orders, can be carried out by information criteria. The proposed methods are applied to the estimation of the inverse autocorrelation function and the related problem of selecting the optimal interpolator, and for the identification of spectral peaks.
This is no longer a popular technique. probably because we don’t need to shoe-horn everything into FFTs these days, but modern researchers can still encounter it in the mysteriously-still popular MFCC analysis used in machine listening.