Stability in linear dynamical systems

Pole representations

In the univariate, discrete-time case, in discrete-time linear systems terms, these are systems that have no poles outside the unit circle, but might have poles on the unit circle. In continuous time it is about systems that have no poles with positive real part. For finitely realizable systems this boils down to tracking trigonometric roots, e.g. Megretski (2003).

In a multivariate context we might consider eigenvalues of the transfer matrix in a similar light.

van Handel (2017) for example mention the standard result that the eigenvalues of a symmetric matrix \(X\) are the roots of the characteristic polynomial \(\chi(t)=\operatorname{det}(t I-X)\) and, equivalently, the poles of the Stieltjes transform \(s(t):=\operatorname{Tr}\left[(t I-X)^{-1}\right]=\frac{d}{d t} \log \chi(t)\)

Reparameterisation

We can use cunning reparameterisation to keep systems stable. This Betancourt podcast on Sarah Heaps’ paper (Heaps 2020) on parameterising stationarity in vector auto regressions is deep and IMO points the way to some other neat tricks in neural nets. She constructs interesting priors for this case, using some reparametrisations by Ansley and Kohn (1986).

Maybe related: Roy, Mcelroy, and Linton (2019)

Continuous time

TBC.

Stability and gradient descent

What if we are incrementally learning a system and wish the gradient descent steps not to push it away from stability? In such a case, we can possibly side-step the problem by using a topology which maximises system stability (Laroche 2007).

References