Curved exponential families

April 19, 2016 — July 8, 2024

geometry
probability
statistics
Figure 1

Curved exponential families generalise exponential families while preserving some of the magic, I guess?

Credited to (Efron 2004) and used in Efron (1978) to show some nice behaviour for degrees of freedom-based penalties. Will read soon.

Anirban DasGupta:

There are some important examples in which the density (pmf) has the basic Exponential family form \(f(x \mid \theta)=e^{\sum_{i=1}^k \eta_i(\theta) T_i(X)-\psi(\theta)} h(x)\), but the assumption that the dimensions of \(\Theta\), and that of the range space of \(\left(\eta_1(\theta), \cdots, \eta_k(\theta)\right)\) are the same is violated. More precisely, the dimension of \(\Theta\) is some positive integer \(q\) strictly less than \(k\). Let us start with an example.[…]

Suppose \(X \sim N\left(\mu, \mu^2\right), \mu \neq 0\). Writing \(\mu=\theta\), the density of \(X\) is \[ \begin{gathered} f(x \mid \theta)=\frac{1}{\sqrt{2 \pi}|\theta|} e^{-\frac{1}{2 \theta^2}(x-\theta)^2} I_{x \in \mathcal{R}} \\ =\frac{1}{\sqrt{2 \pi}} e^{-\frac{x^2}{2 \theta^2}+\frac{x}{\theta}-\frac{1}{2}-\log |\theta|} I_{x \in \mathcal{R}} \end{gathered} \]

Writing \(\eta_1(\theta)=-\frac{1}{2 \theta^2}, \eta_2(\theta)=\frac{1}{\theta}, T_1(x)=x^2, T_2(x)=x, \psi(\theta)=\frac{1}{2}+\log |\theta|\), and \(h(x)=\frac{1}{\sqrt{2 \pi}} I_{x \in \mathcal{R}}\), this is in the form \(f(x \mid \theta)=e^{\sum_{i=1}^k \eta_i(\theta) T_i(x)-\psi(\theta)} h(x)\), with \(k=2\), although \(\theta \in \mathcal{R}\), which is only one dimensional. The two functions \(\eta_1(\theta)=-\frac{1}{2 \theta^2}\) and \(\eta_2(\theta)=\frac{1}{\theta}\) are related to each other by the identity \(\eta_1=-\frac{\eta_2^2}{2}\), so that a plot of \(\left(\eta_1, \eta_2\right)\) in the plane would be a curve, not a straight line. Distributions of this kind go by the name of curved Exponential family. The dimension of the natural sufficient statistic is more than the dimension of \(\Theta\) for such distributions.

Examples?

1 References

Efron. 1975. Defining the Curvature of a Statistical Problem (with Applications to Second Order Efficiency).” The Annals of Statistics.
———. 1978. The Geometry of Exponential Families.” The Annals of Statistics.
———. 2004. The Estimation of Prediction Error.” Journal of the American Statistical Association.
Keener. 2009. Curved Exponential Families.” In Theoretical Statistics. Springer Texts in Statistics.