Performative prediction

Placeholder. An ML formalization of hyperstition, probably implicit in recommender dynamics, adversarial classification and some types of external validity, among other places. HT TJ for the pointer.

In classical supervised learning we fit parameters \(\theta\) to minimize expected loss over a distribution \(\mathcal{D}\) that sits passively and lets itself be measured, \(\theta_{\mathrm{SL}} = \arg\min_\theta \mathbb{E}_{Z\sim\mathcal{D}}\,\ell(Z;\theta)\), with \(Z=(X,Y)\) a feature–outcome pair and \(\ell\) the loss. Specifically, while we assume the model watches the world, we assume that the world does not watch back. Performative prediction (Perdomo et al. 2020) is a formalization of a different regime wherein deploying the model changes the subsequent data distribution.

The stock example (familiar from fairness) is assessing people for credit-worthiness. If we predict someone to be at high risk of credit default, we might protectively assign them a punishing interest rate, which in turn increases their propensity to default, thereby confirming our prediction.

We address this phenomenon formally by assuming that each choice of parameters produces a perturbed data distribution \(\mathcal{D}(\theta)\) — the data we would see if we deployed \(\theta\) to do things in the world. In the case that the perturbation ‘looks like a self-fulfilling prophecy’, we call that a hyperstition.

We define the performative risk, \[\mathrm{PR}(\theta) = \mathbb{E}_{Z\sim\mathcal{D}(\theta)}\,\ell(Z;\theta).\] Here \(\theta\) appears twice: once as the model being graded, once inside the distribution it produces. That second appearance clearly clashes with ordinary regression — we cannot simply descend the gradient of the loss in isolation, because moving \(\theta\) also moves the target.

There are two extensions to the definition which recover something like that classical supervised, static setting.

A performative optimum \(\theta_{\mathrm{PO}} = \arg\min_\theta \mathrm{PR}(\theta)\) minimizes the performative risk with both copies of \(\theta\) moving together, via the implicit dependence of the distribution on the parameters. We derive this by applying the chain rule to the performative risk: \[\nabla_\theta \mathrm{PR}(\theta) = \underbrace{\mathbb{E}_{Z\sim\mathcal{D}(\theta)}\big[\nabla_\theta \ell(Z;\theta)\big]}_{\text{fit term}} \;+\; \underbrace{\nabla_\theta\,\mathbb{E}_{Z\sim\mathcal{D}(\theta)}\big[\ell(Z;\theta')\big]\Big|_{\theta'=\theta}}_{\text{reshaping term}}.\] The fit term is the gradient we would write down if the distribution were static. The reshaping term is the part new to performative prediction: it measures how perturbing \(\theta\) deforms the distribution \(\mathcal{D}(\theta)\) itself, and how much that deformation costs us in expected loss. The optimum is (defined to be?) where the two cancel — where the marginal gain from fitting the data better is exactly offset by the marginal cost it produces.

A performatively stable point \(\theta_{\mathrm{PS}}\) instead satisfies a fixed-point condition, \[\theta_{\mathrm{PS}} = \arg\min_\theta \mathbb{E}_{Z\sim\mathcal{D}(\theta_{\mathrm{PS}})}\,\ell(Z;\theta),\] i.e. we are satisfied with a \(\theta_{\mathrm{PS}}\) if, given an induced world, the model is already optimal for it, so refitting recovers the same parameters.

Apparently, the two do not coincide; the stable model is not in general the best one. Its special feature is that we suspect it can be found by hyperstitious iteration. Practitioners already talk about this as retraining: refit on whatever distribution the last model produced, \[\theta_{t+1} = \arg\min_\theta \mathbb{E}_{Z\sim\mathcal{D}(\theta_t)}\,\ell(Z;\theta).\]

Perdomo and co-authors call this repeated risk minimization and derive a contraction guarantee: if the loss is smooth and strongly convex, and the map \(\mathcal{D}(\cdot)\) is sufficiently Lipschitz in Wasserstein distance, then the iteration converges to a stable point at a linear rate, the error shrinking by a constant factor each step. Put another way, “does the hyperstition converge or explode?” turns upon the question of how sensitive the world is to the model.