The observer is never the observed.

There is a class of measurement that alters the thing it measures. Physicists call it the observer effect. Sociologists call it reflexivity. In product analytics, it is called Tuesday.

The dashboard as a mirror

The moment you put a metric on a dashboard, you have told everyone in the room that this number matters. It does not matter whether it actually matters. What matters is that it is visible, named, and reported weekly. The number will go up. The question is whether the thing it represents also goes up.

Sometimes they move together. More often, they drift. A support team learns that the metric is tickets closed, so they close tickets. The customers are no more helped, but the number is better. This is not fraud — it is rational behavior in response to a signal.

Goodhart’s law, plainly stated

When a measure becomes a target, it ceases to be a good measure. Goodhart said this in the context of monetary policy; it generalised immediately. The mechanism is always the same: a proxy is elevated to the status of the thing itself, and then people optimise for the proxy.

The interesting question is not how to avoid this — you cannot, entirely — but how to slow it down. My experience is that three factors matter: lag, cost, and opacity.

Lag is how long it takes to game the metric. Metrics with short feedback loops get gamed faster. Cost is how expensive it is to move the metric without moving the underlying. Opacity is how visible the gaming is to observers outside the system.

Inference under load

The most interesting cases I have encountered are metrics used for inference — that is, for making decisions about the world rather than just reporting on it. A model that predicts churn. A classifier that routes support tickets. A score that determines loan approval.

These metrics do not merely reflect the world. They constitute part of the world they measure. A churn model that is used to target retention spend changes who churns. A classifier that is used to route work changes what kind of work gets done. The loop is not a bug; it is the system.

What to do

Three things that have actually helped:

Rotate the metrics. Not randomly, but deliberately, on a schedule. The act of changing what is measured is itself a signal that the map is not the territory.

Keep a dark metric. One number that is never shown in a dashboard, never set as a target, never mentioned in a review. It is your canary. If the dark metric starts to diverge from the public ones, you have found the drift.

Name the model. Every metric is a model. It has assumptions, blind spots, a designer. Making those explicit — even informally — is surprisingly effective at slowing the collapse from proxy to target.

None of these solve the problem. They extend the period of usefulness before the next recalibration. That is usually enough.