Performance indicators, measurement, analytics

Data-driven business plans. Measuring how well we are doing, while minimising exposure to Goodhardt’s law and other hyperselection effects. See also guesstimation.

• Benn Stancil, A method for measuring analytical work:

  Analysts should judge their work by how quickly people make decisions with it.

• Fun Economic Facts is Guzey’s roundup of broken metrics used in economic planning

• Measurement and power in organisations (which mentions the Hubbard book on measurement).

• Commoncog’s posts

  • Reduce Noise, Not Cognitive Biases
  • How To Reduce Decision Noise on Kahnemann’s protocol for improving decision making

• Reminder: Goodhart’s Law: when a measure becomes a target, it cease to be a good measure

• C. Thi Nguyen on Games, Art, Values, and Agency

  […] I used to be at the University of Chicago, which obviously has always been academically super-duper strong, but back in the day, it wasn’t the place you applied if you were interested in Harvard or Stanford or Princeton, it was less well known. So suddenly, there was a strategy that they undertook at the University of Chicago because they were being hurt in the US News rankings, and they were being hurt because the only people who applied to the University of Chicago were the ones who really wanted to go there. And you are rewarded in the US News rankings by having a high selectivity, by rejecting most of the people who apply, so they intentionally encouraged people to apply knowing they would reject them, ’cause it increased their selectivity, and they leapt up in the rankings.

• Analytics: The complete minicourse - by Cassie Kozyrkov

The Sobering Truth About the Impact of Your Business Ideas:

But, science-based organizations are rigorously quantifying this impact and have learned some sobering lessons:

1. The vast majority of business ideas fail to generate a positive impact.
2. Most companies are unaware of this.
3. It is unlikely that companies will increase the success rate for their business ideas.

The definition used by Cassie Kozyrkov is slightly different again, but this still lands: To Recognize Risks Earlier, Invest in Analytics. (She thinks of analytics as more like what I call exploratory data analysis.)