Our Thoughts

Praise, punishment, and regression towards the mean

15th July 2019

Regression towards the mean is a statistical phenomenon that most people have heard of, but few really think through the repercussions. How does it feature in the world of customer insight?

Imagine an organisation runs a customer survey and reports a satisfaction score for each of their 50 branches. The MD praises and rewards the branches who do best, and punishes, or at least demands improvement from, those that did worst. In the next survey, the MD is pleased to see that the poorest-scoring branches have upped their game - clearly his/her no-nonsense approach has yielded results. Unfortunately the top performers seem to have been resting on their laurels, and most of them have seen drops in satisfaction. Who could blame our MD for concluding that the stick is more effective than the carrot?

In reality what has happened here is likely to be the result of regression towards the mean. In simple terms this means that when we look at any data which is subject to statistical variability, everybody's scores are pushed a little bit away from the "true" scores we really want to know. On average that difference is random, but at the level of the individual it will vary from pushing the score up a bit to pushing it down a bit, and when we rank people on their scores that means that the people whose true scores have been pushed up at random are more likely to be towards the top, and the people whose true scores are pushed down are more likely to be towards the bottom.

To understand "regression towards the mean" you need to know that people who are a long way above average at one point in time are likely to get a score that's closer to the average in the next survey, and so are people who are a long way below average. It doesn't mean that there's no difference in the real scores, just that a bit of luck can make a good score seem even better. Look at something like the number of goals a football player scores in a season. Mo Salah scored 32 goals in the 2017/2018 season, a record for the current length of 38 games. He had to be very good to set that record, but we'd expect his performance to be a little less extreme in the next season (as it was - he scored 22 in 2018/19).

All of this is extremely hard to think about clearly, because it goes against our natural instincts. To go back to our MD, Daniel Kahneman points out that this statistical oddity tends to reinforce our belief that punishing failure is more effective than praising success:

"...because we tend to reward others when they do well and punish them when they do badly, and because there is regression to the mean, it is part of the human condition that we are statistically punished for rewarding others and rewarded for punishing them."

So what can we do about it? Be aware that exteme performances, good or bad, are likely to be made up of a mixture of real performance and luck. Give more weight to consistent performance over time than to one-off extreme scores (the salesperson who is third every month might just be the best). Understand the cognitive biases that may trick you into seeing punishment as more effective than praise.

01484 517575
Taylor Hill Mill, Huddersfield HD4 6JA
Twitter LinkedIn
...