How Statistics Rewards Us for Being Mean to Each Other
It's an unfortunate staple of human nature: all too often, we're mean to others when they screw up or perform poorly. Flunk a test? Then a parent might ground you. Lose a match? Then your coach might yell at you. Botch a presentation? Then your boss might demote you. Even more unfortunate, statistics reinforces these acts of nastiness, even though psychological research suggests they don't actually help.
Nobel Prize-winning psychologist Daniel Kahneman explained this twisted phenomenon in his bestselling book Thinking, Fast and Slow. Earlier in his career, he was teaching flight instructors in the Israeli Air Force that rewarding good performance works better than punishing mistakes, as evinced by scientific research. One of his "students" ardently disagreed, however, citing his own experience that after praising cadets for a well-executed flight maneuver, they usually perform worse on the next try. On the other hand, a harsh rebuke after a failed attempt almost always leads to a better subsequent outcome.
Kahneman wasn't deterred by the flight instructor's anecdote. Instead, he was enthused.
"The instructor was right–but he was also completely wrong!" he recollected. "The inference he had drawn about the efficacy of reward and punishment was completely off the mark."
As Kahneman explained, the instructor was fooled by a statistical quirk called "regression to the mean," the tendency for a variable to return to average. His praise wasn't causing his students' performance to sink, just as his admonishments weren't leading to improvements. Rather, the students were simply executing their tasks above average in some instances and below average in others – as predicted by statistical chance – then subsequently returning to average.
"The instructor had attached a causal interpretation to the inevitable fluctuations of a random process," Kahneman concluded.
As humans, we try to fit narratives to everything and seek causes when sometimes there really are none. Sadly, regression to the mean fools us into believing that being mean boosts others' performance while being nice does not.