Have you heard of the ‘Dunning-Kruger effect’? It’s the (apparent) tendency for unskilled people to overestimate their competence. Discovered in 1999 by psychologists Justin Kruger and David Dunning, the effect has since become famous.
Except there’s a problem.
The Dunning-Kruger effect also emerges from data in which it shouldn’t. For instance, if you carefully craft random data so that it does not contain a Dunning-Kruger effect, you will still find the effect. The reason turns out to be embarrassingly simple: the Dunning-Kruger effect has nothing to do with human psychology.1 It is a statistical artifact — a stunning example of autocorrelation.
EDIT: see response from dustyData and the article they linked to https://www.bps.org.uk/psychologist/dunning-kruger-effect-and-its-discontents
Look at the chart in the article. Let’s say actual ability is x and perceived ability is y. The graph effectively asks us to compare the difference between perceived ability and actual ability, which we can write as y-x. Thus, the chart effectively graphs y-x over x, which can be written as -x+y over x. There is always a strong correlation unless y approximates x.
IMO, the conclusion of that analysis should be “Dunning−Kruger is a truism”, not “Dunning−Kruger doesn’t exist”.