I tend to think the Dunning-Krueger Effect is the most misunderestimated finding in all sciences.
The basic argument is simple: people lacking expertise don’t know how much they don’t know, and consequently, they are far more confident about their status than they otherwise might be. Thus, we–for all of us are ignorant at least some of the time–become dangerously confident that things that, in fact, are ain’t. (I have just been informed that this quote, often attributed to Mark Twain, actually comes from Josh Billings. Things you learn!)
On the one hand, this is not an especially surprising or disturbing phenomenon: we are rarely required to know much about evolutionary biology, quantum mechanics, or financial regulations. As a British pilot who escaped from a German POW camp wrote of his strategy, humans are ultimately creatures of habit who accept everyday things as normal and defer to the usual marks of “authority” when necessary. We will know just enough of such esoteric topics to do well on exams, if we ever take such coursework, and forget them soon afterwards. (I may be the only person that I know who still wakes up in the middle of night with realization that why Prof. X said what he did in quantum electrodynamics or microecon theory so many years ago…) Getting the right answer in light of the “truth” is not pertinent to our lives: for most of us, the only judges are our peers, bosses, and certificates. This may be brutal, but true.
There was an article in a Skeptics Society publication from a couple of decades ago that gripped my attention: some guy decided to survey the Nepalese as to whether they believed in the Abominable Snowman. (I cannot track down the citation or a link…I’ll append if I ever rediscover the article) To his surprise, he found that vast majority of Nepalese believed in its existence–not shocking, since it is part of their national folklore. The only exceptions were Nepalese wildlife experts–scientists, hunters, game wardens, etc. Even in Nepal, most people are not familiar with their own wildlife. So they believe their fantasies independent of data–or the lack thereof. Those who do have actual data, based on their experience, and have real need to make use of it–because they actually live and work in the wildlife–base their assessment on something more tangible.
This was the end of the story in my previous incarnation, so to speak, but these days, I’ve grown a bit more curious: do Nepalese wildlife experts actually “know” that there is no Yeti? The answer cannot be NO! In fact, this is the single most important point to recognize about epistemology and statistics. We never “know” the truth. We can only infer the probabilities based on what data we have and the knowledge that everything we know is, in some sense, “wrong.” The real power of statistics is not that we “know” the truth and reliably predict the “right answers,” but we now how wrong what we think to be the “right answers” are based on the data. In other words, we know the “variance” of our summary statistics.
The idea of variance, as well as the actual calculation thereof, is a beautiful thing. If we have a sample of N measurements and the mean is X, then X is the best guess we have, in absence of any better knowledge, of the answer. But is X the “right” answer? Of course not. It’s only statistics, and every statistic is “wrong.” The advantage in knowing the principles behind the statistics, in addition to the statistics themselves (pun very much intended) is that we also know how wrong these statistics are in light of the real life data. Indeed, the formula for variance is literally how wrong you are on average (squared) if you guess that the answer is always the mean (or the average)–i.e. a literal measure of the average wrongness of your best answer. And being able to assess, quantitatively or not, how wrong even your best answers is what separates “science” from “book l’arning.”
Going back to the tale of abominable snowman, what this means is that even the Nepalese wildlife experts who don’t believe there is such thing, cannot actually “know” that such creatures exist. They are justified in their skepticism since it is extremely improbable that such creatures should have escaped their awareness for so many years. Yet, their sense of being is not wrapped up in the non-existence of Yeti: if such creatures are indeed found, they may be surprised, but perhaps pleasantly so, aweigh with curiosity as to how they managed to avoid being found. (NB: Scholastic theology, ultimately grounded on logic, in fact, brought recognition of this fundamental uncertainty to the realm of Roman Catholic theology. A major point of theology is that one cannot ever “know” that he or she is in state of grace, even though one may hope and pray that he or she is or will be and be confident in that faith–but not “knowledge.” This subtle point underlay one of the questions brought against Joan of Arc at her trial, whether she thought she was in state of grace. Her answer, indicating her absolute conviction in her faith, but also total lack of “knowledge” (“If I am not, may God place me there; if I am, may God so keep me.”) was the only possible acceptable answer and that such an answer was given by a peasant girl with no theological education was among the arguments for her eventual canonization.)
“O felix culpa quae talem et tantum meruit habere redemptorem” is the Catholic exsultet from the Easter vigil mass–“oh, happy sin that earned for us the Redeemer.” This concept has an almost exact analogue in sciences: happy is the variance that shows our theories are wrong, for if our theories explain all of universe, there is nothing to learn. Much the way sin and its recognition lies at the root of Christianity, variance and its recognition–and measurement–lies at the root of science. When we despise the variance and treat it as a nuisance to be eliminated, we are no longer doing “science” but falling to hubris that refuses to learn. But no less than is Christianity an exercise in avoiding sin, even if total sinlessness is beyond mere human capability, so is the role of science to reduce the variance by learning.