Myth of the Myth of the Independent Voter

A popular belief in voter psychology is that there is no such thing as a true independent voter:  everyone has some partisan proclivities and that they will act according to this proclivity.  This is bullshit.  Every (usual) voter has some partisan proclivity and they will vote according to this proclivity, but not everyone votes in most elections.  By definition, these non-voters see very little difference between the two (or more) sides in the election.  While Mitt Romney and Barack Obama, say, would look nothing alike to the “usual” voters, to the “not-so-usual” voters, they are basically same people, who look alike, dress alike, and talk alike.  The only problem, electorally speaking, is that the “not-so-usual” voters don’t vote in large numbers, so their impact on most elections is limited.

Why these voters might go missing–and why they might show up sometimes–does not have an obvious explanation:  they are actuated by impulses different from the usual political dividing lines.   The so-called “missing white voters” phenomenon attempted to get at this phenomenon with, naturally, mistaken implications.  Because many of these voters voted for Ronald Reagan and were actively courted by Richard Nixon, Republican pollsters assumed that they were naturally “Republican” voters who would tip the balance only if they showed up.  They conveniently forgot that they were called “Reagan Democrats” for a reason, and that many of them voted reliably for various Democratic Congressional candidates for decades, even while they were voting for Reagan and, to a lesser extent, Bush the elder and even Ross Perot.  For these voters, the elimination of many old time Democratic House candidates and the subsequent homogenization of the Democratic Party unintentionally (presumably) brought by 1994 elections constituted a major turning point:  they were not Republicans, but they were not exactly fond of the transforming Democratic Party either.  So the first patterns of these missing white voters show up as early as 1996, depending on how hard you look–and are willing to extrapolate from feint signs, not something one is advised to do statistically–although the pattern intensifies until it becomes clearly visible only in 2010-14 elections (more detailed numbers, with statistical analyses, to be shown later, once I’m done running the numbers…)

Put differently, “the missing white voters,” remain a significant electoral force for altogether different reasons than what fascinates the Republicans:  these are the voters who could swing the control of House to the Democrats and force the Republicans to moderate, by making the elections more competitive in presently “safe” Republican districts, if the Democrats can somehow make an appeal to them.

There are signs that at least some in the Democratic Party might be getting this, as this  post on Washington Monthly indicates.  #2 and #5 of the reasons they list for why Sanders might be the better candidate for the Democrats flatly contradict each other.  If #2 is true, there are no “rebel” voters who might ditch political orthodoxy.  The reason such rebel voters exist in the first place is because, as far as they are concerned, Obama and Romney are the same people and that they’d show up only if they get something that is sufficiently different from the “usual suspects” and if they are sufficiently unsatisfied with the political status quo.  To them, Hillary Clinton is just another Republicrat who all look alike.  To the usual voters, to whom Democrats and Republicans look completely different, this possibility is downright unthinkable and as such, both Sanders and Trump, along with their supporters, make no sense.

To the degree that many voters vote the same party downstream as the presidential candidates they choose, both Sanders and Trump pose interesting prospects for their respective parties.  The “missing white voters” have been the bane of the Republicans in presidential elections:  even when they do well in House elections, they do so because so few voters turn up in their “safe” districts.  The turnout in safe Republican districts in 2012, for example, was lower by as much as 20-25% compared to similarly safe Democratic districts by my estimate (numbers to be supplied when I’m done with the general analysis).  Republican House candidates thus win with relatively few votes while the Democratic presidential coattail brings out additional voters mostly where they are not needed.  So Republicans capture a large House majority with an oddly small voteshare but are uncompetitive for presidential elections.  Much of this has to be due to the “missing white voter” phenomenon:  neither party’s presidential candidates just don’t appeal to them enough.  If the Republicans get a candidate who can bring them out, they will be competitive for the White House.  If the Democrats get a candidate who can do the same, the Democrats gain a reasonable shot at capturing the House.  Sanders and Trump offer this prospect for each party.


Misuse of Statistics Begetting More Misuse

I am ambivalent, if you hadn’t guessed already, about the popular use of statistics.   Dan Kahneman, among others, showed that people are inherently bad at statistical thinking.  My contention on this point has been that it is not simply that people don’t understand the basic statistical concepts, but that the subtle implications run counterintuitive to how people normally think.  The powers of statistics is that it summarizes a complex data into a neat, simple package, PLUS it provides a summary statistic of how wrong that package is, conditional on the sample.  Everyone loves the simple package, not the fine prints that comes with it.

This article on Fivethirtyeight strikes me as emblematic of exactly this sort of seemingly profound misuse of statistics.  I don’t doubt that, in general, pitchouts, sac bunts, and other self-sacrificial baseball strategery has been fairly ineffective.  But I also find it hard to believe that they are universally ineffective, period.  The more likely scenario is that the use of such strategery has generally included a small proportion of useful applications mixed in with many ineffective ones, with the situations difficult to disentangle through simple statistics.  Thus, the average sac bunt in the sample of the past was indeed ineffective, but with a small but meaningful if only it could be identified subset that was effective.

This is not an argument for a blanket condemnation of sacrifice bunts, then, but a call for better, more nuanced research.  What are the effective use of sacrifice bunts?  How do you sacrifice bunt more in situations when they are actually valuable and avoid using them when they are counterproductive?  One might suspect, in fact, that, if the baseball people are not as stupid as naive sabermetricians assume them to be, and are actually combining their traditionally minded skills with sabermetric insights productively, ineffective sacrifice bunts have been eliminated from the sample much faster than the effective ones.  So has the value of sacrifice bunts changing over time, then?  One might think it should be so, and we’d be able to tell if we have developed a useful methodology for distinguishing effective sacrifice bunts from ineffective ones, rather than trying to place a value on the average sacrifice bunt the way so much misuse of statistics winds up becoming.

More on Politics

Will continue with the review of the Hedges book soon…but in the meantime, an interesting note on current politics.

The Washington Examiner, as far as as I know, is a political rag with an ideological agenda, but this article makes an interesting observation about what makes for a successful campaign strategy that is often forgotten.

We expect, naively, that voters vote for whoever that most closely approximates their own ideals and preferences.  In this sense, people assume that the social conservative Evangelicals can only be captured by an “obviously” religious person.  Many people whose religiosity was questionable suddenly discover God when they try to run for a major office as a Republican…and fail miserably.  I am actually quite surprised me that even a shrewd marketer like Trump fell to this trap momentarily, if the article can be trusted.

Of course, this is not how political marketing works.  If any politician (or a marketer of any kind) expects to meet the wants and needs of any voter/consumer segment exactly, he is deluded.  Not only will he fail to match the exact needs and wants of any more than a handful of persons, the attempt to match anyone too closely may easily lose the rest of the market.  The politician has to choose:  either small group (but relatively large, compared to others) of people who happen to be very fussy about their demands and match them very closely, while charging them a premium, or just deliver a solid product that meets general needs of many people but not particularly closely to any one of them and charge relatively little.  The former, of course, is the Apple strategy while the latter is the PC strategy, but the same logic applies to any number of different settings.

While neither is inherently superior, the fact is that not everyone has the wherewithal to deliver one or the other equally.  Why should any customer pay a premium for a good or service?  In the end, it comes to “credibility,” specific to that particular segment, both the buyer and the seller, something that induces them to hand over the extra dollar for what the seller offers.  What makes for credibility?  It varies.  One presumes, however, that credibility cannot be turned out on the fly like hotcakes.  For all we know, Trump the private person could be a deeply religious man, but Trump the public persona certainly isn’t, and it is the latter that’s running for presidency. Just showing the family Bible and such will not win this credibility.  Trump the public persona, however, is a business man whose leadership and financial prowess, rightly or wrongly, enjoys credibility with wide swaths of Republican voters, religious and otherwise.  It makes no sense for Trump to waste time trying to convince people of his religiosity that nobody will take too seriously, whether it is sincere or not.  His public persona, however, is not a bad asset given the market and he should double down on it, and he has.

Will this work?  Who knows.  Trump’s bet is that his business prowess meets enough of the “general needs” among the Republican voters that they are willing to overlook his areligiosity, and indeed, profanity.  We do not know the distribution of the elasticity of substitution between religiosity and business savvy among the Republican voters.  Maybe it is sufficiently elastic for enough of them that Trump’s gamble will pay off, or not.  Still, it is refreshing to see someone who understands the basics of marketing running a campaign.

Science and “Variance”

I tend to think the Dunning-Krueger Effect is the most misunderestimated finding in all sciences.

The basic argument is simple:  people lacking expertise don’t know how much they don’t know, and consequently, they are far more confident about their status than they otherwise might be.  Thus, we–for all of us are ignorant at least some of the time–become dangerously confident that  things that, in fact, are ain’t.  (I have just been informed that this quote, often attributed to Mark Twain, actually comes from Josh Billings.  Things you learn!)

On the one hand, this is not an especially surprising or disturbing phenomenon:  we are rarely required to know much about evolutionary biology, quantum mechanics, or financial regulations.  As a British pilot who escaped from a German POW camp wrote of his strategy, humans are ultimately creatures of habit who accept everyday things as normal and defer to the usual marks of “authority” when necessary.  We will know just enough of such esoteric topics to do well on exams, if we ever take such coursework, and forget them soon afterwards.  (I may be the only person that I know who still wakes up in the middle of night with realization that why Prof. X said what he did in quantum electrodynamics or microecon theory so many years ago…)  Getting the right answer in light of the “truth” is not pertinent to our lives:  for most of us, the only judges are our peers, bosses, and certificates.  This may be brutal, but true.

There was an article in a Skeptics Society publication from a couple of decades ago that gripped my attention:  some guy decided to survey the Nepalese as to whether they believed in the Abominable Snowman.  (I cannot track down the citation or a link…I’ll append if I ever rediscover the article)  To his surprise, he found that vast majority of Nepalese believed in its existence–not shocking, since it is part of their national folklore.  The only exceptions were Nepalese wildlife experts–scientists, hunters, game wardens, etc.  Even in Nepal, most people are not familiar with their own wildlife.  So they believe their fantasies independent of data–or the lack thereof.  Those who do have actual data, based on their experience, and have real need to make use of it–because they actually live and work in the wildlife–base their assessment on something more tangible.

This was the end of the story in my previous incarnation, so to speak, but these days, I’ve grown a bit more curious:  do Nepalese wildlife experts actually “know” that there is no Yeti?  The answer cannot be NO!  In fact, this is the single most important point to recognize about epistemology and statistics.  We never “know” the truth.  We can only infer the probabilities based on what data we have and the knowledge that everything we know is, in some sense, “wrong.”  The real power of statistics is not that we “know” the truth and reliably predict the “right answers,” but we now how wrong what we think to be the “right answers” are based on the data.  In other words, we know the “variance” of our summary statistics.

The idea of variance, as well as the actual calculation thereof, is a beautiful thing.  If we have a sample of N measurements and the mean is X, then X is the best guess we have, in absence of any better knowledge, of the answer.  But is X the “right” answer?  Of course not.  It’s only statistics, and every statistic is “wrong.”  The advantage in knowing the principles behind the statistics, in addition to the statistics themselves (pun very much intended) is that we also know how wrong these statistics are in light of the real life data.  Indeed, the formula for variance is literally how wrong you are on average (squared) if you guess that the answer is always the mean (or the average)–i.e. a literal measure of the average wrongness of your best answer.  And being able to assess, quantitatively or not, how wrong even your best answers is what separates “science” from “book l’arning.”

Going back to the tale of abominable snowman, what this means is that even the Nepalese wildlife experts who don’t believe there is such thing, cannot actually “know” that such creatures exist.  They are justified in their skepticism since it is extremely improbable that such creatures should have escaped their awareness for so many years.  Yet, their sense of being is not wrapped up in the non-existence of Yeti:  if such creatures are indeed found, they may be surprised, but perhaps pleasantly so, aweigh with curiosity as to how they managed to avoid being found.  (NB:  Scholastic theology, ultimately grounded on logic, in fact, brought recognition of this fundamental uncertainty to the realm of Roman Catholic theology.  A major point of theology is that one cannot ever “know” that he or she is in state of grace, even though one may hope and pray that he or she is or will be and be confident in that faith–but not “knowledge.”  This subtle point underlay one of the questions brought against Joan of Arc at her trial, whether she thought she was in state of grace.   Her answer, indicating her absolute conviction in her faith, but also total lack of “knowledge” (“If I am not, may God place me there; if I am, may God so keep me.”) was the only possible acceptable answer and that such an answer was given by a peasant girl with no theological education was among the arguments for her eventual canonization.)

“O felix culpa quae talem et tantum meruit habere redemptorem” is the Catholic exsultet from the Easter vigil mass–“oh, happy sin that earned for us the Redeemer.”  This concept has an almost exact analogue in sciences:  happy is the variance that shows our theories are wrong, for if our theories explain all of universe, there is nothing to learn.  Much the way sin and its recognition lies at the root of Christianity, variance and its recognition–and measurement–lies at the root of science.  When we despise the variance and treat it as a nuisance to be eliminated, we are no longer doing “science” but falling to hubris that refuses to learn.  But no less than is Christianity an exercise in avoiding sin, even if total sinlessness is beyond mere human capability, so is the role of science to reduce the variance by learning.



BS and Politics

Searching for the words “BS Detector” and “Trump” uncovers this blog post.

Interesting set of observations, except for one flaw:  it presupposes, implicitly, that Trump spouts a uniquely large amount of BS and that his supporters are extraordinarily gullible.  I don’t think this is the case.

The world is full of self-claimed BS detection artists trolling others to somehow accidentally “expose” themselves:  in the entertainment universe alone, we have seen Penn and Teller, Bill Maher, and Sascha Baron Cohen, among others go to sometimes tasteless extent to bait their putative enemies to exposing themselves as gullible simpletons and ignoramuses.  We have seen many falling for the the infamous “Dihydrogen Monoxide” hoax.  We have seen academic research showing politically active and more “knowledgeable” voters more readily believing untruths that happen to be merely consistent with their existing beliefs.

All these underscore one thing:  people are more gullible when they hear things that they consider “truthy.”  People are not blank slates:  they believe certain things are more likely to be true than not.  When the readily available data confirm their beliefs, they are more likely to believe what they see to be “true” even when the evidence is inadequate.  When the data is incompatible, they need more to be convinced than if they were truly blank slates.  There is nothing wrong with this:  this is simply true mathematically–it is very easy to demonstrate how this works in terms of probability theory.  The existing beliefs are, after all, accumulated from their life experiences and are, in a sense, “right.”  Humans are designed to learn things faster and more efficiently rather than more accurately and more slowly, and drawing on the existing body of more or less “right” information already present in the brains to supplement the new data to help draw conclusions makes good sense, in this context.

The problem is that priors for various subsets of population, this existing knowledge in their brains, increasingly diverge from one another (I noted this in a previous post.  What “intuitively” makes sense for one group makes absolutely no sense to those without the same priors.  2008 election showed this in form of the political Rohrshach test that was Barak Obama:  for many, especially of the younger, more liberal segments, he was “inspiring” and “transformative,” all on the strength of very scanty information, while for others, especially those are older and more  conservative, he seemed “foreign” and “untrustworthy,” again without much data to corroborate.  2016 is showing another version, with much the same demographic divide, in the person of Donald Trump.

Trump’s appeal, to those who like and trust him (which is quite different from “believing” him), is predicated on his ability to call others’ sanctimonious BS in a shrill and exaggerated fashion.   This makes good sense for those who don’t buy into the conventional politicians’ BS, while constitutes an act of blasphemy for those who do.  For those who do buy into the sanctimonious BS of their own favorite politicians, in fact, Trump’s attacks themselves constitute BS–which they are, in fact. Everyone has sacred cows.  Their droppings are BS to the nonbelievers…but not so the believers.

I do not propose that calling BS on everyone to be a good idea.  In fact, all of politics is an act of gilding calves and calling it golden, of creating idols and BS in form of conventions and rituals through which we do business.  The problem today is not so much that we have idols, just that we happen to be lacking unifying idols.  Each tribe, increasingly mutually exclusive, has its own idols and are insistent on calling others’ (and only others’) idols BS.  We need false gods we can all believe in, in other words.

Trump, the Genius Troller?

I’d been following Scott Adams’ insufferable fawning of Donald Trump for some time with a mixture of curiosity and bemusement.

Mind you that I think Trump’s support is built on very real political foundations, of the very significant segment of population who are genuinely discontented with the conventional politics.  I also believe that Adams is on to something potentially big, but I don’t see Trump’s rhetorical strategy as something particularly masterful or ingenious.  What it does illustrate, though, is how weak his enemies really are and how their weakness can only be obscured by a carefully staged masquerade where everyone agrees to pretend to not notice the obvious elephants in the room.

What makes Trump’s strategy work?  It works because, even if his statements are distorted and twisted, there is a fundamental kernel of truth.  If you try to explain why the statement is untrue, you will have to expose equally damaging truth along the way.  Ted Cruz WAS born in Canada.  Bill Clinton did have sex with “that woman,” and Hillary Clinton did her part to prop up her husband’s political reputation.  The list can go on.  Legally, Cruz’s Canadian birth does not (as far as I know) disqualify him from becoming president any more than John McCain’s Panamian birth (on US military base in the Canal Zone, where his father was stationed).  The answer, however, is a legalistic one that will not be so easily explainable.  In case of the Clintons, even the best explanations are probably a bit dodgy and are unlikely to leave them looking like anything other than power-hungry hypocrits.

This is not new in American politics:  Bob Shrum (I think) allegedly said, “if you explain, you lose.”  Explanations are only slightly worse than the accusations in terms of content, and coming from your own mouth while looking defensive, they will look worse.  But not defending is not necessarily helpful either:  the accusations do draw attention to the very real problems, even if in distorted, misleading, and exaggerated fashion.  Throwing damning half truth, daring the other side to defend themselves and in so doing slide deeper, is the gold standard of mudslinging in American politics, with history going back to the second presidential election, bitterly contested between Adams and Jefferson.  There is, naturally, a nifty web comic that addresses this, playing off of the emperor’s new clothes story.

The best solution to this, if the accused has thick enough skin, is to turn accusation on its head.  There is an interesting variant of this dating from ancient China, reproduced in the historical novel The Romance of Three Kingdoms.  When warlord Liu Bei was being attacked by the vastly more powerful army of his rival Cao Cao, he sent his top advisor, the strategic genius Zhuge Liang, to enlist aid from the so-far neutral state of Wu.  Much Cao propaganda rested on his “invincible million man army,” which was not quite one million in number but “only” 800,000.  The slogan operated in much the same fashion as Trump’s exaggerated stories:  those who knew knew that it is an embelishment, but by exposing the lie, they have to bring up the fact that Cao does indeed have a vast and highly experienced army that conquered several powerful warlords already.  So what does Zhuge Liang do when he meets Sun Quan, the lord of the Wu state?  He offers an even more exaggerated account of how Cao Cao squashed flat other rivals and that how his army is probably even larger and more fearsome than his own propaganda.  His point was simple:  Cao Cao’s army might be vast and powerful, but that is not relevant to the strategic needs of the Wu state and its defensive advantages.  The strategic needs are served by allying with Liu Bei and the defensive advantages are sufficient to wipe out the advantage of numbers and experience that Cao Cao’s army was bringing whatever their numbers might be.  In context of American politics, Grover Cleveland pulled off something along this vein when accused by his opponents of immorality due to fathering a son out of wedlock and marrying a much younger woman who was his mistress:  “Ma, ma, where’s my pa?”  “Gone to the White House, ha, ha, ha.”

If Ted Cruz and Hillary Clinton had such obvious positive qualities that can generate actual support from the electorate, the trivial and annoying accusations by Trump would be irrelevant.  Cruz should show up at an NHL game and sing “O Canada,” say.  But therein lies the problem:  Cruz and Clinton (and all the other candidates) are such lightweights that they can only count on banal and meaningless platitudes for cheap appeal to the voters, which, in turn, are quickly and mercilessly exposed by Trump’s equally cheap shots.  Whatever positive qualities these characters have are worth no more than mere wisecracks by Trump.  This is reminiscent of how Jon Stewart skewered all the stuffy panelists on Crossfire by merciless ridicule which put that miserable excuse of a political chat show out of its misery.  Trump, like Stewart, is a product of the media world and is highly skilled in the fine art of acerbic dark comedy.

I would not suggest that the Democrats (or the Republicans) should try to counter Trump by bringing in another comedian–a good comedian does not make for a good president.  They can, however, start by thinking who, with what qualities, would for make an acceptable president in the eyes of a majority of Americans even if he or she is a Canadian who did have sex with that woman.

Consensus and Rationality

One term that gets thrown about a lot to the point of serious abuse is “rationality.”  Often, accusation of “irrationality” is thrown about to indicate denigration towards “the other,” while many pride themselves in how “rational” they are.  Postmodernists have taken this to the other absurd extreme, by denying that there is anything called rationality and declaring all manner of beliefs as simply constructions that are equally valid, regardless of how absurd they might be.

Thomas Kuhn approached this problem from the perspective of someone who had a foot in both worlds, a humanities scholar who was trained originally as a rigorous scientist.  His conclusion that undergirds The Structure of Scientific Revolutions is that what is “rational” is simply a set of logically consistent beliefs that are also compabible with the observed reality, within the degree of precision given the available technology of the time.  (Not necessarily in so many words, perhaps, but one might remember that, while describing the Support Vector Machines and the motivating logic behind statistical learning in general, Vladimir Vapnik invoked the philosophy of science to explain what is and what is not learnable from the observed data–which, in turn, comes from a long tradition of mathematical philosophy.)

This implies a perfectly “rational” reasons between a disagreement, even in face of a common observation.  The disagreeing parties could easily have fully logically consistent set of beliefs about how the world operates that can account for all the observed reality.  They agree on the facets of reality that they see, but they do not agree on how they arrived at it, and there is no way to tell one explanation is more right than the other, or indeed, if there is a more “right” explanation.  This is not a new observation, of course:  as  a trained physicist, I wonder if Kuhn was channeling Einstein’s famous thought experiment about the equivalence principle.

A simple explanation of the equivalence principle is illustrated at this web page, but the point for us is even more elementary:  there are times when no experiment, however cleverly they might sidestep the laws of physics, it is impossible to distinguish between explanations X and Y because, for all intents are purposes, they ARE exactly equivalent.  The real challenge is to understand both explanations well enough so that they can be conjoined together as a single explanation.  Of course, this is what Einstein did, when he started working towards the General Theory of Relativity.

This is not, unfortunately, how most people react in face of disagreements.  The existence of the disagreement is usually chalked up to the ignorance, irrationality, or duplicity on the part of “the other.”  That the other side might have a perfectly logistically consistent reason for their explanations eludes most people.  Lives of physicists is easier in that, even if they might disagree about the particulars, they do have a common language, mathematics, and the shared assumption that the physical universe obeys mathematical logic, in addition to the absence of personal and emotional stakes about the “rightness” of an explanation in general.  Even so, connecting the dots does not come easily.  For physicists, the logic of gravity and the logic acceleration were fully laid out and uncontroversial, literally centuries before Einstein.  That the two explanations were logically equivalent and indistinguishable, however, did elude physicists a long time nevertheless.

The challenge outside physics, and even sciences, is that disagreement in face of common facts is rarely taken as something to be explained.  If one community believes X and the other Y, it is taken as an indication of how “wrong” the other side is since X (or Y) is so self-evident!  The universe where data is plentiful but access to which is easily customizable thanks to the Google search engine magic does not help matters.  Achieving a consensus even on the basic facts is increasingly difficult, and with such little common data, the universe of potentially competing explanations, each fully consistent within itself, grows exponentially.  If a set of competing explanations have  to account for, say, both the sun rising in the east and setting in the west, all explanations that imply the sun setting in the north can be discarded.  If the only shared “data” consists of the sun rising in the east, there is no reason that a sun setting in the north (or the south) should become necessarily “irrational” within confines of its logic (and shared observations of the reality).

“Universal” rationality, then, is not significant for being “rational” in itself, but for being “universal.”  It is significant only so far as everyone in a given community accepts as such.  When everyone believes the physical universe obeys rules of mathematics, those who make up the community around this belief can communicate among themselves and build a logical structure building on that common assumption.  That the explanations should be consistent with the reality that they all share in observing reinforces this process.  Fred Brooks, while discussing authority and project management, said that the important thing is not the manager is right, in factual sense, but that his authority is universally respected so that everyone can get onboard the same boat.  (I can’t remember where I read this–this was not in The Mythical Man Month, if I recall correctly, but in some business publication that I read online…could anyone send the link?)  In the same vein, however, achieving consensus when there is a disagreement requires understanding the logic behind why the disagreeing parties believe as they do and finding a logic that melds the competing explanations together…not beating over the head of one or the other with just one side’s explanation.  This can be hard:  cosmological constant hard.