A curious little Twitter War broke out over whether Barney Frank, the former congressman from Massachusetts, is really a “liberal” from the perspective of a Sanders supporter. To this, Harry Enten, of Fivethirtyeight, added this little tidbit:
“Funny little thing, DW-Nominate indicates that on all roll calls Barney Frank was actually slightly more liberal than Bernie Sanders.”
What the heck is this supposed to mean? It turns out this means next to nothing.
The idea behind DWNominate and related techniques is exactly like the problem of Google Translate. When people thought about using computers to translate, they wanted to translate based on grammatical and lexicographic “principles.” It failed miserably because languages are too complicated and it was impossible to reduce all the rules to something so simple that even a computer could understand. Google Translate was revolutionary in the sense that it gave up trying to understand the language but used statistical analyses of frequencies using allegedly “identical documents” where skilled humans had already translated between the languages to identify the equivalent words and expressions. The result has been generally pretty good for workaday translation, but not if you want to convey something subtle and meaningful across languages.
DWNominate operates on the exactly same principle–it is a machine learning algorithm developed before people knew much about machine learning. Basically, we don’t know what a “liberal” or conservative is, but we know who are liberals and conservatives, supposedly. So, we start by identifying someone in Congress as a liberal (or a conservative). If another congressman votes against that person, that person may be even more liberal or conservative. If a person votes consistently against the original congressman, that person must be conservative. If another person occasionally votes against the original liberal but hardly ever on the same side as the person identified as a conservative, that person must be even more liberal. Keep feeding the votes with hundreds (or thousands, if you want to go historical) of congressmen and tens of thousands (or hundreds of thousands) of votes, rinse and repeat, then you get a set of numbers that represent how “liberal” and “conservative” they are relative to each other–even if you have no idea what exactly a liberal or a conservative is supposed to be. Not bad for a quick comparison and quantification of the more or less obvious liberals, moderates, and conservatives, but troublesome if you take it too seriously–like Google Translate.
The trouble that began brewing in the last couple of decades is that all Democrats began to vote alike and all Republicans voted more or less in unison. Thus all Democrats look “liberal” and all Republicans look “conservative,” according to DW-Nominate. Since their scores are nearly the same, it is not clear what exactly it means if they are slightly different from each other. At best, what it means is that Barney Frank voted with other Democrats pretty much all the time while Sanders occasionally voted on the same side as “known” conservatives (according to DW-Nominate scores) against the Democrats some of the time. Perhaps we can dig into the vote themselves to identify what these are. I presume they are issues like guns, where Sanders is known to differ somewhat from the Democratic conventional wisdom. Does this mean that Sanders is more or less “liberal” than Frank? Who knows? What the heck does “liberal” or “conservative” mean at this level of nuance? (Especially since what they wound up voting on is not itself a random sample anyways–political leaders choose what gets voted on on the floor of Congress, as consequence of all manner of opaque politicking among themselves.)
Perhaps when people claim to be empirical, as in using data, it might help a bit if they actually know of what they are talking about and how that “data” has come to be generated in the first place….