The unexpected election of Donald Trump to presidency, I hope, will get us all to be a bit less “data sciencey” and become more “real sciencey.” (I don’t think that will happen, though–for the reasons to be explained below.)
The trouble with the reality is that it is complicated, full of the unexpected contortions and weirdness. There are, usually, enough predictable patterns that a sort of cheat sheet can be created that summarizes these patterns: what we call “formulas,” “theories,” “models,” “heuristics,” or “shortcuts.” The problem, however, is that while the reality is, well, real, these fancy models of ours are not. They are simplifications created in recognition that we don’t really understand (the whole of) reality. They are “usually” right, but not something to be relied upon as God-given truth at all times. They are certain to go wrong, and very wrong sometimes. We need to be cognizant of it and be ready to revise our theories when we run into the patches of reality that don’t fit our notion of how they should be. This is how real science works.
Physics has a much easier time dealing with the realities defying our theories. After all, if the particles, say, decide that they don’t want to obey our stupid theories, we are clever enough to know that just yelling at how foolish they are to disregard our wonderful theories is absurd. At most, we can yell at physicists who saw misbehaving particles if they really saw what they thought they saw. Experimental errors are common enough that, usually, this is the main reason why particles were seen misbheaving. But, sometimes, particles really did misbehave, and physicists rejoice: we need new theories!
This is not the case with social sciences. Like with beauty contest games that Keynes recognized, there often isn’t a clear “good answer” beyond what people think is the good answer. People are not trying to mess with the social scientists necessarily: they are simply trying to gravitate towards what they think is the best answer. If the social scientists say that the answer is A, then they will operate on the premise that the answer must be A, and in so doing producing evidence that the answer really is A. The more theorizing that social science does, the more data that its members dig up to support its models and theories and formulas, the more credence it generates for people to subscribe to its theories, and the more people follow social science theories, the more data humans generate that justifies social science models and theories. With greater “empirical” support from the data comes the hubris that these models and theories are not mere simplifications of the complex reality to something digestible, but the reality itself, superior to the actual reality. When pieces of reality deviates from the theory, it is they that are wrong that must be lectured at to change their ways, not the theory that is problematic. When DW-Nominate is used to “predict” the votes, the legislators who depart from the prediction are called the “errors.” Sam Popkin thought, when I casually used the terminology, now more than a decade into the past, that that was just plain wrong. I suppose that has had tremendous effect on how I viewed models after all–although not necessarily for good in the past decade.
Because of the inherent malleability of humans and their willingness to change their behavior, social science is particularly prone to becoming a cargo cult, stuck in its own echo chambers. The “data science-y” and “wonkish” attitude, obsessed with collecting and analyzing data that support its own findings and aversion towards negative findings, which, granted, is common enough in any empirical research endeavor, becomes particularly destructive in social sciences for twofold reasons: the willingness of people to “listen to experts” and change their ways makes negative results harder to find anyways, and the relative ease with which people can be made to change their behavior (and change the data that is generated) makes it easier to stand by the theory rather than try to update theory when contrary data emerges. The attitude that the only worthwhile goal is to change policy, rather than to understand the world better and develop better models and theories, furthers the bubble: you do not care for generating data that challenges the received wisdom and forces models to be revised, but for compiling data that support your worldview and, furthermore, change the future stream of data so that they comport to your models. That is, of course, exactly the prescription for a cargo cult science, or antithesis of real science. What’s the point of collecting data if all that they do is to confirm your theories? But if your interest is in promoting certain goals and the models and theories are merely tools to that end, that worldview starts to make more sense.
So that brings us to an awkward realization: social science, especially when paired with wonkism, is not a science at all. I noted before that science does not “believe.” If you place your goals, and associated models, formulas, and theories above the data, especially the contrary data, then you subscribe to a religion. Much of social science wonkism, I’ve come to suspect, is, in a sense, the worst kind of religious cult: a cult that does not realize it’s a cult and perceives itself to be a “science,” or worse, “the truth,” superior in morality to all unbelievers. This is something that needs to stop.
I repeat this a lot, but, at the turn of 20th century, Weber saw this coming: his idea of ideal bureaucracy is predicated on the observation that, in the end, the most valuable asset that the “bureaucrat” has for the decisionmakers is that the former has a better understanding of the reality, which is valuable as long as the decisionmakers want to base their choices on the reality–which is almost always the case. This comes with the caveat that the bureaucrat has no political agenda and is not trying to use his information as leverage to effect policy change that he likes–which, if true, will cause his advice to be rejected as a mere attempt at manipulation. Thus the bureaucrat, especially if he is involved in policymaking, cautioned Weber, has to be as apolitical as possible, or, to paraphrase the alleged story about Truman and his economic advisors, they need to grow multiple arms to account for different contingencies. Of course, in Truman’s case, the multi-armed economists were a distraction: he wanted to implement a policy and needed support and justification, not an explanation of how different things pan out differently. But you don’t need actual economic analysis for support and justification, just someone who looks the part and act the part. (Maybe not unlike using actors to teach MOOC classes, which was brought up seriously, and, to be honest, makes good sense if all that you need is to deliver the lines like you really mean it–which actors are better at than academics.) So you wind up with the wonks you have: people who can deliver the lines with stronger conviction, can look and act the part, nevermind they don’t care much for the “science.”
If we want to retake science from the wonks and “data science” charlatans, occasional reminders from the real world that our theories are not as good as we think are a good thing. The last big reminder, in 2008, the wonks were able to weasel their way out by being bailed out. One almost hopes that this will actually be a learning opportunity that teaches us some humility.