Book review: Superforecasting redux

In Superforecasting: The Art & Science of Prediction, social scientist Philip E. Tetlock and journalist Dan Gardner (Crown Publishers, September 2015) observe that at its heart, politics is usually about predicting the future. The exercise boils down to finding and implementing policies that will do best for the public interest (general welfare or common good), regardless of how one defines the concept.

What most accurately describes the essence of intelligent, objective, public service-oriented politics? Is it primarily an honest competition among the dominant ideologies of our times, defense of one’s social identity, a self-interested quest for money, influence and/or power or some combination? Does it boil down to understanding the biological functioning of the human mind and how it sees and thinks about the world? Is it some something else entirely?



Subject to caveats, Superforecasting comes down on the side of getting brain biology or cognition right. Everything else is subordinate. Superforecasting describes Tetlock's research into asking what factors, if any, can be identified that contribute to a person’s ability to predict the future. Tetlock asks how well intellectually engaged but otherwise non-professional people can do. The performance of volunteers is compared against experts, including professional national security analysts with access to classified information.

The conscious-unconscious balance: What Tetlock and his team found was that interplay between dominant, unconscious, fact- and common sense-distorting intuitive human cognitive thinking (“System 1” or the “elephant” as described before) and our far less-powerful but conscious, rational thinking (“System 2” or the “rider”) was a key factor in how well people predicted future events. The imbalance of power or bandwidth between conscious thinking and unconsciousness thinking is estimated to be at least 100,000-fold in favor of unconsciousness. The trick to optimal performance appears to be found in people who are able to strike a balance between the two modes of thinking, with the conscious mind constantly self-analyzing to reduce fact distortions and logic biases or flaws that the unconscious mind constantly generates.

Tetlock observes that a “defining feature of intuitive judgment is its insensitivity to the quality of the evidence on which the judgment is based. It has to be that way. System 1 can only do its job of delivering strong conclusions at lightning speed if it never pauses to wonder whether the evidence at hand is flawed or inadequate, or if there is better evidence elsewhere. . . . . we are creative confabulators hardwired to invent stories that impose coherence on the world.”
Coherence can arise even when there's insufficient information. In essence, the human mind evolved an ‘allergy’ to ambiguity, contradictions and concepts that are threatening to personal morals, identity and/or self-interest. To deal with that, we rapidly and unconsciously makes those uncomfortable things go away.

It turns out, that with some training and the right mind set, a few people, “superforecasters”, routinely trounce professional experts at predicting future events. Based on a 4-year study, Tetlock’s “Good Judgment Project”, funded by the DoD’s Intelligence Advanced Research Projects Agency, about 2,800 volunteers made over a million predictions on topics that ranged from potential conflicts between countries to currency and commodity, e.g., oil, price fluctuations. The predictions had to be precise enough to be analyzed and scored.

About 1% of the 2,800 volunteers turned out to be superforecasters who beat national security analysts by about 30% at the end of the first year. One even beat commodities futures markets by 40%. The superforecaster volunteers did whatever they could to get information, but they nonetheless beat professional analysts who were backed by computers and programmers, spies, spy satellites, drones, informants, databases, newspapers, books and whatever else that professionals with security clearances have access to. As Tetlock put it, “. . . . these superforecasters are amateurs forecasting global events in their spare time with whatever information they can dig up. Yet they somehow managed to set the performance bar high enough that even the professionals have struggled to get over it, let alone clear it with enough room to justify their offices, salaries and pensions.”

What makes superforecasters so good?: The top 1-2% of volunteers were analyzed for personal traits. In general, superforecasters tended to be people who were open-minded about collecting information, their world view and opposing opinions. They were also able to step outside of themselves and look at problems from an “outside view.” To do that they searched out and integrated other opinions into their own thinking.

Those traits go counter to the standard human tendency to seek out information that confirms what we already know or want to believe. That bias is called confirmation bias. The open minded trait also tended to reduce unconscious System 1 distortion of problems and potential outcomes by other unconscious cognitive biases such as the powerful but subtle “what you see is all there is” bias, hindsight bias and scope insensitivity, i.e., not giving proper weight to the scope of a problem.


 

Superforecasters tended to break complex questions down into component parts so that relevant factors could be considered separately. That tends to reduce unconscious bias-induced fact and logic distortions. In general, superforecaster susceptibility to unconscious biases was lower than for other volunteers in the GJP. That appeared to be due mostly to their capacity to use conscious (System 2) thinking to recognize and then reduce unconscious (System 1) biases. Analysis revealed that superforecasters tended to share 15 traits including (i) cautiousness based on an innate knowledge that little or nothing was certain, (ii) being reflective, i.e., introspective and self-critical, (iii) being comfortable with numbers and probabilities, (iv) being pragmatic and not wedded to any particular agenda or ideology, and, most importantly, (v) intelligence, and (vi) being comfortable with (a) updating personal beliefs or opinions and (b) belief in self-improvement (having a growth mindset). Tetlock refers to that mind set as being in “perpeutal beta” mode.

Unlike political ideologues, superforecasters tended to be pragmatic, i.e., they generally did not try to “squeeze complex problems into the preferred cause-effect templates [or treat] what did not fit as irrelevant distractions.” Compare that with politicians who promise to govern as proud progressives or patriotic conservatives and the voters who demand those mind sets.

What the best forecasters knew about a topic and their political ideology was less important than how they thought about problems, gathered information and then updated thinking and changed their minds based on new information. The best engaged in an endless process of information and perspective gathering, weighing information relevance and questioning and updating their own judgments when it made sense, i.e., they were in “perpetual beta” mode. Doing that required effort and discipline. Political ideological rigor such as conservatism or liberalism was generally detrimental.

Regarding common superforecaster traits, Tetlock observed that “a brilliant puzzle solver may have the raw material for forecasting, but if he also doesn’t have an appetite for questioning basic, emotionally-charged beliefs he will often be at a disadvantage relative to a less intelligent person who has a greater capacity for self-critical thinking.” Superforecasters have a real capacity for self-critical thinking. Political, economic and religious ideology is mostly beside the point. Instead, they are actively open-minded, e.g., “beliefs are hypotheses to be tested, not treasures to be protected.”

Tetlock asserts that politicians and partisan pundits opining on all sorts of things routinely fall prey to (i) not checking their assumptions against reality, (ii) making predictions that can’t be measured for success or failure, and/or (iii) knowingly lying to advance their agendas. Politicians, partisan pundits and experts are usually wrong because of their blinding ideological rigidity and/or self- or group-interest and the intellectual dishonesty that accompanies those mind sets. Given the nature of political rhetoric that dominates the two-party system and the biology of human cognition, it is reasonable to argue that most of what is said or written about politics is more spin (meaningless rhetoric or lies-deceit) than not.

Is Tetlock’s finding of superforecasters real? Does that point to a human potential to at least partially rationalize politics for individuals, groups, societies or nations?

A fact and logic distortion-reducing political ideology

This is my most refined articulation of how social and cognitive science knowledge might be applied to mainstream politics. The point is to describe a mind set, set of morals or political ideology that might partially rationalize politics. Partially rationalized politics means, relative to existing the existing state of affairs, politics based more on (i) unbiased or real facts, and (ii) less biased common sense. The underlying assumption is that politics that is at least somewhat better grounded in reality and 'logical' reason will do better in the long run than "normal" or standard nonsense politics.

People can reject the assertion that mainstream politics is more nonsense than not. Regardless of popular belief, cognitive and social science makes it clear that most people deal more in nonsense (false facts and flawed common sense) than not. That's just how the human mind works when it comes to politics.

Current cognitive and social science of politics strongly suggests that humans generally have a very limited capacity to see unbiased reality or facts and apply unbiased common sense to the reality they think they see. The situation is complicated and multi-faceted. Evolution resulted in a human mental capacity that was at least sufficient for modern humans to survive the early days. Building existing human civilization has been based on about the same mental firepower our modern ancestors had. What evolution conferred was a mind that operates using (i) a high bandwidth unconscious mind or mental processes that can process about 11 million bits of information per second, and (ii) a very low bandwidth conscious mind that can process at most about 45-50 bits per second.

Although our conscious mind believes it is aware of a great deal and is in control of decision-making and behavior, that perception of reality is more illusion than real. Our unconscious thinking exerts much more control over decision-making and behavior than we are aware of. Our conscious mind plays into the illusion. Unconscious innate biases, personal morals, social identity and political ideology all inject distortions into our perceptions of reality or facts and our application of common sense. Conscious reason acts primarily to rationalize or defend unconscious beliefs and rationales, even when they are wrong.

False unconscious beliefs include a widespread fundamental misunderstanding of democracy. Our political thinking and behaviors are usually based on major disconnects with reality. Our unconscious mind is usually moralistic, self-righteous and intolerant. That creates a human social situation where “our righteous minds guarantee that our cooperative groups will always be cursed by moralistic strife.”

Based on that description of the human condition, it's reasonable to believe that mostly irrational human politics cannot be made demonstrably more rational. That may or may not be true. Some evidence that suggests that at least some people can operate with significantly less bias in perceiving reality and conscious reasoning. They are measurably more rational than average. The finding of superforecasters among average people and their mental traits suggests that politics might be partially rationalizable for at least some people, if not societies or nations as a whole.

Research observations on how superforecasters improve over time, i.e., predict, get feedback, revise, and then repeat, there is reason to believe that evidence-based politics could be a route to better policy. Although the effort is in its infancy, there is some real-world evidence that cognitive science-based political policy can be simple but very successful. The trick is figuring a way to how to deal with personal morals, self-interest and other unconscious distortion sources that impedes politics based on less biased reality and common sense.

If it’s possible to rationalize mainstream politics at all, accepting the reality of human cognition and behavior is necessary. There’s no point in denying reality and trying to propose false reality-based solutions. Given that, one needs to accept that (i) politics is fundamentally a matter of personal morals, ideology, and self- or group identity, and (ii) current political, economic, religious and/or philosophical moral sets or ideologies, e.g., liberalism, conservatism, capitalism, socialism, libertarianism, anarchy, etc, are fundamental to what makes people tick in terms of politics.

 One can argue that since existing ideological or moral frameworks have failed to rationalize politics beyond what it is now, and probably always has been, then a new moral or ideological framework is necessary (although maybe not sufficient). Since morals are personal and they vary significantly among people, there’s no reason to believe that a set of morals or ideological principles cannot be conceived that could temper or significantly substitute for existing morals such as the care-harm moral foundation that tends to drive liberal perceptions and beliefs, or the loyalty-betrayal and other foundations that drives conservatives.

How can one rationalize politics? Swim downstream: Why swim upstream if there’s a potential solution to be had by swimming downstream with the cognitive current? Morals or variants thereof that essentially everyone already claims to adhere to (even though science says that’s just not the case) seems like a good place to start. Most people (> 97% ?) of all political ideologies claim that they (i) work with unbiased facts, and (ii) unbiased common sense. And, most people believe that their politics and beliefs best serve the public interest (general welfare or common good). Few or no people say they rely on personally biased facts and common sense or that that’s the best way to do politics, although social science argues that that’s exactly how politics works for most people.

Three pragmatic morals:  Can it really be that simple?

If that’s the case, then a set of three already widely accepted morals or political principles that might operate to rationalize politics to some extent without being rejected out of hand. They are (i) fidelity to less biased facts, and (ii) fidelity to less biased common sense, both of which (iii) are applied in service to the public interest.

Service to the public interest: Service to the public interest means governance based on identifying a rational, optimum balance between serving public, individual and commercial interests based on an objective, fact- and logic-based analysis of competing policy choices, while (1) being reasonably transparent and responsive to public opinion, (2) protecting and growing the American economy, (4) fostering individual economic and personal growth opportunity, (5) defending personal freedoms and the American standard of living, (6) protecting national security and the environment, (7) increasing transparency, competition and efficiency in commerce when possible, and (8) fostering global peace, stability and prosperity whenever reasonably possible, all of which is constrained by (i) honest, reality-based fiscal sustainability that limits the scope and size of government and regulation to no more than what is needed and (ii) genuine respect for the U.S. constitution and the rule of law with a particular concern for limiting unwarranted legal complexity and ambiguity to limit opportunities to subvert the constitution and the law.

As explained here, that conception of the public interest is broad. It reflects the reality that politics is a competition for influence and money among competing interests and ideologies, all of whom essentially always claim they want what’s best for the public interest. A broad conception encompasses concepts that fully engage all competing interests, morals and ideologies, e.g., (i) national security defense (a conservative moral or concern), (ii) concern for fostering peace and environmental protection (liberal) and (iii) defense of personal freedom (libertarian). Although broad, that public service conception is meaningfully constrained by the first two pragmatic morals, less biased fact and less biased common sense.

For regular “subjective” or non-pragmatic politics, neither of those are powerful constraints on most people’s perceptions of reality or facts or their conscious thinking about politics. That’s not intended as a criticism of people’s approach to or thinking about politics. It’s intended to be a non-judgmental statement of fact based on research evidence: For politics, “. . . . cherished ideas and judgments we bring to politics are stereotypes and simplifications with little room for adjustment as the facts change. . . . . the real environment is altogether too big, too complex, and too fleeting for direct acquaintance. We are not [intellectually] equipped to deal with so much subtlety, so much variety, so many permutations and combinations. Although we have to act in that environment, we have to reconstruct it on a simpler model before we can manage it.”

From existing mind sets → AN AVALANCHE OF CRITICISMS!: Many or most liberals, conservatives, libertarians and others will instantly jump all over this “political ideology” as nonsense. For example, how could such a broad conception of serving the public interest make one iota of difference in how allegedly distorted political thinking and debate works now?

That’s a good, reasonable question, the answer to which is already given in the discussion, i.e., fidelity to less biased fact and less biased common sense. The assumption is that in the long run, politics better grounded in reality and reason would make a difference for the better. Obviously, people who see a threat to their own beliefs and ideologies will reject that as nonsense. They already believe (know) that they employ unbiased fact and logic to politics, although the scientific evidence strongly argues that’s not true.
Plenty of other criticisms can be raised. Some libertarians and/or conservatives might claim that this subverts personal freedoms and that the concept pays only lip service to defense of personal freedoms. In other words, this ideology seems at best meaningless or at worst a Trojan horse of some sort, e.g., a smoke screen for socialism, fascism and/or tyranny. From a pragmatic POV, it’s easy to see, understand and anticipate that reaction from people trapped in their standard subjective political ideologies, e.g., liberals, conservatives, libertarians, socialists, etc.

What this conception does is it forces everyone and every ideology to (i) defend their policy choices on the basis of a less distorted world view and less biased common sense, and (ii) pay more than self-deluded and/or cynical lip service to serving the public interest. Everyone has to win arguments on less spun merits.

For standard ideologues, that makes this brand of “pragmatic politics” a dead on arrival nonstarter. That’s why politics based on these three political principles may be a new ideology. This won’t work for liberals, conservatives, libertarians, socialists or believers in any other existing ideology or set of morals. To accept this set of political morals, one has to move away from existing mind sets and accept this for what it is, i.e., advocacy of a cold, harsh competition in a brutal marketplace of less spun ideas and arguments based on less spun facts and realities.

Some thought has gone into this. Here are responses to a list of criticisms to this three morals-based political ideology.