[LINK TO THE HOME PAGE OF HIS SITE]

RELATED POST ON CONFIRMATION BIAS [LINK] 

THIS POST IS A CRITICAL REVIEW OF THE US ARMY RESEARCH PAPER UNWRITING THE FUTURE THAT PRESENTS A SERIES OF MILITARY SURPRISES AND SHOCKS IN THE HISTORY OF WARFARE FROM PEARL HARBOR TO THE 911 ATTACKS TO ARGUE THAT THE DATA SHOW THAT THESE ASSUMED SHOCKS WERE IN FACT PREDICTABLE IF WE HAD ONLY KNOWN BACK THEN WHAT DATA TO LOOK FOR AND HOW TO INTERPRET THEM. THE FULL TEXT OF THE RESEARCH PAPER IS AVAILABLE AT THIS SITE IN PDF FORMAT AND CAN BE DOWNLOADED BY CLICKING ON THIS LINK  unwriting

RELEVANT SECTIONS OF THE LARGE SOURCE DOCUMENT ARE PRESENTED BELOW

HERE WE ARGUE THAT THE INTERPRETATION OF THE DATA EX-ANTE MADE WITHOUT INFORMATION ABOUT THE OUTCOME IS RESEARCH INTO THE UNKNOWN WHERE THE ONLY INFORMATION IS THE EX-ANTE DATA AND WHERE A FORECAST IS MADE WITH THE DATA AND NOTHING ELSE. A TRACK RECORD OF CORRECT FORECASTS IN THIS WAY IS EVIDENCE OF SOUND RESEARCH METHODOLOGY AND GOOD DATA. HOWEVER, THE INTERPRETATION OF THE SAME DATA EX-POST BY INDIVIDUALS WITH INFORMATION ABOUT THE OUTCOME IS LIKELY TO BE GUIDED BY THE KNOWLEDGE OF THE OUTCOME AND THUS IMPOSE A FORM OF CIRCULAR REASONING KNOWN AS CONFIRMATION BIAS. IT IMPLIES THAT THE DATA ANALYSIS WAS NOT OBJECTIVE AND UNBIASED BUT RATHER BIASED BY THE KNOWLEDGE OF THE OUTCOME. IN SUCH CASES A MUCH HIGHER PERCENTAGE OF CORRECT FORECASTS CAN BE FOUND BUT THE BETTER PERFORMANCE OF THESE FORECASTS HAVE NO INFORMATION VALUE ABOUT THE UTILITY OF THE DATA IN MAKING AN OBJECTIVE AND UNBIASED FORECAST. CONFIRMATION BIAS DOES NOT LEAD TO USEFUL RESEARCH OUTCOMES. 

THE AUTHORS ACKNOWLEDGE THAT THEIR METHODOLOGY INTRODUCES CONFIRMATION BIAS IN THEIR RESEARCH BUT DEFEND THE METHODOLOGY TWO WAYS. FIRST, THEY POINT OUT THAT IN THE CASE OF THE CUBAN MISSILE CRISIS OF 1962 AND THE ARAB-ISRAELI WAR OF 1967 BOTH THE NIC AND THE CIA WERE ABLE TO MAKE CORRECT FORECASTS USING THE CONFIRMATION BIAS METHODOLOGY IN A SIMILAR BUT NEW SITUATION.

IN A SECOND DEFENSE OF CONFIRMATION BIAS, THE AUTHORS CITE THE CASE OF THE  PEARL HARBOR ATTACK WRITING THAT THE FORECAST FAILURE WAS A CREATION OF NOISE IN THE DATA – THINGS THAT LOOK LIKE DATA BUT ARE REALLY IRRELEVANT. THE PROBLEM WAS THAT THE ANALYSTS WERE BURDENED DOWN WITH “NOISE”, DATA THAT APPEARED TO BE RELEVANT EX-ANTE BUT WERE FOUND TO BE IRRELEVANT EX-POST . 

RESPONSE#1 TO THE DEFENSE OF CONFIRMATION BIAS: IN THE CASE OF THE CORRECT FORECASTS WITH CONFIRMATION BIAS METHODOLOGY, WE WOULD LIKE TO POINT OUT THAT THE ARGUMENT IS NOT THAT CONFIRMATION BIAS GUARANTEES A FALSE FORECAST BUT SIMPLY THAT THE FORECAST IN AND OF ITSELF DOES NOT CONTAIN USEFUL INFORMATION. CONFIRMATION BIAS DOES NOT RULE OUT THE POSSIBILITY OF CORRECT FORECASTS IN SIMILAR SITUATIONS. 

RESPONSE#2 TO THE DEFENSE OF CONFIRMATION BIAS: THE ARGUMENT THAT CONFIRMATION BIAS HELPS US TO UNDERSTAND EX-POST THAT SOME OF THE DATA WERE IRRELEVANT AND THEREFORE NOISE DOES NOT IMPLY THAT THE DATA THAT WERE NOISE IN ONE SITUATION ARE ALSO NOISE IN SITUATIONS THAT ARE NOT EXACTLY THE SAME. 

CONCLUSION: WE CONCLUDE FROM THIS ANALYSIS THAT THE PROPOSAL IN “UNWRITING THE FUTURE” THAT CLAIMS AN ADVANTAGE OF CONFIRMATION BIAS SUCH THAT SITUATIONS THAT WERE PREVIOUSLY THOUGHT TO BE UNPREDICTABLE CAN BE RENDERED PREDICTABLE  BY THE USE OF CONFIRMATION BIAS IN THE INTERPRETATION OF DATA IS STATISTICALLY AND METHODOLOGICALLY FLAWED. THE FINDINGS OF THIS RESEARCH AND THOSE OF ANY RESEARCH THAT RELY ON CONFIRMATION BIAS ARE UNRELIABLE. THE USE OF CONFIRMATION BIAS IS ALSO FOUND IN CLIMATE CHANGE RESEARCH  [LINK] . PREDICTABILITY MEANS CREATING INFORMATION ABOUT THE FUTURE WITH EX-ANTE DATA. THE PROPOSED PROCEDURE OF CREATING INFORMATION ABOUT THE FUTURE WHEN INFORMATION ABOUT THE OUTCOME IS USED IN THE PREDICTION IS NOT PREDICTABILITY. 

RELEVANT SECTIONS OF THE SOURCE DOCUMENT

Unwriting the Future: Decoding the Next Strategic Shock
by Colonel Sean Hunt Kuester
United States Army War College Strategy Research Project
Under the Direction of: Professor William Braun

ABSTRACT: Strategic shocks are predictable and the trend for the next defense relevant strategic shocks are extant and discernible. Specifically, gray zone exploitation, enhanced by technological advances, are elevating the probability of confrontation between the US and competitor states, and erosion of the liberal world order. To avoid the next shock the US must assess gray zone competition and contest gray zone actors through a whole of government approach. Rival states are increasingly emboldened to exploit the gray zone and take advantage of outmoded US security policy and legacy defense concepts. The US defense establishment must prioritize efforts to replace these strategy documents with ones that match the reality of the current and projected security environment.

BLACK SWANS, SHOCKS, AND GRAY ZONES

1. STRATEGIC SHOCK: A strategic shock is an event that punctuates the evolution of a trend, a discontinuity that either rapidly accelerates its pace or significantly changes its trajectory, and, in so doing, undermines the assumption on which current policies are based… Shocks are disruptive by their very nature, and … can change how we think about security and the role of the military. We have to approach one nuance of this definition with caution; that is, the use of the word discontinuity. The Japanese were not shocked by the attack on Pearl Harbor, nor was Usama bin Laden shocked by 9/11. For these actors, the shocks they triggered represented a natural progression or manifestation of their intended continuity. This is a salient point illustrating the need to find the trend with an aim toward thwarting the shock. At the other end of the definition spectrum, Peter Schwartz and Doug Randal offer a simpler yet excellent distillation of this concept. According to Schwartz and Randall, at its most elementary level, a strategic shock is a “game changing event.”7 Shock events require the affected organization to transform.
   Dr. Helene Lavoix adds further clarity, expanding the definition with a view respecting urgency and compulsion toward decision making, “…strategic shocks are unexpected changes occurring in a society’s or polity’s environment and to which actors will and must react…”8 A final characterization offered by Professor Freier will serve as the benchmark, “Strategic shocks jolt convention to such an extent that they force affected institutions to fundamentally reorient strategy, strategic investments and mission. Topical literature incorrectly uses the word surprise and shock interchangeably. Professor Freier argues that, “There is no scientific break point between strategic shock and strategic surprise.”10 To the contrary, the two terms are not the same in this context. A shock is more severe both in initial impact and ramifications. A shock necessarily impels an action, whereas a surprise might impel action. The developments resultant from surprise can be addressed using preexisting resources, doctrines, intellectual models and policy. Conversely, and this is the crucial discriminator, a shock requires “…affected institutions to fundamentally reorient strategy, strategic investments and mission.”11 Broadly speaking, a shock requires immediate and more consequential change, compelling organizational adaptation to avoid an existential consequence.
2. THE BLACK SWAN: Also found within the body of literature is the idea of “black swans.” Black swans are those rare shocks that defy analysis and operate unattached from discernable trends. The 2011 Fukushima nuclear plant accident is a contemporary example of a black swan. Curiosmatic illustrates this point, “With a high sea wall, backup generators and extensive emergency planning, the Fukushima nuclear plant seemed ready for anything. It wasn’t. On March 11, 2011 a massive tsunami struck, resulting in the partial meltdowns of three reactors.”12 A.J. Masys describes the phenomenon most simply and succinctly, “Black swans represent the unpredictable.” This essay will not address or strive to anticipate black swans. By their nature they are unpredictable. Therefore, devoting resources to their prediction is folly. Strategic shock, however, is predictable. Preparing for shocks will reap strategic security benefits, even if the preparation is for a general phenomenon and not a specific event.
3. THE GRAY ZONE; The gray zone, according to Freier and others is “…a broad carrier concept for a collection of sometimes dissimilar defense-relevant challenges.”14 The authors go on further to state, “…A coherent whole-of-government concept for combatting gray zone challenges would be ideal. However, it is likely not forthcoming. Thus, the DoD should not wait for definitive national-level guidance on countering gray zone competition before thoughtfully considering its own options.”15 However, this definition wholly understates that at its’ core, the gray zone is a space where adversaries hope to advance political ends by employing all instruments of power in traditional, unconventional and unpredictable ways. Freier et. al. acknowledge this, but shied away from addressing it in detail and thereby restricted gray zone challenges to a mostly military sphere.16 The authors’ consistent trajectory toward the military instrument of power and forfeiture of national level coordination is antithetical to efficient corporate practice and leads to a misconception of the gray zone. According to Michel J. Mazarr, employing gray zone approaches, Pursues political objectives through cohesive, integrated campaigns; Employs mostly nonmilitary or non-kinetic tools; Strives to remain under key escalatory or red line thresholds to avoid outright, conventional conflict; and, moves gradually toward its objectives rather than seeking conclusive results in a specific period of time. Employed by US challengers, gray zone strategies: use all instruments of power; conspire to bypass US traditional advantages like military hegemony, domain over-match, and strict adherence to international law and accepted norms.
4. SECTION II: TRENDS PRECEDING STRATEGIC SHOCKS. Here we examine historical shocks to make a case for their detectability. The analysis shows that each shock was preceded by large bodies of data that can be related to the shock after the fact. When assembled, these data constitute trend lines indicating the impending shock. The presence of a trend line is the discriminating factor between a strategic shock and a black swan. The following three examples illustrate the predictability of shocks.
5. SHOCK EXAMPLE#1 PEARL HARBOR: The actual attack on Pearl Harbor lasted only 2 hours.18 Yet the brief duration of the physical assault belies decades of political signaling; years of deteriorating Japanese-US relations; and a gradual but evident uptick in Japanese aggression in Southeast Asia and the Pacific. Significantly, the aggression noticeably heightened in the weeks preceding the attack. The US met each act of perceived hostility or aggression with countervailing moves that Japan thought unjust. By 1941, Japan felt existentially threatened by the US. However, the horror and suddenness of the assault result in a consistent focus on the day of the attack, not its lengthy and visible build up. This factor combined with the American narrative of a “surprise attack” obfuscate the underlying causes; implying that Americans may be predisposed to short-termism, viewing events as discrete occurrences versus episodes along a continuum. As early as 1932, US military war games identified the vulnerability of Pearl Harbor to carrier borne aircraft. In these war games, the adversary was understood to be Japan. “The strategic threat of Japanese surprise attacks”was not something the American military missed. Gordon Prange, author of the gold standard account of Pearl Harbor, describes conclusively the net result of trends stretching back into the previous century coupled with the code breaking system known as Magic when he concludes, “Make no mistake about it, Japan was going to war, and those with access to Magic knew it. The trend supporting an impending shock was clear.
6. SHOCK EXAMPLE #2: FALL OF THE SOVIET UNION: Contrary to popular conceptions, the Soviet Union did not collapse. It crumbled steadily, and visibly, the way a sea cliff crumbles under the relentless pressure of the surf. Naturally, foretelling the dissolution of a state as vast as the USSR is more difficult than detecting a single discreet attack such as Pearl Harbor. Nevertheless, the sign posts for this dissolution were abundant. Prior to 1991 the CIA had raised the issue of Soviet state fragility. Specific trends were identified in policy decisions, economic factors, and adjustments by the security apparatus that all supported the CIA assessment that “the communist order was finished”. In his book, Leon Aron begins with the surprise hypothesis but in fact ends up making the case for the utter predictability of the Soviet Union’s downfall in 1991 and the end of the Cold War and the rise of more plentiful and powerful non-state actors, a new and more complex security paradigm.
7. SHOCK EXAMPLE#3: 911 ATTACKS: The September 11th 2001 attacks are the clearest example refuting the myth that the proliferation of non-state actors make prediction impossible. The 9/11 strikes were part of an established Al Qaeda pattern. Some of the precursor incidents occurred in close proximity to the final attack provide ominous indicators, that establish trends. In his book, Erik  Dahl makes a solid case for the trends that unequivocally indicated the coming 911 attack because of the known Al Qaeda intent to strike the US. The strategic warnings were prolific and, as in the case of Pearl Harbor, intensified immediately preceding the final blow. Dahl lays out an explicit trend line dating back as far as 1995. Stephen Marrin illustrates a similar case and reinforces the point that was crystal clear to the intelligence community; Al Qaeda was not just plotting attacks, they were prosecuting them in an ever more lethal fashion. While the attacks were chiefly overseas against US installations such as embassies and warships, it was undeniable that A lQaeda was resolved to strike on US soil. While it is debatable whether or not all the 9/11 hijackings and separate strikes could have been thwarted, there was credible evidence that Al Qaeda intended to target the US with a high profile, mass casualty producing attack employing aircraft.
8. SECTION III: WHY THESE SHOCKS WENT UNDETECTED: It is now evident that each of the aforementioned shocks were not black swans. Each shock was preceded by observable incidences and data whose constituent parts formed obvious trends. So why did governments, academia, and the assemblage of stakeholders fail to see the signals and synthesize this data into action? Unfortunately, there is no answer that will categorically satisfy this question for any of the examples. Here we offer an insight into the reasons for failure, and there relevance to the next strategic shock.
9. Kuester addresses the Issue of Confirmation Bias: The topic of futures versus hindsight pattern recognition must be discussed. Critics will certainly stress that following an event it is easy to sift and pick data to reconstruct the pattern of an impending shock and thereby misrepresent the true difficulty in forecasting said event. There is no argument that after the fact, analytical reverse engineering is possible. By no means, though, does this diminish the ability to conduct predictive analysis. In fact, there is strong historical precedent and it is this:  the National Indications Center (NIC) successfully predicted Cuban missile crisis events in 1962 and the Arab-Israeli war in 1967.  The CIA, like their colleagues in the NIC, also foretold the 1967 Six-Day War.
10. The reality is analytical failures garner more attention than successes. Also, consequences of repeating failure fuel a worthy desire to prevent recurrence. Finally, fixation with failure is attributable in part to human psychology. As Kathryn Schulz states in her book, Being Wrong, “…we positively excel at acknowledging other people’s errors. In fact, if it is sweet to be right, then – let’s not deny it – it is downright savory to point out that someone else is wrong.
11. About the Noise in the Pearl Harbor prediction data: In the case of Pearl Harbor, as Stephen Marrin tells us, there existed “…a body of evidence which could have been pieced together to warn national leaders.” What then, went wrong? If the necessary information was available, why was the garrison at Pearl Harbor not fully prepared for the Japanese onslaught? A major reason is, in addition to the highly valuable intelligence possessed by analysts and decision makers, the former was burdened with sifting through what has come to be known as “noise.” Noise is background information that while related to the case, is for all practical purposes, irrelevant and does not assist analysts or decision makers. What’s more, the Pearl Harbor case revealed “a sea of extraneous noise.” Further, the Pearl Harbor attacks occurred in an era that experienced increased use of radio and trans-oceanic cable communications. This foreshadowed that technology was on the cusp of rapidly accelerating volume and velocity of communication. As Dahl illustrates in numerous case studies from Pearl Harbor in 1941 to the 2012 attack on the US embassy in Benghazi, analysts now confront the challenge of sweeping increases in the sheer quantity of data from which the identification of the relevant is not easy.
12. The Soviet Union and the US: Paradigm Paralysis:  For a quarter century now, the US has suffered from “paradigm paralysis” imposed by the paradigm of containing Soviet communism. This paradigm was the basis of US global policy throughout the Cold War and into the the 21st century. It is derived from a global state of centric order led by the US on the basis of conventional military force as the primary vehicle to ensure compliance. The flaw in this paradigm became evident after the demise of the Soviet Union. At the conclusion of the Cold War it was thought that we had shifted from a bi-polar to uni-polar world but almost immediately a multi-polar world began to take shape.
13. Non State Actors: Additionally, we saw the rise of non-state actors as a significant factor in global affairs. This development is a challenge to the Westphalian notion of state supremacy in the international system. Secondly, one of the more virulent expressions of this challenge became radical Islam. From a US lens this latter hazard eventually emerged as the fundamental ideological threat, just as Communism had been. Yet, US national and defense policy making enterprises are only now waking up to the need for new defense frameworks and paradigms to address these fundamental changes in the global power architecture.
14. The 1991 Gulf War: A significant event in these developments was the 1991 Gulf War. It exposed the false sense of security, overconfidence and complacency that seeps in following what is considered to be a huge success – as in the fall of the Soviet Union. The rest of the world took note of the Gulf War’s results. States and non-state actors alike easily grasped that a head to head classical military confrontation with the US would be ill-advised; new methods were required. Conversely, having subdued Saddam Hussein and with no peer competitors, the West was content. The Gulf War validated the dominance of conventional forces and doctrines; Air Land Battle was the supreme war fighting concept and was even updated in 1993. Moreover, the 1991 Gulf war followed closely on the heels of the fall of the Berlin Wall; Western states desperately thirsted for a long-sought peace dividend. Confident and assured in proven methods and concepts, western military and security related thinking stagnated.
15. US defense doctrine, strategic theory and security thinking became anachronistic. These models were ideally suited for the most dangerous forms of aggression but not for the most likely. Victor Davis Hanson in his 2009 edition of The Western Way of War, captures the essence of the fatal flaw of mirror imaging. This defect sees one preparing for what one most desires, that manner of conflict for which he or she is best suited. Hanson describes the western predilection for classical direct combat as a narcotic we cannot put away.
16. 9/11 and New Conceptions of Warfare. A deficiency of imagination prevented the US defense and security establishments from predicting the 9/11 attacks. They were looking for a different kind of threat. The establishment failed to realize that the nature of warfare was changing and Gulf War era defense concepts were mismatched for emergent threats. The 9/11 attack definitively marked the pivot to a new appreciation for the emerging threat. Since those attacks, numerous other events have solidified this evolution toward new conceptions of warfare. However, in 1989, even before the Gulf War, a small team of military thinkers posited that the character of warfare was, in fact, entering a new phase. These theorists saw clear trends developing in the security environment and referred to this latest evolution as Fourth Generation Warfare, 4GW. Envisioning that US technological and material dominance would necessarily drive competitors to avoid such strengths, this team predicted a rise in unconventional off-setting methods as major features of the 21st century operational environment (OE). Some of the characteristics of this new OE they presaged were: ideological based movements using low tech means to contest high tech forces; the desire of adversaries to collapse enemies from within, avoiding military might altogether and attacking enemy society at home; adversary use of democratic state freedoms against that state; direct attacks against a state’s culture; a “…blurred distinction between war and peace…” and reliance of “non-national or transnational” ideological support.45 Today, these characteristics are all too clear in the visage of terrorism and those nations exploiting the gray zone. Yet, the bureaucratic defense establishment during this era was unable to imagine a new paradigm or conceive that the nation with the globe’s premier military could be appreciably wounded. As stated above, the categorical success seen by US and other Western forces in the 1991 Gulf War against a linear array of conventional forces further solidified the unflagging belief in established doctrine and defense concepts.
17. Yet, the nineteen 9/11 attackers were a loosely confederated group of terrorists driven by ideology, employing methods that did not conform to Westphalian conceptions of warfare and operated on US soil. AQ represented an entirely different sort of enemy whose practices and structure presented a challenge ill-suited for DoD’s mostly conventional wherewithal and conceptions of adversaries and how to confront them. In short, one major reason the 9/11 attacks occurred was that the terrorist archetype was not the preferred adversary the DoD was purpose built to oppose. The 9/11 attack was the unfortunate validation of 4GW. It demonstrated that the US was vulnerable to new unconventional forms of attack that mitigate traditional advantages, particularly military strength. The nearly two decades since have illustrated that the US has yet to address this fact.
18. Larry Hancock describes the depth of institutional, and intellectual malaise at the time of the 9/11 attack. National Security Advisor Condoleeza Rice…testified…that very few government or military officials had considered that hijacked planes might…be used as weapons…no one had ever even contemplated commercial aircraft being used as weapons in attacks on American cities.46
    The 9/11 commission report provided the most damning judgement of all, however, when it stated, “…the most important failure was one of imagination.
19. A corollary to the 9/11 attack, that further illustrates a western lethargy in grasping new adversarial models, is the Russian annexation of Crimea. Like 9/11 and Pearl Harbor, this event was a deliberate, well planned and highly orchestrated action with ample warning signs. There were clear indications Russia would act boldly to maintain sway over those areas it considers within its sphere of influence. Russian intentions toward Ukraine became known as early as the 1990’s with contingency plans to annex this region. Ukrainian leaders issued warnings of the same in the late 2000. In 2008, the Russian foreign minister stated that they will do all they can to prevent Ukraine’s and Georgia’s accession into NATO.
20. In that same year Russia actually invaded Georgia. With NATO and the US consumed in Iraq and Afghanistan the timing was highly calculated and opportunistic. The fact that Georgia was not a NATO member ensured blowback on Russia would be an acceptable risk. Six years after the Georgia invasion Russia deftly seized Crimea. To do so, Russia exploited the gray zone, employing multi-domain and whole of government actions below the threshold of armed conflict representing a mixture of hybrid, unconventional and conventional warfare. The annexation saw no massing of troops on international borders or a declaration of war. Rather, Russia exploited Ukrainian domestic politics and populations, inflamed internal grievances, manipulated social media and saturated the information domain with misinformation.
21. Confronted by a new conception of warfare, the west was outmaneuvered. Yet, like 9/11, this shock had a trend line. Russia was heavily invested in understanding the inner workings of the Crimea; the West was not. As Stephen Blank explains, “Russian intelligence, military, economic, informational, ideological, and other forms of penetration of the Crimea in anticipation of an overall nullification of Ukraine’s de facto…sovereignty over the area have been long apparent.”50 Russian actions were years in the making, but awareness of this 21st century gray zone exploitation is relatively nascent.