The SIGINT Philosopher: The Fallacies Behind the Scenes
Open PDFDocument date: 2011-03-15
Type: SIDtoday
Pages: 2
Genre: newsletter
Classification: UNCLASSIFIED
Dissemination: FOUO
A 2011 NSA SIDtoday opinion column in the "SIGINT Philosopher" series examining four cognitive fallacies — the Texas Sharpshooter, Gambler's, Sunk-Cost, and Appeal to Probability fallacies — and their implications for how analysts unconsciously distort intelligence assessments and decision-making.
-
People mentioned:
Rogers, Kenny, Tversky, Amos
Countries:
-
Tags:
cognitive bias, analytic tradecraft, apophenia, pattern recognition, intelligence analysis, analyst culture, cognitive psychology, organizational culture, evolutionary psychology, logical fallacies
Identifier:
fallacies-behind-the-scenes
PDF SHA-256:
e74d7d8a7032e2de61a159a2d5cd1bffba16fed68ec6fbf55516aa6f6e4c9431
My notes
(U) The SIGINT Philosopher: The Fallacies Behind the Scenes
FROM: (U//FOUO) [REDACTED]
Run Date: 03/15/2011
(U) At the end of the day, much of the work done in SID revolves around getting information to people who need to make a decision. Often that decision involves making an estimate as to how probable or plausible a given event is, and what implications such a hypothetical event might have on our national security. It seems straightforward enough. However, at the outset, there are undercurrents flowing through the minds of every analyst, before the fingers even hit the keyboard. The analyst might not even know they're at work, but they are. They're cognitive fallacies.
(U) There's no blame to be handed out -- these fallacies are hard-wired into our brains as a result of thousands of years of evolutionary instinct and decision-making to simply survive. In relative historic terms, we've only very recently shifted our attention from such perfunctory tasks to more complex avenues. So, in the analysis we undertake in our daily duties, here are a few biases to be aware of, or perhaps attempt to change, on an individual and collective level:
The Texas Sharpshooter Fallacy
(U) This logical fallacy is named for a Texan who shot a hole in his barn, drew red and white circles around it, and gathered the neighbors to brag about how great a shot he was. While finding significance and connections between people and events is what intelligence is all about -- the downside is that sometimes we stretch to make the case that relationships or significance are present where they aren't. When we make the case first, the evidence we gather tends only to reinforce it. The reverse should be the rule: go where the evidence leads, don't lead the evidence.
The Gambler's Fallacy
(U) Kenny Rogers knew when to hold 'em. So should we. Since we're hard-wired to look for patterns, we have a tendency to think of numbers and events as being orderly, when they're anything but. Neuroscientists call this phenomenon apophenia. For instance, if you're taking a turn at the roulette wheel, how many hits on black will it take before you think to yourself, "We're due for a red..."? Or perhaps you think because a number on the wheel has come up so often in the past, it's bound to be more absent in the future? Pioneering cognitive scientist Amos Tversky noted "It's natural to be inclined toward patterns when we observe cause and effect. But in nature, events rarely happen with equal frequency." Events are rarely ever "due" to occur, and the frequency of an event in the past is not always an indicator of its future appearance.
The Sunk-Cost Fallacy
(U) How many times have you been watching a terrible movie, only to convince yourself to stick it out to the end and find out what happens, since you've already invested too much time or money to simply walk away? This "gone too far to stop now" mentality is our built-in mechanism to help us allocate and ration resources. However, it can work to our detriment in prioritizing and deciding which projects or efforts are worth further expenditure of resources, regardless of how much has already been "sunk." As has been said before, insanity is doing the same thing over and over and expecting different results.
The Appeal to Probability
(U) Remember Y2K? We fetched our gas masks and stocked up on canned food for the impending doom that would come at the dawn of the millennium. But alas, midnight came and went with nary a global meltdown. That needless commotion might have been attributed to an appello probabilitatem, where an event is thought to be possible, and is hence erroneously determined to be plausible. In other words, it's the false assumption that because something can happen, it likely will happen. While preparation is a noble goal -- bear in mind the actual probabilities.
(U) There are hundreds of other fallacies that I could touch on, but the main point is to engender awareness that our decision-making processes are directly and unconsciously affected by nature's own hard-wiring. Keeping that fact in mind could have positive impacts on the way we choose to execute our mission. Who knows, perhaps there are cognitive fallacies that have developed -- not resulting from many generations of human evolution -- but from decades of SIGINT evolution? It may be worthwhile to consider the unintended effects on our collective processes that such an evolution may have had, and be on guard against cognitive fallacies and logical biases of all stripes, whether they be recent, or as old as humanity itself.
Now -- back to my poker game with Kenny...
(U) Have thoughts on this topic? See a related Tapioca Pebble.
Published in the Same Report
(10 documents)
-
Dealing With a 'Tsunami' of Intercept SIDtoday [pdf]
As part of NSAs efforts to implement Human Language Technology, prototype processors located at collection sites will soon be able to query content at "line speed" based on the language, it is written in.
People: Kulman, Anita H.The Intercept ↗ (2019-05-29) · “Inside NSA, Officials Privately Criticize "Collect It All" Surveillance” - The Intercept ↗ (2015-05-28) · Maass, Peter -
Is There a Sustainable Ops Tempo in S2? How Can Analysts Deal with the Flood of Collection? SIDtoday [pdf]
The NSA's Deputy Director for Analysis and Production discusses the unsustainable operational tempo caused by overwhelming data collection volumes, advocating for "Analytic Modernization" that would shift from treating SIGINT as "radioactive" NSA-only material to sharing raw intelligence with other agencies for collaborative exploitation.
Programs: A-SPACE -
The SIGINT Philosopher: Cognitive Overflow? SIDtoday [pdf]
A 2011 NSA SIDtoday opinion column reflecting on whether the SIGINT Directorate's collective analytical capacity has a finite "channel capacity" — borrowing from cognitive psychology to ask whether SID's ever-expanding mission, tools, and information load risks exceeding the combined cognitive limits of its workforce.
Marking: UNCLASSIFIED -
'Data Is Not Intelligence' SIDtoday [pdf]
A 2007 NSA SIDtoday summary of a keynote address by Deputy DNI for Analysis Thomas Fingar at the Analytic Transformation Symposium, outlining the Intelligence Community's agenda for improving analytical quality, cross-agency collaboration, and integrated operations through training, shared standards, and new tools like A-Space and the Library of National Intelligence.
Programs: A-SPACE -
"Signal v Noise" Column: Do We Need a Bigger SIGINT Truck? SIDtoday [pdf]
A 2012 NSA SIDtoday opinion column arguing that SID lacks adequate internal metrics to evaluate the value and cost of its SIGINT collection capabilities, using the analogy of melon sellers buying a bigger truck to illustrate how expanding capacity without measuring effectiveness is a flawed response to resource constraints.
-
Op-ed: Leave Bright Pebbles, Not Breadcrumbs, for Those Coming After You SIDtoday [pdf]
A 2010 NSA SIDtoday op-ed by a Chinese language analyst urging analysts to actively document and share their target knowledge through available tools rather than hoarding expertise in personal files or spreadsheets, arguing that the agency's increasing reliance on short tours and rotating assignments makes institutional knowledge-sharing an operational necessity.
-
SIGINT Mission Thread 3 SIDtoday [pdf]
A 2011 NSA SIDtoday article outlining SIGINT Mission Thread 3's strategic initiative to shift the SIGINT Directorate's analytical culture away from data production toward discovery and understanding, introducing tools like the Analytic Body of Knowledge, Analytics Taxonomy, and GAMBIT portal to systematize tradecraft sharing across the Five Eyes enterprise.
Programs: GAMBITPeople: Gladwell, Malcolm -
ELINT Analysts - Overcome by Overload? Help is Here with IM&S SIDtoday [pdf]
A 2011 NSA SIDtoday article announcing the Initial Operational Capability of the ELINT Modernization Information Management and Storage system, designed to give analysts worldwide unified access to five years of ELINT reports and raw data from an expanding range of traditional and non-traditional collection platforms.
-
"Signal v. Noise" Column: Summit Fever SIDtoday [pdf]
A 2012 NSA opinion column in the "Signal v. Noise" series using the mountaineering concept of "summit fever" — the dangerous fixation on reaching a goal despite worsening conditions — as a metaphor for the institutional tendency within SIGINT to continue pursuing failing projects, collection sites, or targets long past the point where objective data would justify stopping.
-
The SIGINT Philosopher: Too Many Choices SIDtoday [pdf]
The inaugural 2011 NSA SIDtoday "SIGINT Philosopher" column by a Russian language analyst, using the everyday paralysis of grocery shopping to introduce the concept of "analysis paralysis" and argue that the SIGINT enterprise's greatest challenge is not a lack of access or information but the cognitive difficulty of prioritizing among an overwhelming abundance of choices.
People: Simon, Herbert
Related Documents
-
The SIGINT Philosopher: Too Many Choices SIDtoday [pdf]
The inaugural 2011 NSA SIDtoday "SIGINT Philosopher" column by a Russian language analyst, using the everyday paralysis of grocery shopping to introduce the concept of "analysis paralysis" and argue that the SIGINT enterprise's greatest challenge is not a lack of access or information but the cognitive difficulty of prioritizing among an overwhelming abundance of choices.
People: Simon, Herbert -
The SIGINT Philosopher: Descartes Would Have Been a Lousy SIGINT Reporter SIDtoday [pdf]
A 2012 SIDtoday SIGINT Philosopher column using Cartesian epistemology to examine the fundamental reliability problem in SIGINT reporting — arguing that by the time raw intercepts pass through collection, translation, interpretation, and reporting, the finished intelligence product is several steps removed from ground truth and risks becoming little more than an analyst's painting of a scene described to them secondhand.
Programs: SCQAWK -
The SIGINT Philosopher Is Back -- with a New Face! SIDtoday [pdf]
The inaugural column of the second SIGINT Philosopher series, in which the new columnist uses a personal polygraph experience and the ethical tension of spying on diplomatic targets to argue — somewhat counterintuitively — that thorough, comprehensive surveillance is more ethical than partial or halfhearted monitoring of non-adversarial targets.
Programs: SCQAWK -
The SIGINT Philosopher: Unlike All My Terrible Teammates, I Am a SIDtoday [pdf]
A 2012 SIDtoday SIGINT Philosopher column using the annual ACE performance review process as a springboard to examine the structural tension between individual competition and collaborative teamwork at NSA, arguing that the agency's promotion and awards system inadvertently incentivizes analysts to hoard work rather than mentor colleagues or enable others.
Programs: SCQAWK -
The SIGINT Philosopher: When Brevity Is Just the Soul of "Huh?" SIDtoday [pdf]
A 2012 SIDtoday SIGINT Philosopher column arguing that the NSA culture of brevity in writing — while generally valuable — can be taken too far, with excessively terse communications creating confusion, follow-up cycles, and ultimately more lost time than a slightly more complete message would have cost in the first place.
Marking: UNCLASSIFIEDPrograms: SCQAWK