The collection challenge
Open PDFDocument date: -
Pages: 1
Genre: slides
Banner: TOP SECRET//COMINT//REL TO USA, FVEY
Classification: TOP SECRET COMINT
Dissemination: REL TO USA, FVEY
A slide stating the collection problem in one line: “Collection is outpacing our ability to ingest, process and store to the ‘norms’ to which we have become accustomed.”
-
People mentioned:
-
Countries:
-
Tags:
big data, data storage, collection volume, data processing, Five Eyes, SIGDEV
Identifier:
the-collection-challenge
PDF SHA-256:
072e1f2cc4a46241379ea315f1af9f59c27c48c000bd0c82001d5324d7d7d21f
My notes
Published in the Same Report
(10 of 62 documents)
-
NSA Intelligence Relationship with Canada's Communications Security Establishment Canada (CSEC) [pdf]
This document describes the close, long-standing SIGINT partnership between the NSA and Canada’s CSEC, emphasizing cooperative efforts in global intelligence collection, cyber operations, information assurance, and critical infrastructure defense. It highlights mutual contributions, including CSEC’s unique geographic access, covert sites, cryptographic capabilities, and R&D collaboration, supporting U.S. and North American security objectives.
“Communication Security Establishment's cyberwarfare toolbox revealed” - CBC News ↗ (2015-03-23) · “Documents Reveal Canada’s Secret Hacking Tactics” - The Intercept ↗ (2015-03-23) · “No Place to Hide” - Metropolitan Books ↗ (2014-05-13) · Seglins, Dave · Hildebrandt, Amber · Pereira, Michael · Greenwald, Glenn · Gallagher, Ryan -
Close Access SIGADS [pdf]
A 10 September 2010 NSA reference list of Close Access SIGADs revealing the codenames, locations and collection methods used to bug the embassies and UN missions of 38 targets — including US allies France, Italy, Greece, Japan, India, Mexico, South Korea and the EU — through implants, computer-screen capture, hard-drive imaging and electromagnetic emanation collection, most conducted against diplomatic facilities in Washington DC and New York.
Programs: KATEEL · POCOMOKE · MERCED · BANISTER · PERDIDO · MAGOTHY · BLACKFOOT · WABASH · NAVARRO · POWELL · KLONDIKE · NASHUA · OSAGE · OSWAYO · BRUNEAU · HEMLOCK · MULBERRY · ALAMITO · FLEMING · DOBIE · SULPHUR · REQUETTE · YUKON · WESTPORT · NAVAJO · PANTHER · LIFESAVER · HIGHLANDS · VAGRANT · MINERALIZ · DROPMIRE · PBX · MAGNETIC · OCEAN · GENIE · BLACKHEART · CRYPTOENABLED · CUSTOMS · DEWSWEEPER · RADON · MINERALIZE · COOP · STARRUBY · TRASHINT“No Place to Hide” - Metropolitan Books ↗ (2014-05-13) · “Espionnage de la NSA : la diplomatie française était sur écoute aux Etats-Unis” - Le Monde ↗ (2013-10-22) · “Da qui ci spiano gli americani” - L'Espresso ↗ (2013-12-05) · Greenwald, Glenn · Follorou, Jacques · Maurizi, Stefania -
Cyber Integration [pdf]
The leaked 2012 GCHQ JTRIG presentation detail a wide range of offensive cyber and influence operations designed to “deny, disrupt, degrade, and deceive” targets, including hackers, foreign governments, companies, and individuals. Tactics include hacking, DDOS attacks, disinformation, honey traps, credential harvesting, and psychological operations, reflecting a shift from traditional surveillance to active disruption and manipulation campaigns.
-
XKEYSCORE intro [pdf]
A TOP SECRET NSA presentation from 2008 introducing XKeyscore — one of the earliest overview documents in the archive — describing the system as a globally distributed Linux cluster across approximately 150 sites and 700+ servers that stores a ~3-day rolling buffer of all unfiltered internet traffic, and demonstrating its core capabilities for finding previously unknown targets by mining metadata, tracking documents by embedded image hashes, detecting encryption and VPN use, and language tracking.
Programs: XKEYSCORE · TURMOIL · TURBULENCE · TRAFFICTHIEF · WEALTHYCLUSTER · MOONSHINE · SEARCHLIGHT · MARINA · PSC · SSO · TIMBERLINEPeople: bin Laden, Osama“'Brasil é um grande alvo', diz jornalista sobre vigilância dos EUA” - O Globo ↗ (2013-07-07) · “XKeyscore: NSA tool collects 'nearly everything a user does on the internet'” - The Guardian ↗ (2013-07-31) · “No Place to Hide” - Metropolitan Books ↗ (2014-05-13) · “XKEYSCORE: NSA’s Google for the World’s Private Communications” - The Intercept ↗ (2015-07-01) · Greenwald, Glenn · Bridi, Sônia · Marquis-Boire, Morgan · Lee, Micah -
Intelligently filtering your data: Brazil and Mexico case studies [pdf]
These documents reveal that the NSA conducted targeted surveillance operations against high-profile Brazilian and Mexican figures, including President Dilma Rousseff and presidential candidate Enrique Peña Nieto, using advanced contact-graph filtering and data analysis techniques to extract relevant intelligence from intercepted communications. These documents show how NSA analysts systematically developed intelligence on foreign leaders during election periods and diplomatic activities, framing it as “target development,” but in practice enabling deep political espionage under the guise of analytical innovation.
“Veja os documentos ultrassecretos que comprovam espionagem a Dilma” - O Globo ↗ (2013-09-02) · “No Place to Hide” - Metropolitan Books ↗ (2014-05-13) · Greenwald, Glenn -
Sharing Computer Network Operations Cryptologic Information with Foreign Partners [pdf]
The document is a secret NSA/CSS policy outlining guidelines for sharing Computer Network Operations (CNO) cryptologic information with foreign partners, including rules for cooperation in CNE and CND within existing intelligence relationships. The “Purpose and Scope” section on the first page appears to be unpublished elsewhere, while the Tier A and Tier B country lists match those publicly revealed in sources like Glenn Greenwald’s No Place to Hide.
“El CNI facilitó el espionaje masivo de EEUU a España” - El Mundo ↗ (2013-10-30) · “No Place to Hide” - Metropolitan Books ↗ (2014-05-13) · Greenwald, Glenn · Aranda, Germán -
NSA Spied On Porn Habits As Part Of Plan To Discredit 'Radicalizers' [pdf]
Top-secret document reveals how NSA has been gathering records of online sexual activity and evidence of visits to pornographic websites as part of a proposed plan to harm the reputations of those whom the agency believes are radicalizing others through incendiary speeches
-
Discovery SIGINT Targeting Scenarios and Compliance [pdf]
This document is a top-secret NSA guidance manual detailing SIGINT targeting scenarios, compliance procedures, and legal guidance for querying foreign actors, malware, and potentially U.S.-linked systems, including considerations for minimizing incidental collection of U.S. persons’ communications. It outlines specific rules, approvals, and exceptions for using SIGINT tools, foreign intelligence tasks, and cases involving ambiguous or dual-use selectors like WikiLeaks, The Pirate Bay, and foreign actors operating through U.S. infrastructure.
-
Elegant Chaos: collect it all, exploit it all [pdf]
This document outlines a classified project, "Elegant CHAOS," focused on advanced signals intelligence (SIGINT) collection, processing, and analysis using cutting-edge technologies like automated surveys and modems. It highlights goals such as real-time data analysis, collaboration with international partners, and integration with cloud-based platforms to manage massive data. See also the document with notes (by GCHQ): "Elegant-Chaos-collect-it-all-exploit-it-all-plus"
Programs: ELEGANTCHAOS · ASPHALT · ASPHALT-A · ASPHALT-A-PLUS · STORMFORCE · DARKQUEST · TORUS · XKEYSCORE · MHS CLOUD · TURMOIL · TARMAC · ASDF · SLR · POPQUIZ · WEALTHYCLUSTER2 · TRAVELLINGWAVE · BILBOBADGER · DRINKYBIRD · GLOBETROTTER · MASTERSHAKE · REST · TRAFFICTHIEF · VENUSAFFECT · FOGHORN · MATCHMAKER · ROADBED · MOONPENNY · AMULETSTELLAR · CLOUD · WEALTHYCLUSTER · MVR · TINT · IPGEOTRAP · MARINA · PLUS -
FASCIA: The NSA's huge trove of location records [pdf]
FASCIA is the National Security Agency's enormous database containing trillions of metadata records, many with device locations, that are collected from a variety of sources. This document shows the volume and types of device-location data collected by the agency.
Related Documents
-
BOUNDLESSINFORMANT - Describing Mission Capabilities from Metadata Records [pdf]
This presentation on the Boundless Informant program (BOUNDLESSINFORMANT), which uses big data tools to analyze and visualize global SIGINT metadata collection in near real time. It explains how the system replaced older manual methods, enabling the NSA to track collection volumes, coverage, and trends for specific countries or sites through cloud-based infrastructure.
People: Patel, Vimesh“NSA-files repeatedly show collection of data «against countries» - not «from»” - Dagbladet ↗ (2013-11-22) · “Boundless Informant: the NSA's secret tool to track global surveillance data” - The Guardian ↗ (2013-06-08) · “Claves y giros del espionaje masivo en España” - El Mundo ↗ (2013-11-04) · Greenwald, Glenn · MacAskill, Ewen · Aranda, Germán -
Tracking Targets on Online Social Networks [pdf]
A TOP SECRET NSA presentation from September 2009 describing XKeyscore techniques for tracking targets across online social networks — covering OSN selector types, six categories of target OSN activity ranked by intelligence value, username and HTTP activity query methods, and specific XKeyscore AppIDs for Facebook and Twitter — noting that counterterrorism targets had been observed using over 50 social networks as of late 2008.
Programs: XKEYSCORE“XKEYSCORE: NSA’s Google for the World’s Private Communications” - The Intercept ↗ (2015-07-01) · “No Place to Hide” - Metropolitan Books ↗ (2014-05-13) · Marquis-Boire, Morgan · Greenwald, Glenn · Lee, Micah -
User-agents: Why and How and How to [pdf]
A TOP SECRET NSA training presentation from 2010 teaching analysts how to use browser user-agent strings — the identifying information your web browser automatically sends to every website you visit — to track surveillance targets across the internet, link their known email activity to their anonymous web browsing, and identify opportunities for hacking their computers based on their operating system and software versions.
-
Tor Stinks [pdf]
A June 2012 joint NSA-GCHQ presentation titled "Tor Stinks" detailing the two agencies' attempts to de-anonymize Tor users through analytics, exploitation, and network disruption — candidly admitting they will never be able to identify all Tor users, but outlining a range of techniques including cookie leakage tracking, QUANTUM attacks, circuit reconstruction, timing analysis, and deliberate network degradation.
Programs: REMATION · EPICFAIL · QUANTUM · NEWTONSCRADLE · QUICKANT · GREATEXPECTATIONS · RONIN · ONIONBREATH · QUANTUMCOOKIE · SHORTSHEET · FOXACID -
SIGINT Development: A Network of Discovery Networks SIDtoday [pdf]
An article describes the mission and recent activities of SIGINT Development, an expanding mission of NSA.
The Intercept ↗ (2016-05-16)