Tag: MapReduce 7 documents
Search documents matching “MapReduce” →Free-text subjects the curators added. Unlike the other axes these are not necessarily from the documents' own vocabulary, so they are the loosest thing to browse by (and the most forgiving).
Graphs
By Banner
By Classification
Refine (combine with another filter, via search)
Sort: Title
Documents
-
BOUNDLESSINFORMANT - Describing Mission Capabilities from Metadata Records [pdf]
This presentation on the Boundless Informant program (BOUNDLESSINFORMANT), which uses big data tools to analyze and visualize global SIGINT metadata collection in near real time. It explains how the system replaced older manual methods, enabling the NSA to track collection volumes, coverage, and trends for specific countries or sites through cloud-based infrastructure.
People: Patel, Vimesh“NSA-files repeatedly show collection of data «against countries» - not «from»” - Dagbladet ↗ (2013-11-22) · “Boundless Informant: the NSA's secret tool to track global surveillance data” - The Guardian ↗ (2013-06-08) · “Claves y giros del espionaje masivo en España” - El Mundo ↗ (2013-11-04) · Greenwald, Glenn · MacAskill, Ewen · Aranda, Germán -
BOUNDLESSINFORMANT – Frequently Asked Questions [pdf]
This document outlines the NSA’s internal metadata analysis tool, designed to provide real-time visualizations and statistics on global SIGINT collection by country, organization, and site. It explains the tool’s purpose, data sources, technical architecture, views, data limitations, and the process for requesting feature enhancements, highlighting its role in cataloging surveillance activity without human intervention.
Programs: BOUNDLESSINFORMANT · ASDF · GAO · RAMPART-A · SPINNERET · GM-PLACE · FALLOUT · TUSKATTIRE · FASCIA · MUSCULAR · Terrestrial RF · MachineShop · TurkeyTower · CloudBase · GXT · ExtGWT · JUGGERNAUT · LOPER · FLAWMILL · SSO · MAPREDUCE“Boundless Informant: the NSA's secret tool to track global surveillance data” - The Guardian ↗ (2013-06-08) · “NSA-files repeatedly show collection of data «against countries» - not «from»” - Dagbladet ↗ (2013-11-22) · “New NSA Revelations: Inside Snowden's Germany File” - Der Spiegel ↗ (2014-06-18) · Greenwald, Glenn · MacAskill, Ewen · Becker, Sven · Gude, Hubert · Horchert, Judith · Müller-Maguhn, Andy · Poitras, Laura · Reißmann, Ole · Rosenbach, Marcel · Schindler, Jörg · Schmid, Fidelius · Sontheimer, Michael · Stark, Holger -
ICTR Cloud Efforts [pdf]
A GCHQ presentation details how the agency developed cloud-based big data analytics to achieve "population scale" surveillance—processing trillions of internet events to build geographic and behavioral profiles on every online identifier worldwide, enabling discovery of unknown targets through pattern analysis rather than traditional keyword searches.
Programs: EVERYASSOC · PROBABILITYCLOUD · SILVERLINING · CULTWEAVE · KARMAPOLICE · EVERYPOLICE · AWKWARDTURTLE · GORDIANKNOT · SANDIA · BLOODHOUND · FIVEALIVE · NGE · MAPREDUCE -
MHS Leverages XKS for QUANTUM Against Yahoo and Hotmail [pdf]
This document describes a technical study conducted at Menwith Hill Station (MHS) in the UK. The study used XKEYSCORE (XKS) to evaluate the feasibility of launching QUANTUMTHEORY "man-on-the-side" attacks against users of Yahoo and Hotmail. Analysts developed a specialized Map/Reduce analytic called DRAGGABLEKITTEN to perform deep packet analysis on bulk internet traffic. The study concluded that 90% of Yahoo sessions and 50% of Hotmail sessions contained the specific technical "keywords" (such as unique cookies or headers) within a single data packet required to trigger a successful automated injection of malware.
-
Real-Time Regional Gateway Cloud Analytics for Forward Users [pdf]
This document outlines the NSA’s Real-Time Regional Gateway (RT-RG) program, a global cloud-based signals intelligence system designed to provide near real-time data analysis and targeting support for military and intelligence operations. It details the transition to scalable cloud architectures such as GHOSTMACHINE and Cloudbase, enabling massive data retention, exploitation, advanced geolocation, and automated analytics for tracking high-value targets across multiple theaters, including Afghanistan, Iraq, and the Horn of Africa.
Programs: RT-RG · GOLDMINER · GEOT · AGENTLOGIC · SHARKFINN · SKS · PANOPTICON · DRT · CIDNE · JUGGERNAUT · LOPERS · TALIS · MATTERHORN · AIRHANDLER · LYCANTHROPE · VOICESAIL · RETURNSPRING · SORTINGLEAD · GHOSTMACHINE · GMROYALE · GMREGAL · GMGEIST · GMIDEA · GMTOTE · GMPEN · GMPAC · GMCALM · GMMATE · GMAPPLE · GMCAVE · GMPEACH · GMDEN · GMMOUNT · GMALAMO · GMLIGHT · GMBALTIC · GMPLACE · GMPARK · GMHALO · GMCARBON · DRAGONFIRE · DOMEEXPOSE · GIANTSQUID · MAINWAY · FURIOUSCHAINSAW · SHARKFURY · NIAGARAFILES · SHARKQUERY · SHARKDOCS · OCTAVE · UTT · BEDDOWN · CHEEKYMONKEY · LAYERCAKE · CYBERTRANS · GMSEMINOLE · GMGULF · GMZILLA · HLT · JEMA · NEVIS · SEDB · CLOUDBASE · MAPREDUCE · CASPORT · WAVELEGAL“Mission Creep: How the NSA’s Game-Changing Targeting System Built for Iraq and Afghanistan Ended Up on the Mexico Border” - The Intercept ↗ (2019-05-29) · “Norges hemmelige krig i Afghanistan” - NRK ↗ (2019-05-29) · Moltke, Henrik · Skille, Øyvind Bye · Døvik, Olav · Hansen, Ståle Ellingsgard · Fjørtoft, Martin -
Summary of DNR and DNI Co-Travel Analytics [pdf]
The document outlines the NSA’s development and testing of multiple “co-traveler” analytics—sophisticated tools that analyze bulk mobile location data to identify people who travel together or follow similar patterns, even if they are not known intelligence targets. It details various analytic systems, their data sources (such as FASCIA, VLR, GCID), architectures, scoring methods, and future enhancements, illustrating a large-scale surveillance framework for mapping human movement and relationships.
Programs: CO-TRAVELER · CHALKFUN · TMI · PACT · RT-RG · SORTINGLEAD · FASCIA · Hectic Snare · DRT · Juggernaut · GM-PLACE · Fast Follower · MGSC · ELKPRINTS · VOICESAIL · CULTWEAVE · OCTAVE · OCTSKYWARD · SEDB · JEMA · The Café · UTT · ASDF · GHOSTMACHINE · CLOUDBASE · FASTSCOPE · GOLDMINER · CHET · GEOT · TMAC · SPCMA · MAPREDUCE · ACCUMULO · GAMBIT · ICREACH -
XKeyscore rules [pdf]
XKeyscore fingerprint and plug-in source code targeting Tor and other anonymity tools — the rules NSA used to flag anyone connecting to Tor directory servers, requesting Tor bridges, visiting the Tor Project or Tails websites, or researching the Tails operating system — including code that specifically excludes Five Eyes users while fingerprinting everyone else, and a note describing Tails as "a comsec mechanism advocated by extremists on extremist forums."
“NSA targets the privacy-conscious” - ARD ↗ (2014-07-03) · Appelbaum, Jacob · Gibson, Aaron · Goetz, John · Kabisch, Volkmar · Kampf, Lena · Ryge, Leif