Program: MAPREDUCE 14 documents
Search documents matching “MAPREDUCE” →Codenames and programs recorded for each document.
Graphs
By Banner
By Classification
Refine (combine with another filter, via search)
Sort: Title
Documents
-
BOUNDLESSINFORMANT - Describing Mission Capabilities from Metadata Records [pdf]
This presentation on the Boundless Informant program (BOUNDLESSINFORMANT), which uses big data tools to analyze and visualize global SIGINT metadata collection in near real time. It explains how the system replaced older manual methods, enabling the NSA to track collection volumes, coverage, and trends for specific countries or sites through cloud-based infrastructure.
People: Patel, Vimesh“NSA-files repeatedly show collection of data «against countries» - not «from»” - Dagbladet ↗ (2013-11-22) · “Boundless Informant: the NSA's secret tool to track global surveillance data” - The Guardian ↗ (2013-06-08) · “Claves y giros del espionaje masivo en España” - El Mundo ↗ (2013-11-04) · Greenwald, Glenn · MacAskill, Ewen · Aranda, Germán -
BOUNDLESSINFORMANT – Frequently Asked Questions [pdf]
This document outlines the NSA’s internal metadata analysis tool, designed to provide real-time visualizations and statistics on global SIGINT collection by country, organization, and site. It explains the tool’s purpose, data sources, technical architecture, views, data limitations, and the process for requesting feature enhancements, highlighting its role in cataloging surveillance activity without human intervention.
Programs: BOUNDLESSINFORMANT · ASDF · GAO · RAMPART-A · SPINNERET · GM-PLACE · FALLOUT · TUSKATTIRE · FASCIA · MUSCULAR · Terrestrial RF · MachineShop · TurkeyTower · CloudBase · GXT · ExtGWT · JUGGERNAUT · LOPER · FLAWMILL · SSO · MAPREDUCE“Boundless Informant: the NSA's secret tool to track global surveillance data” - The Guardian ↗ (2013-06-08) · “NSA-files repeatedly show collection of data «against countries» - not «from»” - Dagbladet ↗ (2013-11-22) · “New NSA Revelations: Inside Snowden's Germany File” - Der Spiegel ↗ (2014-06-18) · Greenwald, Glenn · MacAskill, Ewen · Becker, Sven · Gude, Hubert · Horchert, Judith · Müller-Maguhn, Andy · Poitras, Laura · Reißmann, Ole · Rosenbach, Marcel · Schindler, Jörg · Schmid, Fidelius · Sontheimer, Michael · Stark, Holger -
HIMR Data Mining Research Problem Book [pdf]
This document is a research roadmap created by GCHQ for the Heilbronn Institute for Mathematical Research (HIMR). Its purpose is to guide long-term mathematical and data-mining research that could improve signals intelligence (SIGINT) capabilities. It outlines key research problems, datasets, and collaboration plans between intelligence agencies and academic researchers. The main goal is to develop new algorithms and mathematical tools to analyze extremely large communications datasets produced by modern surveillance systems.
Programs: CLOUD · DISTILLERY · DISCOVER · PRESTON · LECKWITH · TERRAIN · BROADOAK · NGE · FIVEALIVE · CHARTBREAKER · FIRSTCONTACT · GEOFUSION · RADONSHARPEN-B · PRIMETIME · HIDDENOTTER · BAKERSDOZEN · CLASP · SALTYOTTER · BIRCH · SOLIDINK · FLUIDINK · MAMBA · GRINNINGROACH · PIRATECAREBEAR · AUTOASSOC · CROUCHINGSQUIRREL · HRMAP · KACHINA · SQUEAL · QUESTA · METEORSHOWER · SUNSTORM · GOLDMINE · HAGERAWEL · WOODY · BUZZ · SILVERLIBRARY · SILVERLINING · VALHALLA · MOUNTMCKINLEY · SEPANG · BLACKHOLE · SALAMANCA · HAGELAWEL · LUCKYSTRIKE · SPIKYROCK · MYOFIBRIL · INJUNCTION · PSYCHICSALMON · RAGINGBULLFROG · ROBOTICFISH · TIMIDTOAD · AURA · CARBONCOPY · CASK · CHORDAL · CRAN · INSTINCT · MADFORGE · SWAMP · VOLSUNGA · WHITERAVEN · MVR · PPF · JTRIG · MAPREDUCE · MAILORDERPeople: Tukey, John -
ICTR Cloud Efforts [pdf]
A GCHQ presentation details how the agency developed cloud-based big data analytics to achieve "population scale" surveillance—processing trillions of internet events to build geographic and behavioral profiles on every online identifier worldwide, enabling discovery of unknown targets through pattern analysis rather than traditional keyword searches.
Programs: EVERYASSOC · PROBABILITYCLOUD · SILVERLINING · CULTWEAVE · KARMAPOLICE · EVERYPOLICE · AWKWARDTURTLE · GORDIANKNOT · SANDIA · BLOODHOUND · FIVEALIVE · NGE · MAPREDUCE -
LONGHAUL [pdf]
An NSA WikiInfo page for LONGHAUL — the centralised end-to-end attack orchestration and key recovery service that sits at the heart of NSA's capability to break encrypted internet traffic, receiving cryptanalytic data from field collection systems, coordinating attacks against VPN and SSL/TLS encryption through child projects POISONNUT and SCARLETFEVER, and returning recovered keys to deployed CA Servers at collection sites worldwide.
Programs: LONGHAUL · MAILORDER · CASPORT · GECC · VIVIDDREAM · TURMOIL · WEALTHYCLUSTER · TUTELAGE · EINSTEIN · ISLANDTRANSPORT · HADOOP · NINJACOVE · JST · CST · VMM · VDG · ORDC · SCB · ISLANDHIDEAWAY · TURBULENCE · POISONNUT · SCARLETFEVER · HEROICSAND · PLATINUMRING · GALLANTWAVE · JEDIMINDTRICK · CHAOTICSTORMTROOPER · CORALREEF · SCUBADIVER2 · TERRACOTTA · HBASE · SCALA · MAPREDUCE · COALCAR · STOCKCAR · BEACHHEAD · MIDRIFT · WOLVERINESTAKE · NSP · PURGE · SURGE · C2DP · DOORS · HUDSON -
Real-Time Regional Gateway Cloud Analytics for Forward Users [pdf]
This document outlines the NSA’s Real-Time Regional Gateway (RT-RG) program, a global cloud-based signals intelligence system designed to provide near real-time data analysis and targeting support for military and intelligence operations. It details the transition to scalable cloud architectures such as GHOSTMACHINE and Cloudbase, enabling massive data retention, exploitation, advanced geolocation, and automated analytics for tracking high-value targets across multiple theaters, including Afghanistan, Iraq, and the Horn of Africa.
Programs: RT-RG · GOLDMINER · GEOT · AGENTLOGIC · SHARKFINN · SKS · PANOPTICON · DRT · CIDNE · JUGGERNAUT · LOPERS · TALIS · MATTERHORN · AIRHANDLER · LYCANTHROPE · VOICESAIL · RETURNSPRING · SORTINGLEAD · GHOSTMACHINE · GMROYALE · GMREGAL · GMGEIST · GMIDEA · GMTOTE · GMPEN · GMPAC · GMCALM · GMMATE · GMAPPLE · GMCAVE · GMPEACH · GMDEN · GMMOUNT · GMALAMO · GMLIGHT · GMBALTIC · GMPLACE · GMPARK · GMHALO · GMCARBON · DRAGONFIRE · DOMEEXPOSE · GIANTSQUID · MAINWAY · FURIOUSCHAINSAW · SHARKFURY · NIAGARAFILES · SHARKQUERY · SHARKDOCS · OCTAVE · UTT · BEDDOWN · CHEEKYMONKEY · LAYERCAKE · CYBERTRANS · GMSEMINOLE · GMGULF · GMZILLA · HLT · JEMA · NEVIS · SEDB · CLOUDBASE · MAPREDUCE · CASPORT · WAVELEGAL“Mission Creep: How the NSA’s Game-Changing Targeting System Built for Iraq and Afghanistan Ended Up on the Mexico Border” - The Intercept ↗ (2019-05-29) · “Norges hemmelige krig i Afghanistan” - NRK ↗ (2019-05-29) · Moltke, Henrik · Skille, Øyvind Bye · Døvik, Olav · Hansen, Ståle Ellingsgard · Fjørtoft, Martin -
Running Strategic Analytics Affecting Europe and Africa [pdf]
This Top Secret presentation from the NSA ECC (European Cryptologic Center) outlines the strategic challenges and technical methodology for conducting signal intelligence operations across Europe and Africa. It emphasizes a critical reliance on "Second Party" partnerships, particularly with the UK's GCHQ, and details the use of mass surveillance tools like XKEYSCORE to track terrorism, smuggling, and human trafficking networks.
Programs: XKEYSCORE · RTRG · DISTILLERY · CLOUD · GHOSTMACHINE · NIAGARAFILES · WHIZBANG · MAPREDUCE -
SKYNET: Courier Detection via Machine Learning [pdf]
A June 2012 NSA research presentation detailing the machine learning methodology behind SKYNET's courier detection system, which applies a Random Forest classifier to 55 million Pakistani phone records to identify individuals exhibiting courier-like behavioral patterns — with one of the highest-scoring results being Ahmed Zaidan, Al Jazeera journalist flagged as a suspected al-Qaeda courier.
People: Zaidan, Ahmad Muaffaq -
SPINALTAP, Tracking Courier Use of Secure Digital Cards, and QUANTUMFALCON [pdf]
Three NSA presentations published as a single file. SPINALTAP, from Menwith Hill Station, extracts selectors from TAO, SFC and GCHQ endpoint accesses and translates them into XKEYSCORE fingerprints so that machines held by active operations can be recognised again in passive collection. Tracking Courier Use of Secure Digital Cards, from the NSA SIGINT Forensics Center, uses volume serial numbers and volume labels to identify SD cards and USB media that carry no USB controller, and reports identification information for 36 devices not seized during the UBL raid. QUANTUMFALCON summarises selector activity to triage targets for QUANTUM operations: see the Der Spiegel article The Digital Arms Race: NSA Preps America for Future Battle, 17 January 2015.
Programs: AARDVARKSTAKE · ABSOLINEDELTA · ACRIDMINI · AFTERBOOTSOLE · AFTERCLIFFDIVE · AFTERDOGHOUSE · AFTERGASSTATION · AFTERLASTTEAM · AFTERRICHGEAR · AFTERSHORTRUN · AFTERTANKERTRUCK · AFTERTREEFORM · AFTERWAYBACK · AFTERWINDBLOWN · AFTERYARDARM · ANCIENTBREW · APACHERIVER · ARMOREDCONDOR · ASDF · ATOMICCANNON · ATOMICFIREBALL · ATOMICFOG · ATOMICMONKEY · ATOMICPUNCH · ATOMICSTRIKE · AZTECTOMB · BACKSNARF · BEDOUINSTRIKE · BEEFCAKE · BLACK ALCHEMY · BLACKAMETHYST · BLACKMESA · BLOODDIAMOND · BROKENTHOUGHT · BULLETTOOTH · CAFFEINECRASH · CHOCOLATESHIP · COBALTGUPPY · COCOAMELTDOWN · CRISPWARE · CRYPTICSENTINEL · CUDDLYBADGER · CYGNUSOLOR · DANDERSPRITZ · DARKFIRE · DARKHELMET · DARKINTENT · DARKRAVEN · DARKRAZOR · DARKSCREW · DARKTHUNDER · DEADDRUMMER · DEPUTYSHIP · DETASSELJANICE · DIRTDIVER · DISTORTAFFECT · DOUBLETAP · DRINKMINT · DRINKMINTAA · DRUMBEAT · EDITIONHAZE · EFFABLELAMBDA · ELECTRONSWORD · EMPTYMOCHA · FIREBRUSH · FIREEATER · FIRESWAMP · FOXACID · FOXBASE · FRANTICDANCER · FREEACIDRAIN · FREEAIRFARE · FREEARCADEZONE · FREEBACKGAMMON · FREEBADFIBER · FREEBADRENT · FREEBALLROOM · FREEBATTLEZONE · FREEBEACHTREE · FREEBIGBOSS · FREEBITTERCLOUD · FREEBLACKCLOUD · FREEBLOODYWOLF · FREEBLOWNTURBO · FREEBLUEMAT · FREEBRASSBRUSH · FREEBUTTERCLOUD · UTT · FREECANALLOCK · FREECANESUGAR · FREECATBOX · FREECEMENTBLOCK · FREECHERRYCOLA · FREECHESSBOARD · FREECLEARTAPE · FREECOLDTEA · FREECORNHUSK · FREECORNMAZE · FREECREEKMOOR · FREECRUSHEDDISK · FREEDARKSUIT · FREEDATALOSS · FREEDEADBATTERY · FREEDETOURSIGN · FREEDIRTYTRICK · FREEDISCOVERY · FREEDISKBRAKE · FREEDOGCRATE · FREEDOMECUPOLA · FREEDOVETAIL · FREEEMUFARM · FREEENERGYTAX · FREEFAMILYTIE · FREEFASTCAR · FREEFIBERBOARD · FREEFILEDELETE · FREEFLATFIBER · FREEFLOWCHART · FREEFLOWERPEOPLE · FREEFRIEZEFRESCO · FREEGEMSTONE · FREEGLASSTUBE · FREEGLUESTRIP · FREEHAVEFUN · FREEHOMEBASE · FREEHOOKHANDLE · FREEHOOPDREAM · FREEJETFUEL · FREEKIDPOOL · FREEKINGSPAWN · FREEKNOCKOUT · FREELANDLINE · FREELEADSHOT · FREELEADSINGER · FREELIFERAFT · FREELIKESAME · FREELINEDOWN · FREELOLLYPOP · FREEMARBLEBASIN · FREEMETALCRATE · FREEMETALFILE · FREEMETALSHARD · FREEMINETUNNEL · FREEMINTJELLY · FREENAVYBLUE · FREENIGHTTRAIN · FREEOBLIQUECASE · FREEOILLEAK · FREEOILPAINT · FREEOLDBIKE · FREEOUTRUN · FREEPAINTBALL · FREEPICKLEBRINE · FREEPINEPLANK · FREEPLASTICCASE · FREEPONGPLAYER · FREEPOSTMARK · FREEPOWERFAILURE · FREEPUFFYCLOUD · FREEPULLCHAIN · FREERAINCLOUD · FREERAVENTICKET · FREEREDBEER · FREEREDERASER · FREEREDMARKER · FREEREDSHIRT · FREEREDSTAIN · FREERIDEAROUND · FREERIGHTWHALE · FREERIPPINGBLADE · FREEROCKSONG · FREESAFEKEY · FREESALTTRUCK · FREESASHCORD · FREESCHOOLLOCKER · FREESCREENDOOR · FREESEADADDY · FREESHORTCARD · FREESHORTPASS · FREESINEWAVE · FREESLOWFAST · FREESMALLSPACE · FREESMOKESCREEN · FREESNOWCLOUD · FREESNOWSHOVEL · FREESPACEFLIGHT · FREESPEEDTRAP · FREESTATEWARD · FREESTONESHIP · FREESTORAGEROOM · FREETANKSTAND · FREETESTSHEET · FREETHUNDERCLOUD · FREETICKETBOOTH · FREETIMELEGEND · FREETIMESHARE · FREETINYTANK · FREETRICKYKICK · FREETROUTSTREAM · FREETRUEPINBALL · FREETWINBEE · FREEVINYLMESH · FREEWARRIORPAINT · FREEWATERBED · FREEWATERGLASS · FREEWATERTANK · FREEWATERTOWER · FREEWAVECREST · FREEWAYPOINT · FREEWHEELCOVER · FREEWHEELNUT · FREEWINDCLOUD · FREEWINDSHEAR · FREEWOODENSTICK · FURRYEWOK · GHOSTMACHINE · GMPLACE · GOODMONKEY · HAMMERBROTHERS · HASTYCOBRA · HORSEWRAP · ICEBLOCK · IMPUREHOLSTER · INDEPENDENCEPIE · INQUIRY · JAVAFRESCO · JEALOUSJOKER · JEEPFLEA · JEEPFLEAMARKET · JOLLYROGER · KIDSHIPAA · KOOPATROOPA · KUKRISTEEL · LIQUIDSTEEL · LUTEUSASTRO · MAGNUMOPUS · MAGNUMOPUSCC · MARINA · MAXRANKLE · MICEFUR · MIDNIGHTSCORPION · MILKSTEAK · MIRACLEMAX · MUSHROOMKINGDOM · NAPALAN · NATIVEFLORA · OBSCUREBLAZE · OFFICELINEBACKER · OFFICEQUARTERBACK · OPTIMUSPRIME · PARLAYBUFFET · PHANTOMSTARFISH · PLUMREVOLVER · PRETZELDOG · QBISCUIT · QDIRK · QUANTUM · QUANTUMBISCUIT · QUANTUMDIRK · QUANTUMFALCON · REACTOR · ROLLEDHAT · SANDPALACE · SCARFSLOOP · SHADYNINJA · SHAKEWEIGHT · SHATTEREDSHIELD · SILENTTONGUES · SILLYBUNNY · SILVERJUMP · SKYJACKBRAD · SLYNINJA · SLYSNOW · SLYWIZARD · SNAPKEY · SPARTANFURY · SPIKEYFARM · SPINALTAP · STARPROC · STEELSKY · STEELSKYDELTA · STEELSKYECHO · STEELSKYFOXTROT · STEELSKYGOLF · STONEHENGE · STRAITLACED · SWITCHDOWN · SWITCHDOWNIRAW · SWITCHDOWNIRBR · SWITCHDOWNIRCD · THIEVESQUARTER · TOADYTEAL · TOTALDAGGER · TOXICSNOW · TROPICALSTORM · UMBRAGESPIDER · UNITEDRAKE · UPPERMUTANT · VALIDATOR · VEILEDMAGIC · WATERCASKET · WATERWINGS · WAXCHIP · WICKEDAMP · WILDCHOCOBO · WITHEREDFRUIT · WOLFACID · WOLFACIDANISE · WOLFACIDARGON · WOLFACIDBARIUM · WOLFACIDCHILI · WOLFACIDIODINE · WOLFACIDIRON · WOLFACIDJUPITER · WOLFACIDLEAD · WOLFACIDPRECIOUS · WOLFACIDTIN · WOLFACIDURANIUM · WOLFACIDZINC · XKEYSCORE · YELLOWFAN · MUSCULAR · INCENSER · DANCINGOASIS · MAPREDUCE“Documents Reveal Top NSA Hacking Unit” - Der Spiegel ↗ (2013-12-29) · “How the NSA Plans to Infect ‘Millions’ of Computers with Malware” - The Intercept ↗ (2014-03-12) · “The Digital Arms Race: NSA Preps America for Future Battle” - Der Spiegel ↗ (2015-01-17) · Appelbaum, Jacob · Poitras, Laura · Rosenbach, Marcel · Stöcker, Christian · Schindler, Jörg · Stark, Holger · Gallagher, Ryan · Greenwald, Glenn · Gibson, Aaron · Guarnieri, Claudio · Müller-Maguhn, Andy · Ryge, Leif · Schmundt, Hilmar · Sontheimer, Michael -
Summary of DNR and DNI Co-Travel Analytics [pdf]
The document outlines the NSA’s development and testing of multiple “co-traveler” analytics—sophisticated tools that analyze bulk mobile location data to identify people who travel together or follow similar patterns, even if they are not known intelligence targets. It details various analytic systems, their data sources (such as FASCIA, VLR, GCID), architectures, scoring methods, and future enhancements, illustrating a large-scale surveillance framework for mapping human movement and relationships.
Programs: CO-TRAVELER · CHALKFUN · TMI · PACT · RT-RG · SORTINGLEAD · FASCIA · Hectic Snare · DRT · Juggernaut · GM-PLACE · Fast Follower · MGSC · ELKPRINTS · VOICESAIL · CULTWEAVE · OCTAVE · OCTSKYWARD · SEDB · JEMA · The Café · UTT · ASDF · GHOSTMACHINE · CLOUDBASE · FASTSCOPE · GOLDMINER · CHET · GEOT · TMAC · SPCMA · MAPREDUCE · ACCUMULO · GAMBIT · ICREACH -
"The EDGE" - National Information Assurance Research Laboratory (NIARL) Science, Technology, and Personnel Highlights, 2008 [pdf]
The September 2008 edition of "The EDGE" — the internal newsletter of NSA's National Information Assurance Research Laboratory (NIARL/R2) — covering research highlights including the TUNDRA project investigating a new AES cryptanalysis technique, the TUNDRA Tau statistic for codebook analysis, photonic logic research at Sandia, anti-tamper work, the CLARIFYMIND wireless pilot, FLOWBEE network metadata collection, POPQUIZ/DISTILLERY analytics and FMAC mandatory access controls for OpenSolaris, notable for containing the first published description of the TUNDRA cryptanalysis project against AES.
Programs: DISTILLERY · HAIRBALL · GRAPHVIZ · POPEYESEAR · RANDOMIZERS · TUNDRA · CLOUD · STARFIRE · CARPAT · MAPREDUCE · NUCLEARWINTER · CLARIFYMIND · FLOWBEE · POPQUIZ · BYZANTINEHADES · TREEVIZ · GROUPDIVE · BLUESASH -
The Tale of Two Sources [pdf]
Programs: SUPERDRAKE · CROUCHINGSQUIRREL · HALTERHITCH · MVR · PPF · GORDIANKNOT · XKEYSCORE · SAMUELPEPYS · ALPINEBUTTERFLY · TRYST · FRACTALJOKER · TM · SPAY · BLACKHOLE · RAPIDTAPIR · HIDDENOTTER · SALTYOTTER · 8BALL · SORCERER · PACMAN · SNORT · WAFTER · CRIMSON · SHORTFALL · GCNET · BROADOAK · RAPTOR · GEOFUSION · MUGSHOT · HACIENDA · FOXTRAIL · FANNER · GLOBALSURGE · PENSIVEGIRAFFE · HRMAP · MUTANTBROTH · SAMPEPYS · INTEGERSPIN · MWX · MARBLEPOLLS · MOONRAKER · RT · DISTILLERY · SQUEAL · DEADSEA · GTE · WDM · TKB · TRAFFIC MASTER · CHOKEPOINT · CONTENT CLOUD · MAPREDUCE · EVENTS · CLOUD · AGILITY · PALANTIR -
XKeyscore rules [pdf]
XKeyscore fingerprint and plug-in source code targeting Tor and other anonymity tools — the rules NSA used to flag anyone connecting to Tor directory servers, requesting Tor bridges, visiting the Tor Project or Tails websites, or researching the Tails operating system — including code that specifically excludes Five Eyes users while fingerprinting everyone else, and a note describing Tails as "a comsec mechanism advocated by extremists on extremist forums."
“NSA targets the privacy-conscious” - ARD ↗ (2014-07-03) · Appelbaum, Jacob · Gibson, Aaron · Goetz, John · Kabisch, Volkmar · Kampf, Lena · Ryge, Leif -
XKEYSCORE System Administration [pdf]
A December 2012 TOP SECRET XKeyscore System Administration training manual — the detailed technical guide for the engineers who ran XKeyscore field-site clusters — covering the system's Red Hat Linux architecture, master/slave cluster design, database and processing pipeline, the GUI (including built-in Tor, botnet and CNE search menus), the DeepDive full-take packet processor, the xks command-line tool, and monitoring and troubleshooting, with live screenshots from the Timberline site (USF-790) and Menwith Hill Station.
Programs: XKEYSCORE · WEALTHYCLUSTER · MAILORDER · CADENCE · ROGUESTITCH · QUANTUMBOT · COTTONCHAINSAW · MPTRII · SCARFACE · ZAMENSIS · MAPREDUCE · GENESIS · REDUCER · DEEPDIVE · ASDF · CYBER · BULKCRYPT · METTLESOME · PACKETSPLAYYER · PROMOTER · DEFRAG · XFIP · TRAFFICTHIEF · ENDACE · TIMBERLINE · MUSCULAR · FIST · STARPROC