Program: BLACKHOLE 19 documents
Search documents matching “BLACKHOLE” →Codenames and programs recorded for each document.
Graphs
By Banner
By Classification
Refine (combine with another filter, via search)
Sort: Title
Documents
-
QFD BLACKHOLE Technology Behind INOC [pdf]
A UK SECRET STRAP2 COMINT ORCON GCHQ Applied Research presentation from March 2009 documenting the BLACK HOLE bulk data repository's scale and structure, and the suite of analytical tools — MUTANT BROTH, KARMA POLICE, INFINITE MONKEYS and MARBLED GECKO — built on top of it to answer specific intelligence questions about online presence, browsing activity, forum usage and geolocation searches.
Programs: BLACKHOLE · MESME · NETEZZA · CLOUDS · MUTANTBROTH · KARMAPOLICE · INFINITEMONKEYS · MARBLEDGECKO · GOOB · AUTOASSOC · PSOUP · SOCIALANIMAL · SAMUELPEPYS · MEMORYHOLE · HRMAP · PSOUPALERT · LAUGHINGHYENA -
BLACK HOLE ANALYTICS, ADD/SD briefing, September 2009 [pdf]
This September 2009 GCHQ briefing describes the BLACK HOLE analytics infrastructure and Next Generation Events (NGE) project, which planned to scale from ~8 10-gigabit connections to ~200 10Gs by March 2011, storing intercepted data in massive flat-file repositories at Benhall and Bude before loading it into "Question Focused Dataset" (QFD) databases that power surveillance tools like MUTANT BROTH, KARMA POLICE, INFINITE MONKEYS, and others. The document outlines how "probes" extract Target Detection Identifiers (TDIs) from intercepted data feeds, store them in BLACK HOLE repositories, then enable analysts to profile targets' online activities, create social networks, investigate websites and forums, track web searches and Google Earth usage, and perform large-scale data mining across internet and telephony data.
Programs: BLACKHOLE · MUTANTBROTH · AUTOASSOC · INFINITEMONKEYS · KARMAPOLICE · SOCIALANIMAL · HRMAP · MEMORYHOLE · MARBLEDGECKO · HARDASSOC · SAMUELPEPYS · BLAZINGSADDLES · TELLURIAN · BUFFER · DISTILLERY · LOOKINGGLASS · NGEPeople: bin Laden, Osama -
Blazing Saddles [pdf]
This document describes BLAZING SADDLES, a GCHQ program designed to massively scale the storage and analysis of bulk internet surveillance data through multiple specialized databases that track website visits, online identities, communications patterns, and geolocation queries across the global internet population.
Programs: BLAZINGSADDLES · BLACKHOLE · SOCIALANTHROPOID · AUTOASSOC · HRMAP · INFINITEMONKEYS · KARMAPOLICE · MARBLEDGECKO · MUTANTBROTH · MEMORYHOLE · SAMUELPEPYS · SOCIALANIMAL · HAUSTORIUM · GOLDENAXE · NGE -
Broadcast/Internet Radio Exploitation and Analysis, 6th November 2009, GCHQ [pdf]
This GCHQ report analyzes 6.68 million internet radio listening events collected over three months in 2009, detailing global broadcasting patterns, user behaviors, and specifically examining Islamic radio stations and Pakistan-UK connections for potential radicalization intelligence, while recommending integration with bulk surveillance databases like KARMA POLICE for enhanced user profiling.
-
Data Stored in BLACK HOLE [pdf]
This document describes BLACK HOLE, a TOP SECRET data repository that stores approximately 200GB per day of internet activity "events" such as webmail, chat, and browsing. It details the system's various datamarts, including MUTANT BROTH and KARMA POLICE, which collectively hold terabytes of data for intelligence analysis.
Programs: BLACKHOLE · B13B · MUTANTBROTH · KARMAPOLICE · MEMORYHOLE · AUTOASSOC · SOCIALANIMAL · HRMAP · INFINITEMONKEYS -
Demystifying NGE Rock Ridge [pdf]
This presentation outlines the ROCK RIDGE project, which delivers a suite of "Question Focused Databases" (QFDs) like MUTANT BROTH and KARMA POLICE to analyze massive internet event data. These specialized databases are designed to answer specific intelligence questions and are developed using the BLACK HOLE data repository, a key component of the ROUGH DIAMOND program.
Programs: ROCKRIDGE · ROUGHDIAMONG · BLACKHOLE · HAUSTORIUM · HRMAP · MUTANTBROTH · KARMAPOLICE · GOOBZS · INFINITEMONKEYS · AUTOASSOC · SOCIALANIMAL · MARBLEDGECKO · MEMORYHOLE · SAMUELPEPYS · SALAMANCA · MONOPOLY · RADIUS · HARBOURPILOT · LOOKINGGLASS · FIRESTORM · SUPERDRAKE · BLAZINGSADDLES · BLUESHIFT · SUPPORTINGINO · TERRAINS · GEOFUSION · NGEPeople: Lobban, Iain -
Exceptionally Controlled Information Compartments (January 2013) [pdf]
This document, dated January 2013, provides a list of Exceptionally Controlled Information (ECI) compartments utilized by the NSA and its Five Eyes partners. This document categorizes highly sensitive intelligence programs by their codenames, three-letter trigraphs, and the specific organizations responsible for them. The primary organizations listed include TAO, NCSC, and CES. These compartments represent the highest level of information security, designed to restrict access to sensitive technical data and specific intelligence sources even among individuals with high-level security clearances.
Programs: AMBULANT · APERIODIC · AUNTIE · BLACKANT · BLACKAXE · BLACKBALL · BLACKCLOUD · BLACKHOLE · BLACKJACK · BLACKTIE · BLACKVULTURE · BLACKWIDOW · BOXWOOD · CELESTIALGLOBE · CHIEFDOM · CLICKUMBER · CRIMSONREGENT · CRIMSTONSTEAL · CRUMPET · DECKSTOP · DEVILFISH · DIXIESPRING · DOMINATE · FASTIDIOUS · FIRESCREEN · FIRSTDOWN · FISSURESALUTE · FLYLEAF · FORBIDDEN · FORBORNE · FOXHEAT · FREELUNCH · FROTHYTWOPACK · FURTIVERELIANCE · GOLDENCALF · HELLFIRE · HISTORY · HYSSOP · INVEIGH · KESSELRUN · LIGHTNINGTHIEF · LITTLECROWN · LONGSERPENT · MAGICSTROKE · MERCURYTAO · MUSICBOX · NITEHAWK · OCELLUS · PAINTEDEAGLE · PAWLEYS · PAWNSHOP · PENDLETON · PENDRAGON · PERKYAUTUMN · PHENYLDOUR · PICARESQUE · PICAROON · PIEDMONT · PITCHFORD · PLACEBO · POMPANO · PRESSURETWIN · REDHARVEST · REEFPOINT · REFRACTOR · REVELRY · RIVERROAD · RUBIOUS · SAILWINDS · SCABBARD · SILVERCLOUD · SINKGOAL · STAIRWELL · STARCHART · SUITESWIVEL · TRICKSHOT · WAXOFF · WHIPGENIE -
Full-Spectrum Cyber Effects [pdf]
The leaked 2012 GCHQ JTRIG “Full-Spectrum Cyber Effects” presentation outlines the UK’s offensive cyber capabilities, detailing strategies to destroy, deny, degrade, disrupt, deceive, and influence targets across digital and physical domains. It describes a wide range of operations—from network attacks, spoofing, and psychological operations on social media to hotel surveillance (Royal Concierge), mobile tracking, and advanced vulnerability exploitation—demonstrating how GCHQ integrates cyber operations, intelligence, and information warfare on a global scale.
Programs: JTRIG · SALAMANCA · RADIUS · ROYAL CONCIERGE · BLACKHOLE · GUILTY SPARK · MIDDLESEX GREEN · PPF · HOTWIRE -
GCHQ Analytic Cloud Challenges [pdf]
The 2012 GCHQ Analytic Cloud document outlines a sophisticated data processing infrastructure handling approximately 50 billion daily events across multiple sites, utilizing QFD databases and HADOOP clusters for various analytic capabilities targeting intelligence gathering and analysis. It describes architectural components, challenges in scaling and enabling analysts to cope with data complexity, and proposes future technology options including consolidation approaches and new cloud architectures to enhance capabilities.
Programs: EVOLVEDMUTANTBROTH · MUTANTBROTH · HARDASSOC · HRMAP · KARMAPOLICE · SOCIALANTHROPOID · AUTOASSOC · GEOFUSION · MVR · TERRAIN · SALAMANCA · BLACKHOLE · SAMUELPEPYS · BROKER · VAIL · GUIDINGLIGHT · FIRSTCONTACT · CLOUDYCOBRA · RUMOURMILL · EVERYASSOC · EVERYPOLICE · EVERYCREATURE · EVERYEAD · EVERYCIPHER · GREYFOX · SILVERFOX · GOLDENAXE · TRIBALCARNEM · PUBLICANEMONE · EPICFAIL · STERLINGMOTH · FOGHORN · LOOKINGGLASS · FIREENGINE · NGGEO · ALPHACENTAURI · CLOUDBASE · ACCUMULO · ASEM · SEM · HAKIM · STREAMING · HBASE · LAUREL · SKINNYCLOUD · HARDY · GCHQCORE -
HIMR Data Mining Research Problem Book [pdf]
This document is a research roadmap created by GCHQ for the Heilbronn Institute for Mathematical Research (HIMR). Its purpose is to guide long-term mathematical and data-mining research that could improve signals intelligence (SIGINT) capabilities. It outlines key research problems, datasets, and collaboration plans between intelligence agencies and academic researchers. The main goal is to develop new algorithms and mathematical tools to analyze extremely large communications datasets produced by modern surveillance systems.
Programs: CLOUD · DISTILLERY · DISCOVER · PRESTON · LECKWITH · TERRAIN · BROADOAK · NGE · FIVEALIVE · CHARTBREAKER · FIRSTCONTACT · GEOFUSION · RADONSHARPEN-B · PRIMETIME · HIDDENOTTER · BAKERSDOZEN · CLASP · SALTYOTTER · BIRCH · SOLIDINK · FLUIDINK · MAMBA · GRINNINGROACH · PIRATECAREBEAR · AUTOASSOC · CROUCHINGSQUIRREL · HRMAP · KACHINA · SQUEAL · QUESTA · METEORSHOWER · SUNSTORM · GOLDMINE · HAGERAWEL · WOODY · BUZZ · SILVERLIBRARY · SILVERLINING · VALHALLA · MOUNTMCKINLEY · SEPANG · BLACKHOLE · SALAMANCA · HAGELAWEL · LUCKYSTRIKE · SPIKYROCK · MYOFIBRIL · INJUNCTION · PSYCHICSALMON · RAGINGBULLFROG · ROBOTICFISH · TIMIDTOAD · AURA · CARBONCOPY · CASK · CHORDAL · CRAN · INSTINCT · MADFORGE · SWAMP · VOLSUNGA · WHITERAVEN · MVR · PPF · JTRIG · MAPREDUCE · MAILORDERPeople: Tukey, John -
iPhone target analysis and exploitation with unique device identifiers [pdf]
A GCHQ technical paper from November 2010 describing iPhone target analysis and exploitation using the QUANTUM system — covering UDID-based target tracking and correlation across MUTANT BROTH and AUTOASSOC, a Safari PDF exploit delivering WARRIORPRIDE implants to target iPhones via QUANTUM redirection, and a confirmed successful exploitation of a jailbroken target phone yielding address book, SMS, call logs, notes, WiFi logs, browser history and images.
Programs: QUANTUM · DEBITCARD · AUTOASSOC · HARDASSOC · MUTANTBROTH · BROKER · HANDEX · WARRIORPRIDE · ABSOLINEEPILSON · BROADOAK · SOLARSHOCK · BLACKHOLE · SHORTSHEET · XKEYSCORE · MARINA · LOOKINGGLASS · OVERLIT · SLIDE · SEP · PRESTON · WHIPSAW · CROWNPRINCE · CORINTH · MVR · SOCIALANTHROPOID -
Menwith initiatives maximizing our access [pdf]
11 p.This document details Menwith Hill Station’s initiatives to maximize and optimize SIGINT collection, processing, and exploitation, including large-scale monitoring of internet and communications traffic, metadata analysis, and cross-platform target tracking. It describes programs and techniques for surveying, processing, and exploiting signals, enabling analysts to identify, geolocate, and track targets across multiple networks and devices.
Programs: ASPHALT · BLACKTOP · TARMAC · GTE · INDEX · ROADBED · DARKQUEST · EVIL EYE · Spider · SHAREDVISION · QUEST · NOSEYPARKER · XKEYSCORE · GRANDPIANO · FOGHORN · BASSQUEST · APPARITION · MUTANT BROTH · KARMAPOLICE · SOCIALANIMAL · MEMORY HOLE · TURBULENCE · TELLURIAN · MAILORDER · AUTO ASSOC · SLR · BLACKHOLE · JTRIG · PPF · LECKWITH · SCEPTRE · CLOUDPeople: Alexander, Keith B.“GCHQ taps fibre-optic cables for secret access to world's communications” - The Guardian ↗ (2013-06-21) · “Inside Menwith Hill: The NSA’s British Base at the Heart of U.S. Targeted Killing” - The Intercept ↗ (2016-09-06) · MacAskill, Ewen · Borger, Julian · Hopkins, Nick · Davies, Nick · Ball, James · Gallagher, Ryan -
Mobile Apps Checkpoint Meeting Archives [pdf]
This GCHQ document tracks project checkpoints from December 2009 to July 2010, focusing on the rollout and operationalization of mobile apps, TERRAIN system upgrades, GTP/GRX access, and converged event processing. Key themes include coordinating multi-team dependencies, addressing technical gaps in handset and app data characterization, and ensuring timely delivery of milestones like MIP-1.
Programs: TERRAIN · PRESTON · XKEYSCORE · BEGAL · TAMING PASTRIES · MILKWHITE · MVR · SOCIAL ANTHROPOID · INTEGER SPIN · MUTANT BROTH · GEO FUSION · MARBLED GEKO · BLACKHOLE · ACTOR ACTION · TRAFFICMASTER · HARD ASSOC · MIRTF · HEARTBEAT · SILVER LINING · GREENHEART · EREPO · MAGLITE · POKERFACE · BOSTROM · MIDDLESEX GREEN · ENCHANTERSS · HAUSTORIUM · SALAMANCA · BROADOAK · MONKEY PUZZLE · TACHO · EREPO GAMMA · NINJA · PEBBLE BED · GOLDEN EYE · STRAPROC · STARPROC · LOOKING GLASS · SORCERER · SOCIAL ANIMAL · GTE · NGE · PPF · IIB · UDAQ · MES · TERRAINS · CCDF · SINO · GDR · OSDS · LECKWITH -
Mobile apps doubleheader: BADASS Angry Birds [pdf]
A joint GCHQ/GTE/CSEC presentation describing BADASS — the BEGAL Automated Deployment and Survey System for rapidly developing and deploying mobile app exploitation rules — illustrated through detailed analysis of Angry Birds' Dataflurry analytics traffic leaking IMEI, IMSI, cell network metadata and device identifiers, alongside AdMob, Mobclix and Mydas ad network traffic leaking UDIDs and IMEIs across iOS, Android, BlackBerry and Windows Phone 7 platforms.
Programs: BADASS · BEGAL · SEM · UNIQUELYCHALLENGED · BLACKHOLE · MOUNTAINVIEW · VOB · BABELFISH · BISHOP · MVR · PPF · AEG · DATAFLURRY -
NGE BLACK HOLE ConOp [pdf]
This document outlines the "BLACK HOLE" Concept of Operations (ConOp) for the "Next Generation Events" (NGE) project within "BLAZING SADDLES," detailing a flat file data store designed to house various intelligence feeds. Its primary purpose is to serve as a source for Query Focused Datasets (QFDs) and support research, bulk analysis, and target discovery, with two operational instances at Benhall and Bude handling massively increased data volumes.
Programs: NGE · BLAZINGSADDLES · BLACKHOLE · MUTANTBROTH · KARMAPOLICE · SOCIALANIMAL · TELLURIAN · BUFFER · MAILORDER · MARBLEDGECKO · SALAMANCA · SMOKINGSADDLES · CIRCUIT · BLACKFIND · BLACKCAT · DATAFLOWCAB · CHORDAL · DEBIT CARD · B13B -
Report on Architectural Risk 2012 - Summary [pdf]
This 2012 GCHQ report identifies that a historical under-investment in IT architecture has created significant risk, especially with new challenges like massive data growth, cyber defense, and increased partnering. The document recommends immediate, prioritized investment in big data strategies, integrated analytics frameworks, and security services to address the strain on the system.
Programs: BLACKHOLE -
SOCIAL ANTHROPOID [pdf]
A SECRET STRAP1 GCHQ internal training and rollout presentation for SOCIAL ANTHROPOID — a new converged communications database replacing SALAMANCA, SOCIAL ANIMAL and HAUSTORIUM — demonstrating its query interface, data types, filtering capabilities and operational examples including telephony, webmail, instant messaging, BlackBerry, GPRS tunnels and leaky gateway events, with rollout planned across GCHQ analyst divisions in tranches during January-February 2011.
Programs: SOCIALANTHROPOID · SOCIALANIMAL · REFORMER · HAUSTORIUM · SALAMANCA · BROADOAK · DISCOVER · BLACKHOLE · HARDASSOCIATION · CHORDAL -
The Tale of Two Sources [pdf]
Programs: SUPERDRAKE · CROUCHINGSQUIRREL · HALTERHITCH · MVR · PPF · GORDIANKNOT · XKEYSCORE · SAMUELPEPYS · ALPINEBUTTERFLY · TRYST · FRACTALJOKER · TM · SPAY · BLACKHOLE · RAPIDTAPIR · HIDDENOTTER · SALTYOTTER · 8BALL · SORCERER · PACMAN · SNORT · WAFTER · CRIMSON · SHORTFALL · GCNET · BROADOAK · RAPTOR · GEOFUSION · MUGSHOT · HACIENDA · FOXTRAIL · FANNER · GLOBALSURGE · PENSIVEGIRAFFE · HRMAP · MUTANTBROTH · SAMPEPYS · INTEGERSPIN · MWX · MARBLEPOLLS · MOONRAKER · RT · DISTILLERY · SQUEAL · DEADSEA · GTE · WDM · TKB · TRAFFIC MASTER · CHOKEPOINT · CONTENT CLOUD · MAPREDUCE · EVENTS · CLOUD · AGILITY · PALANTIR -
TINT External July 2009 [pdf]
A GCHQ TINT external document from July 2009 describing the BLACKHOLE bulk metadata repository — designed to export unselected metadata in SLR format to enable higher-order analytics — noting anticipated very high volumes expected to exceed MAILORDER transport capacity and proposing a direct connection instead.