Events Analysis
Open PDFDocument date: 2008-11-27
Pages: 1
Genre: slides
Classification: -
Dissemination: -
This GCHQ document explains authorization policies for querying metadata databases (SALAMANCA, HAUSTORIUM, THUGGEE, IMMINGLE), stating that metadata is "less intrusive than communications content" and that authorization is "not needed for individuals in the UK," though searches are logged and audited with necessity and proportionality considerations still applying.
SALAMANCA, HAUSTORIUM, THUGGEE, IMMINGLE
People mentioned:
-
Countries:
-
Tags:
surveillance databases, metadata analysis, event analysis, authorization, legal framework, legal loophole, analyst guide, tasking authorization, metadata repositories, surveillance oversight
Identifier:
master-current-20081127
PDF SHA-256:
54562c338674b86b49e8e604d6557d4b0a174b367f59793c5f3e6f0338d820c1
My notes
Events analysis
SALAMANCA, HAUSTORIUM, THUGGEE, IMMINGLE
- less intrusive than communications content
- authorisation not needed for individuals in the UK
- necessity and proportionality still matter
- searches are logged and audited
This information is exempt under the Freedom of Information Act 2000 (FOIA) and may be exempt under other UK information legislation. Refer any FOIA queries to GCHQ on [REDACTED]
Published in the Same Report
(10 of 28 documents)
-
Supporting Internet Operations - Special Source Access [pdf]
In this document GCHQ outlines its plans for expanding internet operations, focusing on increasing access to 10G bearers and enhancing its intercept and egress capabilities by 2011. The agency highlighted that “25 percent of all internet traffic” transits the U.K. through approximately 1,600 different 10-gigabit cables, and it noted that it could “survey the majority of the 1,600” to “select the most valuable to switch into our processing systems.”
-
Event (SIGINT) [pdf]
5 p.This document explains that within GCHQ and NSA SIGINT operations, an “Event” refers to metadata about observable communications or user actions rather than message content, categorized into types like Communications, Presence, Social, and Geo-reference Events. These events are collected, stored in various databases, and analyzed through convergence and correlation processes, with standardized formats defined by Interface Control Documents such as INTERSTELLAR DUST (IDUST) and Actor Action (AA).
Programs: INTERSTELLAR DUST · HAUSTORIUM · SOCIAL ANTHROPOID · ACTOR-ACTION · SAMREF · GEOFUSION · HARD ASSOC · SALAMANCA · ASSOCIATION · BANYAN · MAINWAY · FASCIA · CULTWEAVE II · VOICESAIL · TEEDALE · PILBEAM · MUTANTBROTH · AUTOASSOC · MARBLEDGECKO · KARMAPOLICE · HRMAP · MEMORYHOLE · INFINITEMONKEYS · SOCIALANIMAL · AUTOTDI · SAMUELPEPYS · Mobile Applications Project · NGE · PPF · SLR -
Access: Vision 2013 [pdf]
This July 2010 GCHQ strategic document outlines the agency's vision for 2013 to create "the world's biggest SIGINT engine" with access to 800 10-gigabit connections (up from existing capacity), expanding overseas operations to 60% while reducing UK-based to 40%, and developing new capabilities including computer network exploitation (CNE), exfiltration, beaconry, geolocation, and collection of "Internet Pocket Litter" like cookies and browsing data. The strategy emphasizes "scale and pace," "balance," and "agility" while claiming to operate within legal frameworks of "necessity, proportionality and authorisation," and highlights partnerships with second and third parties to access "intractable and hard targets."
Marking: SECRET -
BLACK HOLE ANALYTICS, ADD/SD briefing, September 2009 [pdf]
This September 2009 GCHQ briefing describes the BLACK HOLE analytics infrastructure and Next Generation Events (NGE) project, which planned to scale from ~8 10-gigabit connections to ~200 10Gs by March 2011, storing intercepted data in massive flat-file repositories at Benhall and Bude before loading it into "Question Focused Dataset" (QFD) databases that power surveillance tools like MUTANT BROTH, KARMA POLICE, INFINITE MONKEYS, and others. The document outlines how "probes" extract Target Detection Identifiers (TDIs) from intercepted data feeds, store them in BLACK HOLE repositories, then enable analysts to profile targets' online activities, create social networks, investigate websites and forums, track web searches and Google Earth usage, and perform large-scale data mining across internet and telephony data.
Programs: BLACKHOLE · MUTANTBROTH · AUTOASSOC · INFINITEMONKEYS · KARMAPOLICE · SOCIALANIMAL · HRMAP · MEMORYHOLE · MARBLEDGECKO · HARDASSOC · SAMUELPEPYS · BLAZINGSADDLES · TELLURIAN · BUFFER · DISTILLERY · LOOKINGGLASS · NGEPeople: bin Laden, Osama -
Report on Architectural Risk 2012 - Summary [pdf]
This 2012 GCHQ report identifies that a historical under-investment in IT architecture has created significant risk, especially with new challenges like massive data growth, cyber defense, and increased partnering. The document recommends immediate, prioritized investment in big data strategies, integrated analytics frameworks, and security services to address the strain on the system.
Programs: BLACKHOLE -
Blazing Saddles [pdf]
This document describes BLAZING SADDLES, a GCHQ program designed to massively scale the storage and analysis of bulk internet surveillance data through multiple specialized databases that track website visits, online identities, communications patterns, and geolocation queries across the global internet population.
Programs: BLAZINGSADDLES · BLACKHOLE · SOCIALANTHROPOID · AUTOASSOC · HRMAP · INFINITEMONKEYS · KARMAPOLICE · MARBLEDGECKO · MUTANTBROTH · MEMORYHOLE · SAMUELPEPYS · SOCIALANIMAL · HAUSTORIUM · GOLDENAXE · NGE -
Broadcast/Internet Radio Exploitation and Analysis, 6th November 2009, GCHQ [pdf]
This GCHQ report analyzes 6.68 million internet radio listening events collected over three months in 2009, detailing global broadcasting patterns, user behaviors, and specifically examining Islamic radio stations and Pakistan-UK connections for potential radicalization intelligence, while recommending integration with bulk surveillance databases like KARMA POLICE for enhanced user profiling.
-
ICTR Cloud Efforts [pdf]
A GCHQ presentation details how the agency developed cloud-based big data analytics to achieve "population scale" surveillance—processing trillions of internet events to build geographic and behavioral profiles on every online identifier worldwide, enabling discovery of unknown targets through pattern analysis rather than traditional keyword searches.
Programs: EVERYASSOC · PROBABILITYCLOUD · SILVERLINING · CULTWEAVE · KARMAPOLICE · EVERYPOLICE · AWKWARDTURTLE · GORDIANKNOT · SANDIA · BLOODHOUND · FIVEALIVE · NGE · MAPREDUCE -
Content-Metadata Matrix [pdf]
GCHQ's internal taxonomy sorts intercepted data into "content" or "metadata" to determine what can be searched without individual warrants—email subject lines, full URLs, and buddy lists fall on opposite sides of the line, enabling analysts to mine massive troves of IP addresses, phone numbers, location data, and browsing history with just a few keystrokes and a brief justification.
-
Cyber Defence Operations Legal Policy [pdf]
GCHQ's internal wiki laying out the legal plumbing for its Cyber Defence Operations (formerly NDIST)—detailing warrant types (RIPA, ISA, CNE), data retention schedules, targeting rules for UK/Five Eyes infrastructure, XKEYSCORE query protocols, and the bureaucratic hoops analysts must jump through to search attacker IPs, fingerprint UK networks, and share malware intelligence without tripping over location/nationality safeguards or commercial sensitivities.
Programs: JTRIG · SPAY · GORDIANKNOT · CHORDAL · DISCOVER · EVENTS · CASPORT · XKEYSCORE · CONTENT CLOUD · SQUEAL · MIRAGE · MUTANTBROTH · ANXIOUS · HALTERHITCH · BROADOAK · DONKEYKONG · ZOOL · PALANTIR · OVERLINK · RT · GATEKEEPER · IRONHAND · HARUSPEX · B3M · LOOKINGGLASS · GOLDENEYE · BLAZINGSADDLES · MOONRAKER · TRITON
Related Documents
-
Operational Engineering November 2010 [pdf]
This internal GCHQ briefing from late 2010 documents the Events Product Centre's suite of bulk internet and communications surveillance tools — including IMMINGLE, SALAMANCA, SOCIAL ANTHROPOID, MUTANT BROTH, KARMA POLICE, SAMUEL PEPYS and GUIDING LIGHT — covering scaling targets, new data sources, and operational capabilities across telephony, webmail, instant messaging, social media, GPRS and VoIP collection.
Programs: EVENTS · IMMINGLE · SALAMANCA · GUIDINGLIGHT · B3M · HARDASSOC · MAINWAY · THUGGEE · FASCIA · HAUSTORIUM · FIRESTORM · TERRAIN · SHAREOWN · ESCHAR · BRIO · INFINITEMONKEY · KARMAPOLICE · MARBLEDGECKO · MEMORYHOLE · MUTANTBROTH · SAMUELPEPYS · SOCIALANIMAL · SOCIALANTHROPOID · HEARTBEAT · SANTHROPOID · LEAKYGATEWAYS · GTPTUNNEL · LUSTRE · GLASSBACK · COLLATERAL · HRMAP · SWORDPLAY · BROADOAK · REFORMER · ARTEMIS · MERLOT · CONTRAOCTAVE · CRYINGFOWL · CULTWEAVE · SAMDYCE · GCQUERY · BLAZZINGSADDLES · EREPO · JTRIG · SOUNDER -
Next Generation Events [pdf]
A 2009 TOP SECRET briefing outlines the “Next Generation Events” program to scale internet metadata and content analysis across Five Eyes, addressing explosive growth in data volume and the limits of legacy ingest systems. It presents a phased roadmap combining cloud infrastructure, large‑scale event processing, and enhanced profiling to enable bulk analytics and contact chaining.
Programs: NGE · HAUSTORIUM · BLAZINGSADDLES · HARBOURPILOT · MUTANTBROTH · AUTOASSOC · KARMAPOLICE · SOCIALANIMAL · INFINITEMONKEYS · HRMAP · MEMORYHOLE · MARBLEDGECKO · SIMMER · ROCKRIDGE · SAMUELPEPYS · CAFFEINEHIT · LAUGHINGHYENA · SALAMANCA · CLOUD · MONTEVISTA · LOOKINGGLASS · XKEYSCORE · FUMECUPBOARD · IIB · MoaG -
Event (SIGINT) [pdf]
5 p.This document explains that within GCHQ and NSA SIGINT operations, an “Event” refers to metadata about observable communications or user actions rather than message content, categorized into types like Communications, Presence, Social, and Geo-reference Events. These events are collected, stored in various databases, and analyzed through convergence and correlation processes, with standardized formats defined by Interface Control Documents such as INTERSTELLAR DUST (IDUST) and Actor Action (AA).
Programs: INTERSTELLAR DUST · HAUSTORIUM · SOCIAL ANTHROPOID · ACTOR-ACTION · SAMREF · GEOFUSION · HARD ASSOC · SALAMANCA · ASSOCIATION · BANYAN · MAINWAY · FASCIA · CULTWEAVE II · VOICESAIL · TEEDALE · PILBEAM · MUTANTBROTH · AUTOASSOC · MARBLEDGECKO · KARMAPOLICE · HRMAP · MEMORYHOLE · INFINITEMONKEYS · SOCIALANIMAL · AUTOTDI · SAMUELPEPYS · Mobile Applications Project · NGE · PPF · SLR -
Legalities - GCHQ Databases eg Pilbeam, Salamanca, UDAQ etc [pdf]
This October 2007 GCHQ document outlines the legal authorization requirements for querying surveillance databases, revealing that GCHQ analysts can search their own databases (like PILBEAM, SALAMANCA, UDAQ) for metadata on anyone regardless of nationality without additional authorization, while NSA databases (like MARINA, MAINWAY, DISHFIRE) prohibit searches for Americans and other Five Eyes citizens but permit searches for UK persons.
-
PRESTON Architecture [pdf]
The PRESTON architecture, centered on GCHQ’s PRESTON Processing system, is detailed across multiple views showing how the agency intercepts, processes, and stores communications. It outlines a sophisticated surveillance framework that integrates lawful interception, data processing, and analyst access, emphasizing covert monitoring and management of intercepted data.
Programs: PRESTON · FAST GROK · BOXSTER · MARMION · GENTIAN · TERRAIN · BLACKNIGHT · CORINTH · FARNDALE · SAMDYCE · B3M · VSLT · SAMBOK · HAUSTORIUM · FUNFAIR · Cheltenham · SALAMANCA · IIB · LOCHNVAR · MONACO · HOTLINE · Benhall · GDR · CE · BRIGHTON · ARSENIDE · SPQR · TACHO · TRAFFIC MASTER · KEYCARD · XKEYSCORE · COURIERSKILL · MONKEYPUZZLE · PILBEAM · GREENHEART · CCDF · GTE · CADENCE · TERRAINS · GCNET · HENCE