Program: KARMAPOLICE 17 documents
Search documents matching “KARMAPOLICE” →Codenames and programs recorded for each document.
Graphs
By Banner
By Classification
Refine (combine with another filter, via search)
Sort: Title
Documents
-
BLACK HOLE ANALYTICS, ADD/SD briefing, September 2009 [pdf]
This September 2009 GCHQ briefing describes the BLACK HOLE analytics infrastructure and Next Generation Events (NGE) project, which planned to scale from ~8 10-gigabit connections to ~200 10Gs by March 2011, storing intercepted data in massive flat-file repositories at Benhall and Bude before loading it into "Question Focused Dataset" (QFD) databases that power surveillance tools like MUTANT BROTH, KARMA POLICE, INFINITE MONKEYS, and others. The document outlines how "probes" extract Target Detection Identifiers (TDIs) from intercepted data feeds, store them in BLACK HOLE repositories, then enable analysts to profile targets' online activities, create social networks, investigate websites and forums, track web searches and Google Earth usage, and perform large-scale data mining across internet and telephony data.
Programs: BLACKHOLE · MUTANTBROTH · AUTOASSOC · INFINITEMONKEYS · KARMAPOLICE · SOCIALANIMAL · HRMAP · MEMORYHOLE · MARBLEDGECKO · HARDASSOC · SAMUELPEPYS · BLAZINGSADDLES · TELLURIAN · BUFFER · DISTILLERY · LOOKINGGLASS · NGEPeople: bin Laden, Osama -
Blazing Saddles [pdf]
This document describes BLAZING SADDLES, a GCHQ program designed to massively scale the storage and analysis of bulk internet surveillance data through multiple specialized databases that track website visits, online identities, communications patterns, and geolocation queries across the global internet population.
Programs: BLAZINGSADDLES · BLACKHOLE · SOCIALANTHROPOID · AUTOASSOC · HRMAP · INFINITEMONKEYS · KARMAPOLICE · MARBLEDGECKO · MUTANTBROTH · MEMORYHOLE · SAMUELPEPYS · SOCIALANIMAL · HAUSTORIUM · GOLDENAXE · NGE -
Broadcast/Internet Radio Exploitation and Analysis, 6th November 2009, GCHQ [pdf]
This GCHQ report analyzes 6.68 million internet radio listening events collected over three months in 2009, detailing global broadcasting patterns, user behaviors, and specifically examining Islamic radio stations and Pakistan-UK connections for potential radicalization intelligence, while recommending integration with bulk surveillance databases like KARMA POLICE for enhanced user profiling.
-
Crypt Discovery Joint Collaboration Activity 20 January 2011 [pdf]
A TOP SECRET NSA/GCHQ joint collaboration activity plan from January 2011 describing a research programme to understand and mitigate the threat that rising encryption adoption poses to target discovery and development — measuring encryption prevalence across SIGINT collection, assessing the impact on metadata-based targeting tradecraft, and developing new techniques for identifying targets behind encrypted traffic using a joint cloud environment fed by MUTANT BROTH, BEARDED PIGGY, KARMA POLICE, MEMORY HOLE, MARBLED GECKO, SOCIAL ANIMAL and ASDF metadata.
Marking: TOP SECRETPrograms: TICKETWINDOW · MUTANTBROTH · BEARDEDPIGGY · KARMAPOLICE · MEMORYHOLE · MARBLEDGECKO · SOCIALANIMAL · XKEYSCORE · ASDF“Prying Eyes: Inside the NSA's War on Internet Security” - Der Spiegel ↗ (2014-12-28) · “Profiled: From Radio to Porn, British Spies Track Web Users' Online Identities” - The Intercept ↗ (2015-09-25) · Appelbaum, Jacob · Gibson, Aaron · Grothoff, Christian · Müller-Maguhn, Andy · Poitras, Laura · Sontheimer, Michael · Stöcker, Christian · Gallagher, Ryan -
Data Stored in BLACK HOLE [pdf]
This document describes BLACK HOLE, a TOP SECRET data repository that stores approximately 200GB per day of internet activity "events" such as webmail, chat, and browsing. It details the system's various datamarts, including MUTANT BROTH and KARMA POLICE, which collectively hold terabytes of data for intelligence analysis.
Programs: BLACKHOLE · B13B · MUTANTBROTH · KARMAPOLICE · MEMORYHOLE · AUTOASSOC · SOCIALANIMAL · HRMAP · INFINITEMONKEYS -
Demystifying NGE Rock Ridge [pdf]
This presentation outlines the ROCK RIDGE project, which delivers a suite of "Question Focused Databases" (QFDs) like MUTANT BROTH and KARMA POLICE to analyze massive internet event data. These specialized databases are designed to answer specific intelligence questions and are developed using the BLACK HOLE data repository, a key component of the ROUGH DIAMOND program.
Programs: ROCKRIDGE · ROUGHDIAMONG · BLACKHOLE · HAUSTORIUM · HRMAP · MUTANTBROTH · KARMAPOLICE · GOOBZS · INFINITEMONKEYS · AUTOASSOC · SOCIALANIMAL · MARBLEDGECKO · MEMORYHOLE · SAMUELPEPYS · SALAMANCA · MONOPOLY · RADIUS · HARBOURPILOT · LOOKINGGLASS · FIRESTORM · SUPERDRAKE · BLAZINGSADDLES · BLUESHIFT · SUPPORTINGINO · TERRAINS · GEOFUSION · NGEPeople: Lobban, Iain -
Event (SIGINT) [pdf]
5 p.This document explains that within GCHQ and NSA SIGINT operations, an “Event” refers to metadata about observable communications or user actions rather than message content, categorized into types like Communications, Presence, Social, and Geo-reference Events. These events are collected, stored in various databases, and analyzed through convergence and correlation processes, with standardized formats defined by Interface Control Documents such as INTERSTELLAR DUST (IDUST) and Actor Action (AA).
Programs: INTERSTELLAR DUST · HAUSTORIUM · SOCIAL ANTHROPOID · ACTOR-ACTION · SAMREF · GEOFUSION · HARD ASSOC · SALAMANCA · ASSOCIATION · BANYAN · MAINWAY · FASCIA · CULTWEAVE II · VOICESAIL · TEEDALE · PILBEAM · MUTANTBROTH · AUTOASSOC · MARBLEDGECKO · KARMAPOLICE · HRMAP · MEMORYHOLE · INFINITEMONKEYS · SOCIALANIMAL · AUTOTDI · SAMUELPEPYS · Mobile Applications Project · NGE · PPF · SLR -
GCHQ Analytic Cloud Challenges [pdf]
The 2012 GCHQ Analytic Cloud document outlines a sophisticated data processing infrastructure handling approximately 50 billion daily events across multiple sites, utilizing QFD databases and HADOOP clusters for various analytic capabilities targeting intelligence gathering and analysis. It describes architectural components, challenges in scaling and enabling analysts to cope with data complexity, and proposes future technology options including consolidation approaches and new cloud architectures to enhance capabilities.
Programs: EVOLVEDMUTANTBROTH · MUTANTBROTH · HARDASSOC · HRMAP · KARMAPOLICE · SOCIALANTHROPOID · AUTOASSOC · GEOFUSION · MVR · TERRAIN · SALAMANCA · BLACKHOLE · SAMUELPEPYS · BROKER · VAIL · GUIDINGLIGHT · FIRSTCONTACT · CLOUDYCOBRA · RUMOURMILL · EVERYASSOC · EVERYPOLICE · EVERYCREATURE · EVERYEAD · EVERYCIPHER · GREYFOX · SILVERFOX · GOLDENAXE · TRIBALCARNEM · PUBLICANEMONE · EPICFAIL · STERLINGMOTH · FOGHORN · LOOKINGGLASS · FIREENGINE · NGGEO · ALPHACENTAURI · CLOUDBASE · ACCUMULO · ASEM · SEM · HAKIM · STREAMING · HBASE · LAUREL · SKINNYCLOUD · HARDY · GCHQCORE -
ICTR Cloud Efforts [pdf]
A GCHQ presentation details how the agency developed cloud-based big data analytics to achieve "population scale" surveillance—processing trillions of internet events to build geographic and behavioral profiles on every online identifier worldwide, enabling discovery of unknown targets through pattern analysis rather than traditional keyword searches.
Programs: EVERYASSOC · PROBABILITYCLOUD · SILVERLINING · CULTWEAVE · KARMAPOLICE · EVERYPOLICE · AWKWARDTURTLE · GORDIANKNOT · SANDIA · BLOODHOUND · FIVEALIVE · NGE · MAPREDUCE -
Menwith initiatives maximizing our access [pdf]
11 p.This document details Menwith Hill Station’s initiatives to maximize and optimize SIGINT collection, processing, and exploitation, including large-scale monitoring of internet and communications traffic, metadata analysis, and cross-platform target tracking. It describes programs and techniques for surveying, processing, and exploiting signals, enabling analysts to identify, geolocate, and track targets across multiple networks and devices.
Programs: ASPHALT · BLACKTOP · TARMAC · GTE · INDEX · ROADBED · DARKQUEST · EVIL EYE · Spider · SHAREDVISION · QUEST · NOSEYPARKER · XKEYSCORE · GRANDPIANO · FOGHORN · BASSQUEST · APPARITION · MUTANT BROTH · KARMAPOLICE · SOCIALANIMAL · MEMORY HOLE · TURBULENCE · TELLURIAN · MAILORDER · AUTO ASSOC · SLR · BLACKHOLE · JTRIG · PPF · LECKWITH · SCEPTRE · CLOUDPeople: Alexander, Keith B.“GCHQ taps fibre-optic cables for secret access to world's communications” - The Guardian ↗ (2013-06-21) · “Inside Menwith Hill: The NSA’s British Base at the Heart of U.S. Targeted Killing” - The Intercept ↗ (2016-09-06) · MacAskill, Ewen · Borger, Julian · Hopkins, Nick · Davies, Nick · Ball, James · Gallagher, Ryan -
Next Generation Events [pdf]
A 2009 TOP SECRET briefing outlines the “Next Generation Events” program to scale internet metadata and content analysis across Five Eyes, addressing explosive growth in data volume and the limits of legacy ingest systems. It presents a phased roadmap combining cloud infrastructure, large‑scale event processing, and enhanced profiling to enable bulk analytics and contact chaining.
Programs: NGE · HAUSTORIUM · BLAZINGSADDLES · HARBOURPILOT · MUTANTBROTH · AUTOASSOC · KARMAPOLICE · SOCIALANIMAL · INFINITEMONKEYS · HRMAP · MEMORYHOLE · MARBLEDGECKO · SIMMER · ROCKRIDGE · SAMUELPEPYS · CAFFEINEHIT · LAUGHINGHYENA · SALAMANCA · CLOUD · MONTEVISTA · LOOKINGGLASS · XKEYSCORE · FUMECUPBOARD · IIB · MoaG -
NGE BLACK HOLE ConOp [pdf]
This document outlines the "BLACK HOLE" Concept of Operations (ConOp) for the "Next Generation Events" (NGE) project within "BLAZING SADDLES," detailing a flat file data store designed to house various intelligence feeds. Its primary purpose is to serve as a source for Query Focused Datasets (QFDs) and support research, bulk analysis, and target discovery, with two operational instances at Benhall and Bude handling massively increased data volumes.
Programs: NGE · BLAZINGSADDLES · BLACKHOLE · MUTANTBROTH · KARMAPOLICE · SOCIALANIMAL · TELLURIAN · BUFFER · MAILORDER · MARBLEDGECKO · SALAMANCA · SMOKINGSADDLES · CIRCUIT · BLACKFIND · BLACKCAT · DATAFLOWCAB · CHORDAL · DEBIT CARD · B13B -
Operational Engineering November 2010 [pdf]
This internal GCHQ briefing from late 2010 documents the Events Product Centre's suite of bulk internet and communications surveillance tools — including IMMINGLE, SALAMANCA, SOCIAL ANTHROPOID, MUTANT BROTH, KARMA POLICE, SAMUEL PEPYS and GUIDING LIGHT — covering scaling targets, new data sources, and operational capabilities across telephony, webmail, instant messaging, social media, GPRS and VoIP collection.
Programs: EVENTS · IMMINGLE · SALAMANCA · GUIDINGLIGHT · B3M · HARDASSOC · MAINWAY · THUGGEE · FASCIA · HAUSTORIUM · FIRESTORM · TERRAIN · SHAREOWN · ESCHAR · BRIO · INFINITEMONKEY · KARMAPOLICE · MARBLEDGECKO · MEMORYHOLE · MUTANTBROTH · SAMUELPEPYS · SOCIALANIMAL · SOCIALANTHROPOID · HEARTBEAT · SANTHROPOID · LEAKYGATEWAYS · GTPTUNNEL · LUSTRE · GLASSBACK · COLLATERAL · HRMAP · SWORDPLAY · BROADOAK · REFORMER · ARTEMIS · MERLOT · CONTRAOCTAVE · CRYINGFOWL · CULTWEAVE · SAMDYCE · GCQUERY · BLAZZINGSADDLES · EREPO · JTRIG · SOUNDER -
Pull Steering Group Minutes [pdf]
A top secret STRAP1 GCHQ PullThrough Steering Group meeting record from February 2008 documenting the approval process for five surveillance and intelligence prototypes including KARMA POLICE — the mass internet browsing profiling system — alongside ORB FINDER for CNE operational relay box identification, SALTY OTTER for cross-media timing analysis, MOOSE MILK for telephone kiosk surveillance, and LIGHTWOOD for email address extraction used by JTRIG.
Programs: MYOFIBRIL · SALTYOTTER · PRIMETIME · KARMAPOLICE · ORBFINDER · MOOSEMILK · LIGHTWOOD · JTRIG · MERAPEAK · MONTEVISTA · KED -
QFD BLACKHOLE Technology Behind INOC [pdf]
A UK SECRET STRAP2 COMINT ORCON GCHQ Applied Research presentation from March 2009 documenting the BLACK HOLE bulk data repository's scale and structure, and the suite of analytical tools — MUTANT BROTH, KARMA POLICE, INFINITE MONKEYS and MARBLED GECKO — built on top of it to answer specific intelligence questions about online presence, browsing activity, forum usage and geolocation searches.
Programs: BLACKHOLE · MESME · NETEZZA · CLOUDS · MUTANTBROTH · KARMAPOLICE · INFINITEMONKEYS · MARBLEDGECKO · GOOB · AUTOASSOC · PSOUP · SOCIALANIMAL · SAMUELPEPYS · MEMORYHOLE · HRMAP · PSOUPALERT · LAUGHINGHYENA -
Target Detection Identifiers [pdf]
A GCHQ presentation from March 2009 introducing the concept of Target Detection Identifiers — the "fundamental atom of the Internet age" — describing how cookies and other unique persistent identifiers extracted from bulk internet traffic at 2,500 per second are used to build the surveillance databases, with a live AutoTDI discovery table showing cookies extracted from Facebook, Yahoo, Google, Reuters, YouPorn, Bebo and other sites.
Programs: MUTANTBROTH · AUTOASSOC · KARMAPOLICE · INFINITEMONKEYS · SOCIALANIMAL · MEMORYHOLE · MARBLEDGECKO · HRMAP · AUTOTDI -
TWO FACE [pdf]
This document describes the GCHQ's TWO FACE project, a part of the Cyber Defence mission, which focuses on enhancing analysis and knowledge sharing tools using Palantir software. It outlines various data sources integrated with Palantir, such as XKEYSCORE and GEOFUSION, and the project's goals to automate and improve data importation, analysis capabilities, and user interface performance. The document also discusses training, user access, and collaboration within the Five Eyes intelligence community, as well as ongoing development to address gaps in existing cyber defense tools. It provides detailed instructions for using Palantir's capabilities to analyze and visualize data from multiple sources, with a focus on cyber defense and network defense operations. Key features include handling large datasets, running complex queries, and automating data integration. The document also highlights ongoing improvements, such as the addition of new data sources and the refinement of analytic tools to better support the cyber defense mission.
Programs: TWOFACE · PALANTIR · FRACTALJOKER · FRACTALWEB · PENSIVEGIRAFFE · XKEYSCORE · LIVE · AUTOASSOC · SOCIALANTHROPOID · INFINITEMONKEYS · KARMAPOLICE · SAMPEPYS · HRMAP · MUTANTBROTH · GEOFUSION · HACIENDA · FOXTRAIL · CROUCHINGSQUIRREL · PITCHFORD · HALTERHITCH · GORDIANKNOT · SAMUELPEPYS · MUGSHOT · RAPIDTAPIR · FIVEALIVE · GOOGLEFUSION · INTEGERSPIN · DEADSEA · MOONRAKER · OBERON · DISCOVER · 8BALL · CASK