Tag: data retention 13 documents
Search documents matching “data retention” →Free-text subjects the curators added. Unlike the other axes these are not necessarily from the documents' own vocabulary, so they are the loosest thing to browse by (and the most forgiving).
Graphs
By Banner
By Classification
Refine (combine with another filter, via search)
Sort: Title
Documents
-
Cyber Defence Operations Legal Policy [pdf]
GCHQ's internal wiki laying out the legal plumbing for its Cyber Defence Operations (formerly NDIST)—detailing warrant types (RIPA, ISA, CNE), data retention schedules, targeting rules for UK/Five Eyes infrastructure, XKEYSCORE query protocols, and the bureaucratic hoops analysts must jump through to search attacker IPs, fingerprint UK networks, and share malware intelligence without tripping over location/nationality safeguards or commercial sensitivities.
Programs: JTRIG · SPAY · GORDIANKNOT · CHORDAL · DISCOVER · EVENTS · CASPORT · XKEYSCORE · CONTENT CLOUD · SQUEAL · MIRAGE · MUTANTBROTH · ANXIOUS · HALTERHITCH · BROADOAK · DONKEYKONG · ZOOL · PALANTIR · OVERLINK · RT · GATEKEEPER · IRONHAND · HARUSPEX · B3M · LOOKINGGLASS · GOLDENEYE · BLAZINGSADDLES · MOONRAKER · TRITON -
Data Stored in BLACK HOLE [pdf]
This document describes BLACK HOLE, a TOP SECRET data repository that stores approximately 200GB per day of internet activity "events" such as webmail, chat, and browsing. It details the system's various datamarts, including MUTANT BROTH and KARMA POLICE, which collectively hold terabytes of data for intelligence analysis.
Programs: BLACKHOLE · B13B · MUTANTBROTH · KARMAPOLICE · MEMORYHOLE · AUTOASSOC · SOCIALANIMAL · HRMAP · INFINITEMONKEYS -
GCHQ Analytic Cloud Challenges [pdf]
The 2012 GCHQ Analytic Cloud document outlines a sophisticated data processing infrastructure handling approximately 50 billion daily events across multiple sites, utilizing QFD databases and HADOOP clusters for various analytic capabilities targeting intelligence gathering and analysis. It describes architectural components, challenges in scaling and enabling analysts to cope with data complexity, and proposes future technology options including consolidation approaches and new cloud architectures to enhance capabilities.
Programs: EVOLVEDMUTANTBROTH · MUTANTBROTH · HARDASSOC · HRMAP · KARMAPOLICE · SOCIALANTHROPOID · AUTOASSOC · GEOFUSION · MVR · TERRAIN · SALAMANCA · BLACKHOLE · SAMUELPEPYS · BROKER · VAIL · GUIDINGLIGHT · FIRSTCONTACT · CLOUDYCOBRA · RUMOURMILL · EVERYASSOC · EVERYPOLICE · EVERYCREATURE · EVERYEAD · EVERYCIPHER · GREYFOX · SILVERFOX · GOLDENAXE · TRIBALCARNEM · PUBLICANEMONE · EPICFAIL · STERLINGMOTH · FOGHORN · LOOKINGGLASS · FIREENGINE · NGGEO · ALPHACENTAURI · CLOUDBASE · ACCUMULO · ASEM · SEM · HAKIM · STREAMING · HBASE · LAUREL · SKINNYCLOUD · HARDY · GCHQCORE -
GCHQ Report on the Technical Capabilities of the Powerful Espionage Program TEMPORA, which Allows for Full Take. [pdf]
In May 2012, GCHQ introduced the TEMPORA internet buffer capability, enabling full take collection of internet communications. This innovation provides NSA and GCHQ analysts access to extensive amounts of data, enhancing their analytical capabilities significantly.
-
Informal Survey Results: When Is a Job Worth Keeping? SIDtoday [pdf]
Junior intelligence analysts hired within the last three years were surveyed informally about the positive and negative aspects of their jobs at the NSA. They identified three key areas for success: challenging and purposeful work; enlightened management; and relevant, ongoing training.
People: Maklowski, DeborahThe Intercept ↗ (2017-09-13) -
Minimization Procedures Used By The National Security Agency In Connection With Acquisitions Of Foreign Intelligence, Information Pursuant To Section 702 Of The Foreign Intelligence Court Surveillance Act Of 1978 [pdf]
The NSA’s Section 702 minimization rules expose how the agency can collect, keep, and use Americans’ communications without warrants, despite claiming to target only foreigners. While the document sets out procedures for handling U.S. data, it includes broad loopholes—like keeping “inadvertently” collected domestic messages for years, sharing unfiltered data with the FBI, CIA, and even foreign governments, and giving NSA analysts wide discretion over who gets surveilled—all under secret FISA Court approval.
People: Holder, Eric H. Jr. -
NGE BLACK HOLE ConOp [pdf]
This document outlines the "BLACK HOLE" Concept of Operations (ConOp) for the "Next Generation Events" (NGE) project within "BLAZING SADDLES," detailing a flat file data store designed to house various intelligence feeds. Its primary purpose is to serve as a source for Query Focused Datasets (QFDs) and support research, bulk analysis, and target discovery, with two operational instances at Benhall and Bude handling massively increased data volumes.
Programs: NGE · BLAZINGSADDLES · BLACKHOLE · MUTANTBROTH · KARMAPOLICE · SOCIALANIMAL · TELLURIAN · BUFFER · MAILORDER · MARBLEDGECKO · SALAMANCA · SMOKINGSADDLES · CIRCUIT · BLACKFIND · BLACKCAT · DATAFLOWCAB · CHORDAL · DEBIT CARD · B13B -
NSA target package [pdf]
This 2011 target package focuses on Muhammad Tahir Shahzad (alias Abu Hamza), a Pakistan-based bomb builder and Al-Qaeda facilitator. The document was produced by the Counter IED Analysis Integration Cell (CAIC) at the NSA. Notably, the document credits the tracking of Shahzad’s 60+ digital selectors with the eventual capture of high-value targets, including himself and Bali bombing suspect Umar Patek. The accompanying Washington Post investigation uses this specific case to highlight the massive scale of "incidental collection." While Shahzad was a valid foreign target under Section 702 of the FISA Amendments Act (FAA), the investigation revealed that for every legitimate target like him, the NSA intercepted nine "non-targeted" users—many of whom were Americans—capturing deeply personal photos, medical records, and private correspondence.
-
OAKSTAR Weekly Update - Program Highlights Week of: March 16-22, 2013 [pdf]
The document is a NSA weekly operations update from March 16-22, 2013, detailing technical issues, maintenance, and intelligence activities within the OAKSTAR and MONKEYROCKET programs—surveillance systems used for data collection. It notes collection outages, system adjustments, and that analysts were using MONKEYROCKET access to track Bitcoin transactions and investigate organized crime and cyber targets using e-currency for money laundering.
Programs: OAKSTAR · PRIMECANE · MONKEYROCKET · NIAGARAFILES · TRANSPORTORO · XKEYSCORE · REMEDY · MARINA · CLOUDRUNNER -
Real-Time Regional Gateway Cloud Analytics for Forward Users [pdf]
This document outlines the NSA’s Real-Time Regional Gateway (RT-RG) program, a global cloud-based signals intelligence system designed to provide near real-time data analysis and targeting support for military and intelligence operations. It details the transition to scalable cloud architectures such as GHOSTMACHINE and Cloudbase, enabling massive data retention, exploitation, advanced geolocation, and automated analytics for tracking high-value targets across multiple theaters, including Afghanistan, Iraq, and the Horn of Africa.
Programs: RT-RG · GOLDMINER · GEOT · AGENTLOGIC · SHARKFINN · SKS · PANOPTICON · DRT · CIDNE · JUGGERNAUT · LOPERS · TALIS · MATTERHORN · AIRHANDLER · LYCANTHROPE · VOICESAIL · RETURNSPRING · SORTINGLEAD · GHOSTMACHINE · GMROYALE · GMREGAL · GMGEIST · GMIDEA · GMTOTE · GMPEN · GMPAC · GMCALM · GMMATE · GMAPPLE · GMCAVE · GMPEACH · GMDEN · GMMOUNT · GMALAMO · GMLIGHT · GMBALTIC · GMPLACE · GMPARK · GMHALO · GMCARBON · DRAGONFIRE · DOMEEXPOSE · GIANTSQUID · MAINWAY · FURIOUSCHAINSAW · SHARKFURY · NIAGARAFILES · SHARKQUERY · SHARKDOCS · OCTAVE · UTT · BEDDOWN · CHEEKYMONKEY · LAYERCAKE · CYBERTRANS · GMSEMINOLE · GMGULF · GMZILLA · HLT · JEMA · NEVIS · SEDB · CLOUDBASE · MAPREDUCE · CASPORT · WAVELEGAL“Mission Creep: How the NSA’s Game-Changing Targeting System Built for Iraq and Afghanistan Ended Up on the Mexico Border” - The Intercept ↗ (2019-05-29) · “Norges hemmelige krig i Afghanistan” - NRK ↗ (2019-05-29) · Moltke, Henrik · Skille, Øyvind Bye · Døvik, Olav · Hansen, Ståle Ellingsgard · Fjørtoft, Martin -
SOMALGET (2012) [pdf]
This 2012 document provides an internal overview of SOMALGET, a sub-program of the MYSTIC surveillance project. It describes a radical shift in telephony intelligence: the move from "targeted" wiretapping to "full-take" nation-scale buffering. SOMALGET allows analysts to record 100% of a country's mobile phone audio and store it for a rolling period of 30 days. This capability, known as "retrospective retrieval," allows the NSA to listen to conversations that occurred before a person was even identified as a target. At the time of this writing, the program was fully operational in the Bahamas and one other redacted location(s).
-
TEMPORA -- "The World's Largest XKEYSCORE" -- Is Now Available to Qualified NSA Users [pdf]
This document announces expanded NSA analyst access to TEMPORA, GCHQ’s massive XKEYSCORE‑based internet buffer, positioning it as the largest single XKEYSCORE data source in existence. It promotes TEMPORA as a high‑value discovery and development platform for large‑scale internet surveillance.
-
The New Voice Processing Tools are Here! SIDtoday [pdf]
As part of SID Desktop Modernization, linguists will transition to the New Voice Architecture, a set of programs that allow linguists to work with language data. It includes "30-day default retention of untouched data" and "5-year default retention of metadata."
Marking: CONFIDENTIALThe Intercept ↗ (2017-04-24)