Tag: data volume 16 documents
Search documents matching “data volume” →Free-text subjects the curators added. Unlike the other axes these are not necessarily from the documents' own vocabulary, so they are the loosest thing to browse by (and the most forgiving).
Graphs
By Banner
By Classification
Refine (combine with another filter, via search)
Sort: Title
Documents
-
BOUNDLESSINFORMANT Chart on Italy [pdf]
A BOUNDLESSINFORMANT signal profile for Italy over 30 days: 45,893,570 records from SIGAD US-987A3005 via DRTBOX.
“Da qui ci spiano gli americani” - L’Espresso ↗ (2013-12-05) · Greenwald, Glenn · Maurizi, Stefania -
BOUNDLESSINFORMANT Chart on United States [pdf]
The BOUNDLESSINFORMANT global overview screen with the United States selected: 97,111,188,358 DNI and 124,808,692,959 DNR records over the last 30 days across 504 SIGADs, of which 3,095,533,478 are attributed to the United States.
Programs: BOUNDLESSINFORMANT -
BOUNDLESSINFORMANT SSO [pdf]
1 p.A BOUNDLESSINFORMANT 30-day profile for Special Source Operations, led by SIGAD US-3171 at 57,788,148,908 records, with XKEYSCORE at 41,996,304,149, LOPERS at 40,940,994,147, TURMOIL at 22,965,148,766, FALLOUT at 12,844,273,427 and FAIRVIEWCOTS at 5,962,942,049.
-
BOUNDLESSINFORMANT, FAIRVIEW [pdf]
1 p.A BOUNDLESSINFORMANT 30-day signal profile for FAIRVIEW (SIGAD US-990): 6,142,932,557 records in total, of which FAIRVIEWCOTS accounts for 5,962,942,049, KEELSON 176,718,447 and SCISSORS 2,614,234.
-
BOUNDLESSINFORMANT, Poland [pdf]
A BOUNDLESSINFORMANT signal profile for Poland over 30 days: 71,819,443 records from SIGAD US-916A via DRTBOX, with PCS, INMARSAT, MOIP, VSAT, HPCP, PSTN and DNI as the top techniques.
-
Data Stored in BLACK HOLE [pdf]
This document describes BLACK HOLE, a TOP SECRET data repository that stores approximately 200GB per day of internet activity "events" such as webmail, chat, and browsing. It details the system's various datamarts, including MUTANT BROTH and KARMA POLICE, which collectively hold terabytes of data for intelligence analysis.
Programs: BLACKHOLE · B13B · MUTANTBROTH · KARMAPOLICE · MEMORYHOLE · AUTOASSOC · SOCIALANIMAL · HRMAP · INFINITEMONKEYS -
ELINT Analysts - Overcome by Overload? Help is Here with IM&S SIDtoday [pdf]
A 2011 NSA SIDtoday article announcing the Initial Operational Capability of the ELINT Modernization Information Management and Storage system, designed to give analysts worldwide unified access to five years of ELINT reports and raw data from an expanding range of traditional and non-traditional collection platforms.
-
GCHQ Analytic Cloud Challenges [pdf]
The 2012 GCHQ Analytic Cloud document outlines a sophisticated data processing infrastructure handling approximately 50 billion daily events across multiple sites, utilizing QFD databases and HADOOP clusters for various analytic capabilities targeting intelligence gathering and analysis. It describes architectural components, challenges in scaling and enabling analysts to cope with data complexity, and proposes future technology options including consolidation approaches and new cloud architectures to enhance capabilities.
Programs: EVOLVEDMUTANTBROTH · MUTANTBROTH · HARDASSOC · HRMAP · KARMAPOLICE · SOCIALANTHROPOID · AUTOASSOC · GEOFUSION · MVR · TERRAIN · SALAMANCA · BLACKHOLE · SAMUELPEPYS · BROKER · VAIL · GUIDINGLIGHT · FIRSTCONTACT · CLOUDYCOBRA · RUMOURMILL · EVERYASSOC · EVERYPOLICE · EVERYCREATURE · EVERYEAD · EVERYCIPHER · GREYFOX · SILVERFOX · GOLDENAXE · TRIBALCARNEM · PUBLICANEMONE · EPICFAIL · STERLINGMOTH · FOGHORN · LOOKINGGLASS · FIREENGINE · NGGEO · ALPHACENTAURI · CLOUDBASE · ACCUMULO · ASEM · SEM · HAKIM · STREAMING · HBASE · LAUREL · SKINNYCLOUD · HARDY · GCHQCORE -
How to configure category throttling in XKEYSCORE [pdf]
An NSA technical guide explains how to configure "category throttling" in XKEYSCORE—a feature that automatically stops forwarding intercepted communications once analyst-defined collection targets hit predetermined data volume or session count limits, allowing the surveillance system to continue processing but prevent overflow from high-volume targets.
-
Is There a Sustainable Ops Tempo in S2? How Can Analysts Deal with the Flood of Collection? SIDtoday [pdf]
The NSA's Deputy Director for Analysis and Production discusses the unsustainable operational tempo caused by overwhelming data collection volumes, advocating for "Analytic Modernization" that would shift from treating SIGINT as "radioactive" NSA-only material to sharing raw intelligence with other agencies for collaborative exploitation.
Programs: A-SPACE -
NGE BLACK HOLE ConOp [pdf]
This document outlines the "BLACK HOLE" Concept of Operations (ConOp) for the "Next Generation Events" (NGE) project within "BLAZING SADDLES," detailing a flat file data store designed to house various intelligence feeds. Its primary purpose is to serve as a source for Query Focused Datasets (QFDs) and support research, bulk analysis, and target discovery, with two operational instances at Benhall and Bude handling massively increased data volumes.
Programs: NGE · BLAZINGSADDLES · BLACKHOLE · MUTANTBROTH · KARMAPOLICE · SOCIALANIMAL · TELLURIAN · BUFFER · MAILORDER · MARBLEDGECKO · SALAMANCA · SMOKINGSADDLES · CIRCUIT · BLACKFIND · BLACKCAT · DATAFLOWCAB · CHORDAL · DEBIT CARD · B13B -
RAMPART-A Sites [pdf]
This document specifies the data throughput and operational constraints of five key RAMPART-A interception sites, including the implementation of the "DNI Throttle" to manage data volume. It also defines the "Support Hub" model for managing signals tasking and explains the role of the REDHARVEST compartment in restricting which analysts can view specific digital network categories.
Programs: RAMPART-A · AZUREPHOENIX · MOONLIGHTPATH · SPINNERET · SMOKYSINK · FIREBIRD · XKEYSCORE · PINWALE · REDHARVEST · CADENCE“How Secret Partners Expand NSA’s Surveillance Dragnet” - The Intercept ↗ (2014-06-18) · “NSA ‘third party’ partners tap the Internet backbone in global surveillance program” - Dagbladet Information ↗ (2014-06-19) · Gallagher, Ryan · Geist, Anton · Gjerding, Sebastian · Moltke, Henrik · Poitras, Laura -
Report on Architectural Risk 2012 - Summary [pdf]
This 2012 GCHQ report identifies that a historical under-investment in IT architecture has created significant risk, especially with new challenges like massive data growth, cyber defense, and increased partnering. The document recommends immediate, prioritized investment in big data strategies, integrated analytics frameworks, and security services to address the strain on the system.
Programs: BLACKHOLE -
SHELLTRUMPET Processes its One Trillionth Metadata Record [pdf]
An SSO news item announcing that SHELLTRUMPET processed its one-trillionth metadata record on 21 December 2012 — almost half of that volume in 2012 alone — feeding TRAFFICTHIEF tipping and Real-Time Regional Gateway filtering from CLASSIC, DANCINGOASIS, RAMPART-M, OAKSTAR, MYSTIC, NCSC, MUSKETEER and Second Party systems.
Programs: SHELLTRUMPET · CLASSIC · TRAFFICTHIEF · RTRG · SSO · DANCINGOASIS · RAMPART-M · OAKSTAR · MYSTIC · NCSC · MUSKETEER -
WINDSTOP - Last 30 Days [pdf]
The WINDSTOP report shows massive NSA collection activity over a 30-day period, with daily DNI (Digital Network Intelligence) and DNR (Dialed Number Recognition) records reaching up to 600 million per day. Most data volume came from DS-300 and DS-200B collection points, processed primarily through XKEYSCORE and TURMOIL, and included a mix of signal types such as PCS, INMAR, MCIP, VSAT, and PSTN. The chart follows the same structure as BOUNDLESSINFORMANT, the NSA’s global SIGINT visualization tool, using bar and pie charts to track and categorize collection volume by signal type, SIGAD, and processing system.
-
WINDSTOP/2P System Highlights [pdf]
A March 2013 NSA Special Source Operations internal report describing a volume management problem at MUSCULAR — the joint NSA/GCHQ programme tapping Yahoo and Google inter-datacenter fiber links — specifically concerning Yahoo's proprietary Narchive email transfer format, which was generating 15GB/day of mostly stale email data representing 25% of the collection site's daily allocation.