Tag: data storage 13 documents
Search documents matching “data storage” →Free-text subjects the curators added. Unlike the other axes these are not necessarily from the documents' own vocabulary, so they are the loosest thing to browse by (and the most forgiving).
Graphs
By Banner
By Classification
Refine (combine with another filter, via search)
Sort: Title
Documents
-
Analysts: A New Tool for Researching Telecommunications Infrastructure is Now Available SIDtoday [pdf]
QUARRYGEM is an application created by the Environmental Analysis Branch to give analysts and other users enhanced access to NSA's data on telecommunications infrastructure systems. It currently only contains data on undersea cable systems but was designed to handle other telecommunications infrastructure data, potentially including satellite, microwave and terrestrial data sources.
The Intercept ↗ (2019-05-29) -
Communications Capability Development Programme [pdf]
The Communications Capability Development (CCD) Programme (formerly the Intercept Modernisation Programme, IMP) was a Home Office–led effort to maintain the U.K.’s lawful interception and communications data capabilities amid changing technologies, emphasizing that law enforcement faced a “significant decline” without new measures. GCHQ supported CCD with technical expertise and interim services like MILKWHITE Enrichment, while stressing that CCD was politically contentious, technically complex, and required legislation to mandate data retention by U.K. network operators.
-
ConSIDer This: Electronic Records SIDtoday [pdf]
How NSA deals with electronic records, including tips on records management and transfer to the Records Center or Archives.
The Intercept ↗ (2016-05-16) -
DFS SIGINT-enabled Cyber [pdf]
This February 2013 top-secret DFS presentation outlines how Japan’s Directorate for Signals Intelligence (DFS) developed a SIGINT-enabled cyber program with U.S. NSA support, launching operations in late 2012 using the MALLARD system to collect and analyze satellite-based internet sessions. It details the project’s buildup, early challenges with massive data volumes, reliance on U.S. technical assistance, and plans for expanded antennas, infrastructure, and long-term system upgrades.
Marking: TOP SECRETPrograms: MALLARD“The Untold Story of Japan's Secret Spy Agency” - The Intercept ↗ (2018-05-19) · “日本の諜報 スクープ 最高機密ファイル” - NHK ↗ (2018-05-19) · Gallagher, Ryan -
Entries From Sample SSO Accesses [pdf]
This document reveals that OAKSTAR is a covert NSA program involving corporate partnerships to access global internet traffic, while MONKEYROCKET, a sub-program of OAKSTAR, specifically targets foreign internet traffic from the Middle East, Europe, South America, and Asia for surveillance, with data funneled to the NSA’s European Technical Center for further analysis. It also notes that the data collected is stored in the YANKEE partition for processing.
-
MILKWHITE Enrichment Service (MES) Programme [pdf]
This March 2011 GCHQ document details the MILKWHITE program, which is enhancing domestic interception and data analysis capabilities. It outlines expanded funding for advanced analytics, bulk data storage, and service scaling to meet growing demand from multiple law enforcement and intelligence agencies, including SOCA, HMRC, MI5, and SIS, thereby broadening access to large volumes of internet-derived metadata for operational use.
-
PRESTON Architecture [pdf]
The PRESTON architecture, centered on GCHQ’s PRESTON Processing system, is detailed across multiple views showing how the agency intercepts, processes, and stores communications. It outlines a sophisticated surveillance framework that integrates lawful interception, data processing, and analyst access, emphasizing covert monitoring and management of intercepted data.
Programs: PRESTON · FAST GROK · BOXSTER · MARMION · GENTIAN · TERRAIN · BLACKNIGHT · CORINTH · FARNDALE · SAMDYCE · B3M · VSLT · SAMBOK · HAUSTORIUM · FUNFAIR · Cheltenham · SALAMANCA · IIB · LOCHNVAR · MONACO · HOTLINE · Benhall · GDR · CE · BRIGHTON · ARSENIDE · SPQR · TACHO · TRAFFIC MASTER · KEYCARD · XKEYSCORE · COURIERSKILL · MONKEYPUZZLE · PILBEAM · GREENHEART · CCDF · GTE · CADENCE · TERRAINS · GCNET · HENCE -
SKIDROWE [pdf]
This document reveals the extensive surveillance capabilities of the NSA's SKIDROWE program at TITANPOINTE, specifically designed to intercept and analyze satellite communications and internet data. It highlights how intercepted data is processed and filtered through advanced systems like XKEYSCORE, which enables real-time intelligence gathering by sifting through massive amounts of digital content, including emails, calls, and internet activity.
Programs: SKIDROWE · WEALTHYCLUSTER2 · WEALTHYCLUSTER · TURNWEALTHY · VPCS · XKEYSCORE · GENESIS · CONTROLAND · ATM · FITBOY · DARKTHUNDER · FIP · METTLESOME · SKYFRAME · PVOICE · XXPY0012 · AGITATOR · STARPROC · SPELLBINDER · BLACKPEARL · BLACKALCHEMY · BLACKGOLD · BLACKFIRE · MONKEYMIND · BCD · EVVIPER · QUANTUM · MISTYPEAK · VALIANTSURF · GALLANTWAVE · SWINE · BLUESNORT · TOSSSWUNG · GNOMEVISION · GNOMEGOGGLES · HIDDENSALAMANDER · TURMOIL · PSC -
SSO Collection Optimization [pdf]
This is another presentation on problems with the National Security Agency's overcollection of address books and buddy lists and its efforts to weed out useless content. See also EFF_-_20131104-wapo-optimization_google
-
SSO GHOSTMACHINE Analytics [pdf]
This document describes the NSA’s cloud analytics platform GHOSTMACHINE, used by Special Source Operations (SSO) to process and analyze massive volumes of data (e.g., from FASCIA) using Hadoop-based infrastructure. It’s essentially the backbone system for querying, correlating, and enriching surveillance data at scale — not just a single analytic, but the analytic/cloud environment powering many programs.
Programs: GHOSTMACHINE · FASCIA · CERF · POPTOP · DRT · SIGDASYS · JUGGERNAUT · SEADIVER · AST128 · TUSKATTIRE · ASDF · BROKENRECORD · REDHAWK · SSO -
TEMPORA -- "The World's Largest XKEYSCORE" -- Is Now Available to Qualified NSA Users [pdf]
This document announces expanded NSA analyst access to TEMPORA, GCHQ’s massive XKEYSCORE‑based internet buffer, positioning it as the largest single XKEYSCORE data source in existence. It promotes TEMPORA as a high‑value discovery and development platform for large‑scale internet surveillance.
-
The collection challenge [pdf]
A slide stating the collection problem in one line: “Collection is outpacing our ability to ingest, process and store to the ‘norms’ to which we have become accustomed.”
-
Too Much of a Good Thing? SIDtoday [pdf]
NSA's UK Menwith Hill Station finds itself "bursting at the seams" with unencrypted internet data collected from commercial satellite networks. A new initiative aims to "thin out the data we store at site".
“Inside Menwith Hill: The NSA’s British Base at the Heart of U.S. Targeted Killing” - The Intercept ↗ (2016-09-06) · The Intercept ↗ (2019-05-29) · Gallagher, Ryan