SHELLTRUMPET Processes its One Trillionth Metadata Record
Open PDFDocument date: 2012-12-31
Pages: 1
Genre: report
Classification: SECRET SI
Dissemination: REL TO USA, FVEY
An SSO news item announcing that SHELLTRUMPET processed its one-trillionth metadata record on 21 December 2012 — almost half of that volume in 2012 alone — feeding TRAFFICTHIEF tipping and Real-Time Regional Gateway filtering from CLASSIC, DANCINGOASIS, RAMPART-M, OAKSTAR, MYSTIC, NCSC, MUSKETEER and Second Party systems.
SHELLTRUMPET, CLASSIC, TRAFFICTHIEF, RTRG, SSO, DANCINGOASIS, RAMPART-M, OAKSTAR, MYSTIC, NCSC, MUSKETEER
People mentioned:
-
Countries:
-
Tags:
metadata, metadata analysis, data volume, real-time data processing, tipping, second party, collection statistics, call detail records
Identifier:
shelltrumpet-processes-its-one-trillionth-metadata-record
PDF SHA-256:
d77e3599fb375c1dae2aa8c9e401f469591f093026e106bbe436a3570d5e9c87
My notes
(S//SI//REL TO USA, FVEY) SHELLTRUMPET Processes it's One Trillionth
Metadata Record
By [REDACTED] on 2012-12-31 0738
(S//SI//REL TO USA, FVEY) On December 21, 2012 SHELLTRUMPET processed its
One Trillionth metadata record. SHELLTRUMPET began as a near-real-time
metadata analyzer on Dec 8, 2007 for a CLASSIC collection system. In its
five year history, numerous other systems from across the Agency have come
to use SHELLTRUMPET's processing capabilities for performance monitoring,
direct E-Mail tip alerting, TRAFFICTHIEF tipping, and Real-Time Regional
Gateway (RTRG) filtering and ingest. Though it took five years to get to
the one trillion mark, almost half of this volume was processed in this
calendar year, and half of that volume was from SSO's DANCINGOASIS.
SHELLTRUMPET is currently processing Two Billion call events/day from
select SSO (Ram-M, OAKSTAR, MYSTIC and NCSC enabled systems), MUSKETEER,
and Second Party systems. We will be expanding its reach into other SSO
systems over the course of 2013. The Trillion records processed have
resulted in over 35 Million tips to TRAFFICTHIEF.
Published in the Same Report
(10 of 62 documents)
-
NSA Intelligence Relationship with Canada's Communications Security Establishment Canada (CSEC) [pdf]
This document describes the close, long-standing SIGINT partnership between the NSA and Canada’s CSEC, emphasizing cooperative efforts in global intelligence collection, cyber operations, information assurance, and critical infrastructure defense. It highlights mutual contributions, including CSEC’s unique geographic access, covert sites, cryptographic capabilities, and R&D collaboration, supporting U.S. and North American security objectives.
“Communication Security Establishment's cyberwarfare toolbox revealed” - CBC News ↗ (2015-03-23) · “Documents Reveal Canada’s Secret Hacking Tactics” - The Intercept ↗ (2015-03-23) · “No Place to Hide” - Metropolitan Books ↗ (2014-05-13) · Seglins, Dave · Hildebrandt, Amber · Pereira, Michael · Greenwald, Glenn · Gallagher, Ryan -
Close Access SIGADS [pdf]
A 10 September 2010 NSA reference list of Close Access SIGADs revealing the codenames, locations and collection methods used to bug the embassies and UN missions of 38 targets — including US allies France, Italy, Greece, Japan, India, Mexico, South Korea and the EU — through implants, computer-screen capture, hard-drive imaging and electromagnetic emanation collection, most conducted against diplomatic facilities in Washington DC and New York.
Programs: KATEEL · POCOMOKE · MERCED · BANISTER · PERDIDO · MAGOTHY · BLACKFOOT · WABASH · NAVARRO · POWELL · KLONDIKE · NASHUA · OSAGE · OSWAYO · BRUNEAU · HEMLOCK · MULBERRY · ALAMITO · FLEMING · DOBIE · SULPHUR · REQUETTE · YUKON · WESTPORT · NAVAJO · PANTHER · LIFESAVER · HIGHLANDS · VAGRANT · MINERALIZ · DROPMIRE · PBX · MAGNETIC · OCEAN · GENIE · BLACKHEART · CRYPTOENABLED · CUSTOMS · DEWSWEEPER · RADON · MINERALIZE · COOP · STARRUBY · TRASHINT“No Place to Hide” - Metropolitan Books ↗ (2014-05-13) · “Espionnage de la NSA : la diplomatie française était sur écoute aux Etats-Unis” - Le Monde ↗ (2013-10-22) · “Da qui ci spiano gli americani” - L'Espresso ↗ (2013-12-05) · Greenwald, Glenn · Follorou, Jacques · Maurizi, Stefania -
Cyber Integration [pdf]
The leaked 2012 GCHQ JTRIG presentation detail a wide range of offensive cyber and influence operations designed to “deny, disrupt, degrade, and deceive” targets, including hackers, foreign governments, companies, and individuals. Tactics include hacking, DDOS attacks, disinformation, honey traps, credential harvesting, and psychological operations, reflecting a shift from traditional surveillance to active disruption and manipulation campaigns.
-
XKEYSCORE intro [pdf]
A TOP SECRET NSA presentation from 2008 introducing XKeyscore — one of the earliest overview documents in the archive — describing the system as a globally distributed Linux cluster across approximately 150 sites and 700+ servers that stores a ~3-day rolling buffer of all unfiltered internet traffic, and demonstrating its core capabilities for finding previously unknown targets by mining metadata, tracking documents by embedded image hashes, detecting encryption and VPN use, and language tracking.
Programs: XKEYSCORE · TURMOIL · TURBULENCE · TRAFFICTHIEF · WEALTHYCLUSTER · MOONSHINE · SEARCHLIGHT · MARINA · PSC · SSO · TIMBERLINEPeople: bin Laden, Osama“'Brasil é um grande alvo', diz jornalista sobre vigilância dos EUA” - O Globo ↗ (2013-07-07) · “XKeyscore: NSA tool collects 'nearly everything a user does on the internet'” - The Guardian ↗ (2013-07-31) · “No Place to Hide” - Metropolitan Books ↗ (2014-05-13) · “XKEYSCORE: NSA’s Google for the World’s Private Communications” - The Intercept ↗ (2015-07-01) · Greenwald, Glenn · Bridi, Sônia · Marquis-Boire, Morgan · Lee, Micah -
Intelligently filtering your data: Brazil and Mexico case studies [pdf]
These documents reveal that the NSA conducted targeted surveillance operations against high-profile Brazilian and Mexican figures, including President Dilma Rousseff and presidential candidate Enrique Peña Nieto, using advanced contact-graph filtering and data analysis techniques to extract relevant intelligence from intercepted communications. These documents show how NSA analysts systematically developed intelligence on foreign leaders during election periods and diplomatic activities, framing it as “target development,” but in practice enabling deep political espionage under the guise of analytical innovation.
“Veja os documentos ultrassecretos que comprovam espionagem a Dilma” - O Globo ↗ (2013-09-02) · “No Place to Hide” - Metropolitan Books ↗ (2014-05-13) · Greenwald, Glenn -
Sharing Computer Network Operations Cryptologic Information with Foreign Partners [pdf]
The document is a secret NSA/CSS policy outlining guidelines for sharing Computer Network Operations (CNO) cryptologic information with foreign partners, including rules for cooperation in CNE and CND within existing intelligence relationships. The “Purpose and Scope” section on the first page appears to be unpublished elsewhere, while the Tier A and Tier B country lists match those publicly revealed in sources like Glenn Greenwald’s No Place to Hide.
“El CNI facilitó el espionaje masivo de EEUU a España” - El Mundo ↗ (2013-10-30) · “No Place to Hide” - Metropolitan Books ↗ (2014-05-13) · Greenwald, Glenn · Aranda, Germán -
NSA Spied On Porn Habits As Part Of Plan To Discredit 'Radicalizers' [pdf]
Top-secret document reveals how NSA has been gathering records of online sexual activity and evidence of visits to pornographic websites as part of a proposed plan to harm the reputations of those whom the agency believes are radicalizing others through incendiary speeches
-
Discovery SIGINT Targeting Scenarios and Compliance [pdf]
This document is a top-secret NSA guidance manual detailing SIGINT targeting scenarios, compliance procedures, and legal guidance for querying foreign actors, malware, and potentially U.S.-linked systems, including considerations for minimizing incidental collection of U.S. persons’ communications. It outlines specific rules, approvals, and exceptions for using SIGINT tools, foreign intelligence tasks, and cases involving ambiguous or dual-use selectors like WikiLeaks, The Pirate Bay, and foreign actors operating through U.S. infrastructure.
-
Elegant Chaos: collect it all, exploit it all [pdf]
This document outlines a classified project, "Elegant CHAOS," focused on advanced signals intelligence (SIGINT) collection, processing, and analysis using cutting-edge technologies like automated surveys and modems. It highlights goals such as real-time data analysis, collaboration with international partners, and integration with cloud-based platforms to manage massive data. See also the document with notes (by GCHQ): "Elegant-Chaos-collect-it-all-exploit-it-all-plus"
Programs: ELEGANTCHAOS · ASPHALT · ASPHALT-A · ASPHALT-A-PLUS · STORMFORCE · DARKQUEST · TORUS · XKEYSCORE · MHS CLOUD · TURMOIL · TARMAC · ASDF · SLR · POPQUIZ · WEALTHYCLUSTER2 · TRAVELLINGWAVE · BILBOBADGER · DRINKYBIRD · GLOBETROTTER · MASTERSHAKE · REST · TRAFFICTHIEF · VENUSAFFECT · FOGHORN · MATCHMAKER · ROADBED · MOONPENNY · AMULETSTELLAR · CLOUD · WEALTHYCLUSTER · MVR · TINT · IPGEOTRAP · MARINA · PLUS -
FASCIA: The NSA's huge trove of location records [pdf]
FASCIA is the National Security Agency's enormous database containing trillions of metadata records, many with device locations, that are collected from a variety of sources. This document shows the volume and types of device-location data collected by the agency.
Related Documents
-
Special Source Collection - March 2013 [pdf]
A March 2013 NSA Special Source Operations briefing for the NSA Threat Operations Center outlining SSO's portfolio of surveillance programs, their legal authorities, and operational constraints — revealing that SSO accounts for approximately 60% of NSA content collection and 75% of metadata collection, and mapping all major collection programs across three access categories: foreign partner, corporate, and unilateral.
-
SSO News on MYSTIC SHELLTRUMPET for June 20, 2012 [pdf]
Internal NSA memo describing how SHELLTRUMPET, part of the MYSTIC program, works in conjunction with Real Time-Regional Gateway (RT-RG) and various Special Source Operations (SSO) programs to provide metadata access for RT-RG Texas. The system obfuscates source information and filters irrelevant metadata, enabling RT-RG Texas to efficiently process metadata from DANCINGOASIS and SPINNERET.
“Mission Creep: How the NSA’s Game-Changing Targeting System Built for Iraq and Afghanistan Ended Up on the Mexico Border” - The Intercept ↗ (2019-05-29) · “Norges hemmelige krig i Afghanistan” - NRK ↗ (2019-05-29) · Moltke, Henrik · Skille, Øyvind Bye · Døvik, Olav · Hansen, Ståle Ellingsgard · Fjørtoft, Martin -
Special Source Operations corporate overview [pdf]
A classified NSA training presentation providing an in-depth operational overview of the FAIRVIEW program — the agency's largest corporate partner surveillance program — detailing its collection capabilities, legal authorities, TAO shaping operations, targeting tools, and productivity statistics, while revealing that BLARNEY/PRISM was the single most productive SSO program in 2012 with 806 serialized product reports.
Programs: FAIRVIEW · BLARNEY · PRISM · RAMPART-A · RAMPART-I · RAMPART-X · RAMPART-T · RAMPART-M · DGO · WINDSTOP · MYSTIC · PINWALE · XKEYSCORE · TURMOIL · STORMBREW · OAKSTAR · OCTAVE · UTT · CADENCE · COWBOY · YANKEE · MAINWAY · TOYGRIPPE · BLACKPEARL · TWISTEDPATH · NUCLEON · DISHFIRE · SPYDER · CARBONFURY · NATIVEFLORA · UNICORNSANDWICH · CROSSEYEDBEAR · SCORCHERSIX · TROPICTHUNDER · DARKTHUNDER · STEELFLAUTA · RAGTIME · SWEETSMACK2 · SOURSMACK2 · EVILOLIVE · SCALEABLE · FLEXIBLE · MONKEYROCKET · SHIFTINGSHADOW · ORANGECRUSH · YACHTSHOP · ORANGEBLOSSOM · SILVERZEPHYR · BLUEZEPHYR · COBALTFALCON · RAGTIME-A · RAGTIME-C · SSO · SURREY“NSA Spying Relies on AT&T’s ‘Extreme Willingness to Help’” - ProPublica ↗ (2015-08-15) · “AT&T Helped U.S. Spy on Internet on a Vast Scale” - The New York Times ↗ (2015-08-15) · “The Wiretap Rooms: The NSA’s Hidden Spy Hubs in Eight U.S. Cities” - The Intercept ↗ (2018-06-25) · Angwin, Julia · Larson, Jeff · Savage, Charlie · Moltke, Henrik · Poitras, Laura · Risen, James · Gallagher, Ryan -
Special Source Operations [pdf]
This document provides an overview of the NSA’s Special Source Operations (SSO), characterizing the division as the premier "Cryptologic Provider" for intelligence derived from global high-capacity telecommunications. It outlines a three-tiered access portfolio—Corporate, Foreign, and Unilateral—designed to manage the "incredible challenge" of intercepting target communications across undersea fiber-optic cables that carry millions of simultaneous sessions.
Programs: RAMPART-A · WINDSTOP · BLARNEY · FAIRVIEW · STORMBREW · OAKSTAR · PRISM · RAMPART-I · RAMPART-T · SSO · RAMPART-X · ClanSIG · MYSTIC · LOPERS · WEALTHYCLUSTER-2 · DEEPDIVE · TURMOIL · FASCIA · PINWALE · MARINA · TRAFFICTHIEF · TWISTEDPATH · BLACKPEARL · ACS“How Secret Partners Expand NSA’s Surveillance Dragnet” - The Intercept ↗ (2014-06-18) · “NSA ‘third party’ partners tap the Internet backbone in global surveillance program” - Dagbladet Information ↗ (2014-06-19) · Gallagher, Ryan · Geist, Anton · Gjerding, Sebastian · Moltke, Henrik · Poitras, Laura -
PRISM/US-984XN Overview [pdf]
A 2013 NSA training presentation revealing that major U.S. tech companies — including Google, Facebook, Apple, Microsoft, and Skype — secretly gave the NSA direct access to their servers, allowing the agency to collect users' emails, chats, photos, and files in real time, with the FBI acting as intermediary.
Programs: PRISM · FAIRVIEW · STORMBREW · BLARNEY · OAKSTAR · UTT · PRINTAURA · PINWALE · NUCLEON · TRAFFICTHIEF · SCISSORS · MARINA · MAINWAY · FALLOUT · CONVEYANCE · REPRISMFISA · SSO“U.S., British intelligence mining data from nine U.S. Internet companies in broad secret program” - The Washington Post ↗ (2013-06-06) · “NSA Prism program taps in to user data of Apple, Google and others” - The Guardian ↗ (2013-06-07) · “L'ampleur de l'espionnage mondial par la NSA” - Le Monde ↗ (2013-10-21) · “NSA files decoded: Edward Snowden's surveillance revelations explained” - The Guardian ↗ (2013-11-01) · “No Place to Hide” - Metropolitan Books ↗ (2014-05-13) · “Dark Mirror: Edward Snowden and the American Surveillance State” - Penguin Press ↗ (2020-05-19) · Gellman, Barton · Poitras, Laura · Greenwald, Glenn · MacAskill, Ewen · Untersinger, Martin · Dance, Gabriel