SSO GHOSTMACHINE Analytics
Open PDFDocument date: -
Pages: 4
Genre: slides
Banner: TOP SECRET//COMINT//NOFORN
Classification: TOP SECRET COMINT
Dissemination: NOFORN
This document describes the NSA’s cloud analytics platform GHOSTMACHINE, used by Special Source Operations (SSO) to process and analyze massive volumes of data (e.g., from FASCIA) using Hadoop-based infrastructure. It’s essentially the backbone system for querying, correlating, and enriching surveillance data at scale — not just a single analytic, but the analytic/cloud environment powering many programs.
GHOSTMACHINE, FASCIA, CERF, POPTOP, DRT, SIGDASYS, JUGGERNAUT, SEADIVER, AST128, TUSKATTIRE, ASDF, BROKENRECORD, REDHAWK, SSO
People mentioned:
-
Countries:
-
Tags:
cloud analytics, cloud computing, big data, data storage, data deduplication, data correlation, telecommunications, metadata processing, Hadoop, data infrastructure, AST 128, SMA Tools, web operations center, SSO Dashboard, DNR Deduplicator, mission support, SSO, HDFS, distributed computing, SIGDASYS, metadata, bulk collection, OPC, DPC, call detail records, infrastructure, analytic tools
Identifier:
EFF_-_20131210-wapo-ghost_machine
Also known as:
nsa-ghost-machineACLU_-_2013-12-10_Ghost_Machine2013-12-10_Ghost_Machine
PDF SHA-256:
d178b4df88d6aa2d9ed57283a89fb89e80858b3fd96af4733be1b2930ec1f2f2
My notes
SSO GHOSTMACHINE Analytics
hadoop
[REDACTED]
SiteStore Infrastructure & Analytics
TOP SECRET//COMINT//NOFORN
Hitting The Wall
Fascia Limit?
Tuskattire Volume Totals
5,000M
4,000M
3,000M
2,000M
1,000M
0M
09-DEC-10
11-DEC-10
13-DEC-10
15-DEC-10
17-DEC-10
19-DEC-10
22-DEC-10
24-DEC-10
26-DEC-10
28-DEC-10
30-DEC-10
01-JAN-11
03-JAN-11
05-JAN-11
CERF
POPTOP
DRT
SIGDASYS
AST128
JUGGERNAUT
SEADIVER
TOP SECRET//COMINT//NOFORN
Data at a Larger Scale
27704057147161 hdfs://r05sv24:9000/in/asdf
27827753336125 hdfs://r05sv24:9000/in/fascia
- Terabytes are available to query in the cloud, not always available in corporate databases
- Real value comes in being able to correlate and enrich with multiple types of data
TOP SECRET//COMINT//NOFORN
SS7 Summary
Current SIGAD | Search | Info
Current SIGADs
SIGAD OPC/DPC Pairs
US-3127 671
US-3145 381
US-3167 1313
US-3180 2022
US-3190 135
US-3237 1816
US-3261 5
US-3273 91
US-3293 20
US-3300 39
US-3311 1
US-983 27
US-990 860
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
US-3300
easMap/data.xml
[REDACTED]
DYNAMIC PAGE - HIGHEST CLASSIFICATION IS
TOP SECRET//COMINT//NOFORN
SMA Tools Blog | WOC | SSO Dashboard| BROKENRECORD | DNR Deduplicator | Mission Support Hub | REDHAWK
TOP SECRET//COMINT//NOFORN
Published in the Same Report
(4 documents)
-
How the NSA Pinpoints a Mobile Device [pdf]
This document is an internal National Security Agency classification guide outlining the agency's abilities to intercept cellular communications.
Programs: ARCVIEW -
Summary of DNR and DNI Co-Travel Analytics [pdf]
The document outlines the NSA’s development and testing of multiple “co-traveler” analytics—sophisticated tools that analyze bulk mobile location data to identify people who travel together or follow similar patterns, even if they are not known intelligence targets. It details various analytic systems, their data sources (such as FASCIA, VLR, GCID), architectures, scoring methods, and future enhancements, illustrating a large-scale surveillance framework for mapping human movement and relationships.
Programs: CO-TRAVELER · CHALKFUN · TMI · PACT · RT-RG · SORTINGLEAD · FASCIA · Hectic Snare · DRT · Juggernaut · GM-PLACE · Fast Follower · MGSC · ELKPRINTS · VOICESAIL · CULTWEAVE · OCTAVE · OCTSKYWARD · SEDB · JEMA · The Café · UTT · ASDF · GHOSTMACHINE · CLOUDBASE · FASTSCOPE · GOLDMINER · CHET · GEOT · TMAC · SPCMA · MAPREDUCE · ACCUMULO · GAMBIT · ICREACH -
Location Cookies [pdf]
These slides contain excerpts from an April 2013 National Security Agency presentation detailing signal surveillance techniques and successes. They reveal that the NSA and its British counterpart, Government Communications Headquarters (GCHQ), use a Google-specific tracking cookie to pinpoint targets for hacking.
-
How the NSA verifies a target's location [pdf]
This is an excerpt from a transcript of National Security Agency training videos that describe how to verify the location of a targeted device. It explains how the NSA monitors different types of mobile signaling information known as HLR and VLR registrations in order to locate individuals. It also makes clear that the agency is able to use location tracking to ascertain whether a target is in the United States.
Related Documents
-
Summary of DNR and DNI Co-Travel Analytics [pdf]
The document outlines the NSA’s development and testing of multiple “co-traveler” analytics—sophisticated tools that analyze bulk mobile location data to identify people who travel together or follow similar patterns, even if they are not known intelligence targets. It details various analytic systems, their data sources (such as FASCIA, VLR, GCID), architectures, scoring methods, and future enhancements, illustrating a large-scale surveillance framework for mapping human movement and relationships.
Programs: CO-TRAVELER · CHALKFUN · TMI · PACT · RT-RG · SORTINGLEAD · FASCIA · Hectic Snare · DRT · Juggernaut · GM-PLACE · Fast Follower · MGSC · ELKPRINTS · VOICESAIL · CULTWEAVE · OCTAVE · OCTSKYWARD · SEDB · JEMA · The Café · UTT · ASDF · GHOSTMACHINE · CLOUDBASE · FASTSCOPE · GOLDMINER · CHET · GEOT · TMAC · SPCMA · MAPREDUCE · ACCUMULO · GAMBIT · ICREACH -
BOUNDLESSINFORMANT – Frequently Asked Questions [pdf]
This document outlines the NSA’s internal metadata analysis tool, designed to provide real-time visualizations and statistics on global SIGINT collection by country, organization, and site. It explains the tool’s purpose, data sources, technical architecture, views, data limitations, and the process for requesting feature enhancements, highlighting its role in cataloging surveillance activity without human intervention.
Programs: BOUNDLESSINFORMANT · ASDF · GAO · RAMPART-A · SPINNERET · GM-PLACE · FALLOUT · TUSKATTIRE · FASCIA · MUSCULAR · Terrestrial RF · MachineShop · TurkeyTower · CloudBase · GXT · ExtGWT · JUGGERNAUT · LOPER · FLAWMILL · SSO · MAPREDUCE“Boundless Informant: the NSA's secret tool to track global surveillance data” - The Guardian ↗ (2013-06-08) · “NSA-files repeatedly show collection of data «against countries» - not «from»” - Dagbladet ↗ (2013-11-22) · “New NSA Revelations: Inside Snowden's Germany File” - Der Spiegel ↗ (2014-06-18) · Greenwald, Glenn · MacAskill, Ewen · Becker, Sven · Gude, Hubert · Horchert, Judith · Müller-Maguhn, Andy · Poitras, Laura · Reißmann, Ole · Rosenbach, Marcel · Schindler, Jörg · Schmid, Fidelius · Sontheimer, Michael · Stark, Holger -
Tasking of SCALAWAG [pdf]
An NSA Special Source Operations memo describing bandwidth and latency problems caused by the SCALAWAG bulk voice collection program, and introducing the RETRO Tool — a capability allowing analysts to reach back up to 30 days into a buffer of recorded phone calls from an entire country to retrieve audio that was never individually targeted.
Programs: SCALAWAG · DOUBLEARROW · UTT · OCTAVE · NUCLEON · RETROSPECTIVE · RETROTOOL · RETRO · FASCIA · SSO -
SSO Collection Optimization [pdf]
The documents reveal that the NSA, in collaboration with GCHQ, accessed private internal networks of Google and Yahoo by intercepting data in transit between their data centers, without breaking into user accounts or servers. This involved analyzing proprietary formats and protocols, such as Google’s Gaia authentication system, using specialized tools to extract intelligence while operating primarily under foreign jurisdiction with fewer legal restrictions.
-
FAIRVIEW dataflow charts April 2012 [pdf]
This document outlines data flow diagrams that show how various types of sensitive communication, including voice, fax, and modem traffic, are handled through systems operated by the NSA and its partners. These partners include corporations, telecommunications companies, ISPs, and other entities involved in processing and managing the data through secure and classified channels. The document covers processing flows for various data types, metadata, and communications systems under NSA's control, detailing how data is routed, stored, and analyzed across different platforms and sites.
Programs: FAIRVIEW · SAGUARO · TURNSTILE · MESA · TITANPOINT · RIMROCK · MAILORDER · PINECONE · BLARNEY · STONEGATE · TINSEL · CONVEYANCE · FISHWAY · NUCLEON · PINWALE · SCISSORS · COURIERSKILL · DRIFTWOOD · ASDF · FALLOUT · MARINA · LOPERS · WEALTHYCLUSTER · PUPR · CLEARSIGHT · MIDFIELD · CADENCE · BUGCATCHER · WAYLAND · DESOTO · GATEKEEP · HIGHDECIBEL · XKEYSCORE · YANKEE · TOPROCK · FASCIA · DISHFIRE · SEAGULL · BIGDIPPER · CORPORATE · LILDIPPER · HAVASU · WEALTHYCLUSTER2 · RAGTIME · TUMULT · TATTOO · DARKTHUNDER · ICYTWINS · TML-STATUS · WATERFRONT · VULCANDEATHGRIP · STEELFLAUTA · UNITY · KEYCARD · UTT · BTUBE · ISLANDTRANSPORT · SSO · MAINWAY · SSDM · TUBE · TITANPOINTE · TURMOIL · VPCS“NSA Spying Relies on AT&T’s ‘Extreme Willingness to Help’” - ProPublica ↗ (2015-08-15) · “AT&T Helped U.S. Spy on Internet on a Vast Scale” - The New York Times ↗ (2015-08-15) · “Titanpointe: The NSA’s Spy Hub in New York, Hidden in Plain Sight” - The Intercept ↗ (2016-11-16) · Angwin, Julia · Larson, Jeff · Savage, Charlie · Moltke, Henrik · Poitras, Laura · Risen, James · Gallagher, Ryan