Real-Time Regional Gateway Cloud Analytics for Forward Users
Open PDFDocument date: 2012-05-10
Pages: 33
Genre: slides
Banner: TOP SECRET//SI//REL USA,FVEY
Classification: TOP SECRET SI
Dissemination: REL USA, FVEY
This document outlines the NSA’s Real-Time Regional Gateway (RT-RG) program, a global cloud-based signals intelligence system designed to provide near real-time data analysis and targeting support for military and intelligence operations. It details the transition to scalable cloud architectures such as GHOSTMACHINE and Cloudbase, enabling massive data retention, exploitation, advanced geolocation, and automated analytics for tracking high-value targets across multiple theaters, including Afghanistan, Iraq, and the Horn of Africa.
My notes
TOP SECRET //SI// REL USA,FVEY
Real-Time Regional Gateway Cloud Analytics for Forward Users
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
[REDACTED]
TOP SECRET//SI//REL USA, FVEY
REAL-TIME REGIONAL GATEWAY
SECRET//REL USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
BLUF
RTRG:
...brings near real-time intelligence to the warfighter
..."grew up" supporting operations in Iraq
...RTRG is now a global architecture
...leveraging the emerging cloud architecture to answer questions we have not been able to do before
2
SECRET//REL USA, FVEY
TOP SECRET//SI//REL USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
RTRG in Afghanistan
(U)
(S//REL)
(S//REL) Area 82 – Bagram Air Base Home of RTRG AF1 & gmTote
- 20 days data retention from AFPAK
- 200-600 daily users
- New upgrades include two systems in Kabul
(TS//REL)
Mission Areas: Tracking high-value targets (HVT), Counter-Insurgency (COIN), Counter-IED (CIED)
Organizations Using RTRG:
- CSG* Afghanistan
- U.S. Marine Corps (USMC) 1st and 2nd Radio Battalions
- U.S. Army SIGINT analysts at BCT* level
- U.S. Air Force National Tactical Integration
- Jalalabad Fusion Cell (USMC)
- S2 TOPI
- NSA-G SWAN Counternarcotics Team
- All special operations task forces
(TS//SI//REL)
“RTRG is the most significant SIGINT support to the war fighter in the last decade"
- General David Petraeus
"USSOCOM has enduring and critical needs for the tools and data that RT-RG provides"
- Admiral William McRaven
*CST- Cryptologic Support Team
*BCT - Brigade Combat Team
*CSG - Cryptologic Support Group
TOP SECRET//SI//REL USA, FVEY
3
TOP SECRET//SI//REL USA,FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
CSG Afghanistan Statistics
- In 2011, RTRG in Afghanistan
- Played a key role in 90% of all SIGINT developed operations
- Leading to 2270 capture/kill operations
- 6534 enemies killed in action
- 1117 detainees
4
TOP SECRET//SI//REL USA, FVEY
TOP SECRET//SI//REL USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
RTRG in the Gulf and Horn of Africa
Monitoring Iranian Navy (IRIN) in Straits of Hormuz
QESHM ISLAND
HIDARABI ISLAND
HENGAM ISLAND
BAR-E-LANGEH
FARUR ISLAND
GREATER TUNB
BANI FARUR
ABU MUSA
JASH
USS GEORGIA
SSGN 729
Photo of IRIN vessel from US Navy aircraft, located by RTRG
Supporting CENTCOM Maritime (NAVCENT)
Navy Information Operations Command- Bahrain (NIOC-B)
Missions supported:
- Iran, Yemen, Persian Gulf
- Recent successes include monitoring of Iranian naval assets
(TS//SI//REL)
RTRG Afloat on subsurface platform USS Georgia (SSGN-729)
(TS//SI//REL)
Missions supported:
- Horn of Africa: In first week of mission, system received 31 million GSM events, leading to 10 high-value target voice ID, and 90 tactical tip-offs
TOP SECRET//SI//REL USA, FVEY
5
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
RTRG Global Operations
NSA/CSS COLORADO
TECHNICAL SIGINT
(U//FOUO)
(U//FOUO)
US-2 - Denver
Global Maritime & ELINT
NATIONAL SECURITY AGENCY
UNITED STATES OF AMERICA
US-1 Fort Meade
Mission Assurance
IQ2 Iraq*
COIN, CIED
AF-1,5 Bagram & Kabul
COIN,CIED
(S//REL)
USS Georgia & USS Florida
US-3 JFCOM
Joint Forces Command
BH-1 NIOC** Bahrain
AFRICOM, EUCOM, PACOM
GE-3 Germany**
AFPAK
Continuity-of-Operations
KO-1,2 USFK** Pangyo
PACOM & North Korea
US-8 NSA-Texas
Counternarcotics, Maritime, Mexico & SOUTHCOM Support
Texas Cryptologic Center
NSA/CSS TEXAS
* In draw-down
** ECC - NSA European Technical Center
USFK-U.S. Forces, Korea
NIOC - Navy Information Operations Command
TOP SECRET//SI//REL TO USA, FVEY
6
UNCLASSIFIED//FOR OFFICIAL USE ONLY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Outline
- RTRG Mission Overview
- RTRG System: Today and Tomorrow
- Target-Centric and Network-Centric Cloud Analytics
- Future Work
7
UNCLASSIFIED//FOR OFFICIAL USE ONLY
SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Current RTRG Architecture
RTRG System
Goldminer
Metadata Search
GeoT
Geospatial Tools
Agent Logic
Alerting Tools
Sharkfinn
Selector Enrichment
SKS
Report / Doc Manager
Panopticon
Target Management
Supporting Tactical Users
Forward Data Centers
Services Layer: Web services, authentication, auditing
Publish and Subscribe Messaging
Oracle relational database & dimensional data model
Ingest and Enrichment Pipeline:
Flexible, high-speed architecture for parser and data processors
Data Feeds:
- DRT
- JUGGERNAUT
- CIDNE
- LOPERS
- TALIS/MATTERHORN
- AIRHANDLER
- LYCANTHROPE
- KL
- VOICESAIL
- RETURNSPRING
... and a growing list of others*
(DNR, DNI collect, tipping and reporting)
A successful architecture for several years. Demand for more data feeds, longer retention, and data-intensive analytics has driven RTRG to seek new solutions
*Based on Afghanistan RTRG data flow
SECRET//SI//REL TO USA, FVEY
SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
RTRG Data Challenges
Current Challenges
- Data Storage & Retention
- "Patterns of Life” analysis needs require 6+ months of data from world-wide collection
- A typical system has capacity for only 4-6 weeks of regional data (~90% user queries are within seven days of “now”)
- Data Use & Computation
- Analytic processes should make maximum use of all available data to find small signals
- Relational databases are unsuited to sophisticated analytics such as correlation and matching
- Data & Technology Heterogeneity
- New types of data must be added to the system continually
- With traditional databases, schema modifications are difficult
- Exotic data management solutions are difficult to adopt due to limited expertise
SECRET//SI//REL TO USA, FVEY
UNCLASSIFIED//FOR OFFICIAL USE ONLY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Cloud Architectures for Analytics
NATIONAL SECURITY AGENCY
UNITED STATES OF AMERICA
hadoop
Apache
Cloudbase
Scalable BigTable Implementation with Security
Emerging NSA Cloud Reference Architecture is well-suited for developing analytics on intelligence data
Scalable: Distributed file systems and databases are built on clusters of commodity hardware, leveraging open source projects and industrial solutions
Computable: The MapReduce programming model simplifies writing efficient parallel computations that operate over large volumes of data
Flexible: Cloud technologies enable flexible schema and leverage large open-source efforts
UNCLASSIFIED//FOR OFFICIAL USE ONLY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Scalability & Computation
Data Challenges in AFPAK
Current RTRG (AF1)
- Current database is 27 terabytes (TB)
- Retention is ~30 days
Future Cloud enabled system
- Even a modest cloud system (3 rack) for storage will be at least 125 TB of storage
- 5x increase in available space
- Actual retention improvement depends on how the space resources are allocated
Cloud supports more data feeds & more days of historical data
Analytic Challenges from Iraq & AFPAK
- Many analytics used by RTRG are based on R6 SORTINGLEAD event summaries
- Event summaries were originally created on relational databases
- Collection increased dramatically, and a mapreduce implementation was needed
- For new analytics with present day collection volumes, a practical parallel execution model is crucial
Cloud supports large-scale analytics
TOP SECRET//SI//REL TO USA, FVEY
11
TOP SECRET//SI//REL USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
NSA Cloud Computing Enterprise
- 5-12 racks commodity hardware
- 150+ data nodes
- 16 GB RAM each
- Apache Hadoop
- 100s of terabytes of storage
- Stores 10s to 100s of billions of events
- NSA Cloudbase
- 100s of nodes serving BigTable implementation
- Stores 100s of billions of entries
gmAPPLE
Yakima
gmCAVE
gmPEACH
Utah Language Center
gmDEN
ADF-C
gmMOUNT
gmLIGHT
gmBALTIC
gmPLACE
gmPARK
gmHALO
gmALAMO
gmCARBON
Europe
gmROYALE
gmREGAL
gmGEIST
Iraq
gmIDEA (Camp Victory)
Afghanistan
gmTOTE (Bagram)
Korea & Japan
gmPEN
S. Korea
gmPAC
Misawa, Japan
gmCALM
Camp Humphreys
S. Korea
Australia
gmMATE
(S//REL) (Alice Springs)
GHOSTMACHINE is a data-intensive cloud system with many fielded instances worldwide
Hadoop clusters have been demonstrated with 10s of petabytes and over 10,000 cores
12
Map does not include all GHOSTMACHINE/SiteStore systems
TOP SECRET//SI//REL USA, FVEY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Today's "Hybrid” Architecture
GTM
GeoT
SKS
PANOPTICON
Applications
Services Layer
Publish and Subscribe Messaging
Relational Database
Ingest and Enrichment
hadoop
Large scale data preprocessing and analysis
Cloudbase
Scalable BigTable Implementation with Security
Massive storage and fast record lookup
GHOSTMACHINE
RTRG and GHOSTMACHINE systems are paired with one another:
MapReduce analytic results are fed back to RTRG relational database
TOP SECRET//SI//REL TO USA, FVEY
13
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Tomorrow's Architecture
User interfaces
- W WidgetFramework
- GTM
- GeoT
- CHET
- H&S
- SKS
- PANOPTICON
- Web Tier
Services
- Application Services
- Alerting
- Ingest
- Real time Enrichment
- Analytics
- Streaming-based
- Analytics
- Map Reduce-based
Data
- Data Storage (NFS)
- Secure Data Access Services
- Casport, Wavelegal, etc.
- Relational Store
- Enrichment, User, & Auxiliary Data
- Cloudbase
- Event Data and Analytic Results
Operating System
- Java VM
- Java VM
- Operating System
- Hardened RH Linux
- Operating System
- Hardened RH Linux
- OpenStack VM
- Utility Cloud
- Hardware Layer
- Data/Analytic Cloud
- Hardware Layer
Data Storage (HDFS)
The Cloud will bring new data-intensive capabilities, support existing missions, and align RTRG installations with emerging NSA and IC standards
TOP SECRET//SI//REL TO USA, FVEY
UNCLASSIFIED//FOR OFFICIAL USE ONLY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Outline
- RTRG Mission Overview
- RTRG System: Today and Tomorrow
- Target-Centric and Network-Centric Cloud Analytics
- Future Work
15
UNCLASSIFIED//FOR OFFICIAL USE ONLY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Analytics Overview
Target Development
Co-Travelers & Meetings
Contact Similarity
High Priority
Untasked Numbers
Graph & Network
Graph Triage
Handset Swap
Target Enrichment & Disambiguation
Beddown
Layercake
Heatmaps
Geo-Spatial
The data-intensive computing capabilities in the system enables a set of graph/network analytics and target development analytics
TOP SECRET//SI//REL TO USA, FVEY
16
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Meeting Target Development Challenges
Past Analysts manually queried multiple, independent repositories, aggregating results in Excel, taking hours or work for search and refinement
Now RT-RG provides a streamlined, integrated workflow saving analyst effort
Tips
Selectors
GIST Report
Detainee Reports
Target Development
Event Type comms.gsm
Event Subtype call.*
Selectors IMSI:12345
NAI Kandahar
?
Submit Query Cancel Query Export To File...
Metadata Search
[REDACTED]
Profile Reports Network Changelog Recommendations
OBJ DEWEY BEACH ASSOCIATE
Bottom Line Pech Valley INS possibly in charge of a "company" sized element. Led the attack
Last Active: (TS//SI//REL TO USA, FVEY) (IMEI)
9 hours ago
Last Location: (TS//SI//REL TO USA, FVEY) (IMEI)
RTRG-Afghanistan Targets 9 hours ago Geo
Processor: AROCC
Last Report: (TS//SI//REL TO USA, FVEY) (MSISDN) view report
5 months ago
VID/SID: 1 year ago
Collaboration
TOP SECRET//COMINT//REL TO U
CST 17 FOB FENTY (PAN): AF IN
(Khaksar)
Thu, 10 May 2012 15:55:20 Zu
TOP SECRET//COMINT//REL TO
NZL//20320108
ASSET: JUGGERNAUT
Geospatial Alerting
Afghanistan/Insurgency:
Taliban Leaders in
Chaman, Pakistan,
Command Insurgent
Operations throughout
Southern Afghanistan-A
SIGINT Assessment (S//
SI//REL TO USA, FVEY)
Afghanistan: Influence of
Chamay on RC South,
Juge-July2010
IIR 6 223 0164 12/DOME
EXPOSE/DRAGON FIRE-
EXPLOITATION OFPAGE 3
Report Text for:
02992282
2017302 AUG 10
FM USJ-800
T O P S E C R E T UMBRA US/UK/CAN/AUS/NZ EYES ONLY
SERIAL: 3/CY/507710-10
SUBJ: Afghanistan/Insurgency: Taliban Leaders in Chaman,
Pakistan, Command Insurgent Operations throughout
Southem
Afghanistan A SIGINT Assessment (S//SI//REL TO
USA, FVEY)
20100119 08) BIMINI
(10020-3AFG)
_Nokia1650
kc taliban ircurg
Enrichment
Outcomes
Geolocation
Capture
TOP SECRET//SI//REL TO USA, FVEY
17
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Target Development with Meetings
Who is at the same UCELLID at the same time?
Manual Process
- Take your selector and query for every unique location he has been and at what time
- Query for other selectors who have been at the same places at the same times (impossible or painful)
- OR compare to another known set of selectors to find overlap (excel / ArcGIS / JEMA) (limiting to what you know)
- Summary statistics on the matching IMSIs using excel or ArcGIS
Cloud Process
- Pre-calculates all UCELLID overlaps between tasked selectors
- Simply query your selector in cloud-generated QFD and view summary statistics
Counts
Meetings
Different Dates
Different Locations
Count
Selectors
TOP SECRET//SI//REL TO USA, FVEY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Target Development with Co-Travelers*
Is there a pair traveling together?
Manual Process
- You could use the same manual process from Meetings, however, this would not find co-travelers on different networks
- Manual comparison of pairs of known selectors is possible with ArcGIS or similar spatial tools - You must know the pairs up front
Cloud Process
- Measures miles-per-hour (MPH) between tasked selectors as they move around.
- Low average MPH = co-traveling.
- Simply query for your selector to view statistics on average MPH, days calculated, etc.
*Also known as "Sidekicks"
TOP SECRET//SI//REL TO USA, FVEY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Meeting Network Challenges
Past Manually query multiple repositories and build network with Analyst Notebook (ANB) - amount of labor can be prohibitive
Now RT-RG tools exist for contact chaining for selector-to-selector & selector-to-report graphs, with more analytics and tools to come
Southwest Asia/Narcotics:
Senior Traffickers from the AbdulAlim Ishaqzai Narcotics Network Located in HelmandProvince, Afghanistan, After Bahram Chah Raid, 5 November2010 (S//REL TO USA, FVEY)
UIM[REDACTED]
ASKS[REDACTED]
[REDACTED]IF HE
HAS HIS MISSING BAG.
[REDACTED]
[REDACTED]
[REDACTED] GETS
PASSED[REDACTED]
PHONE NUMBER.
Selector-to-Report Graph from Enrichment
Search graph...
Calls:2
Calls:3
[REDACTED] IMSI)
// 2040-06-18 // RC EAST
TIPPER // OBX GIANT
SQUID //
selectors
VF // 2010-06-16 // VHF
N2K TIPPER // OBJ GIANT
SQUID //
VF//2010-06-15//CSG
BAGRAM SELECTORS
TIPPER//OBJ GIANT
SQUID//
calls: 718
PREPAID PREPAID
PREPAID
THIS TACREP IS REL
USA AUS,CAN,DEU,DNK,F
RA,GBR,NLD,NO,NZL,SW
E
OBJ COOPER (OMEGA 10)
[REDACTED]
targets
[REDACTED]IMEI)
[REDACTED]
MAINWAY graph in RTRG UI
TOP SECRET//SI//REL TO USA, FVEY
20
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Graph Analysis with Furious Chainsaw
- Graph representation is natural for DNR
- Result is Furious Chainsaw
- Prototype on Cloudbase
- Now supports contact chains and trends
- Will support other graph analytics in the future
- Triage capability for forward users to complement Enterprise databases
- Enables chaining and other analytics, provides foundation for graph algorithms
Metadata matrix in Cloudbase supports fast graph traversal
Graph View in Renoir – but many other analytics are possible
TOP SECRET//SI//REL TO USA, FVEY
21
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Graph Triage: Multiple Views
Contact Neighborhood
Ozone Widget
Friends and Family
IMS[REDACTED]
test
[REDACTED]
[REDACTED]
[REDACTED]
[REDACTED]
[REDACTED]
[REDACTED]
[REDACTED]
[REDACTED]
[REDACTED]
[REDACTED]
ANB
Daily Call Trends in Panopticon
PANOPTICON Create Profiles Partners
Details
Relationships
Discussions
Bar
Event Count
100
80
60
40
20
0
0 1 2 3 4 5 6 7 8 9 1011121314151617181920212223
Hour of Day (Zulu)
Top Associates
Ozone Widget
Top 100 Associations
Top 100 Associations | Event Count
[REDACTED] | 81
[REDACTED] | 63
[REDACTED] | 61
[REDACTED] | 59
[REDACTED] | 49
[REDACTED] | 43
[REDACTED] | 37
[REDACTED] | 34
[REDACTED] | 27
[REDACTED] | 24
Showing 1 to 10 of 100 entries
Previous Next
Weekly Call Trends
Ozone Widget
Day of Week
Bar
Event Count
200
150
100
50
0
Mon
Tue
Wed
Thu
Fri
Sat
Sun
Day of Week
DNR graphs in Furious Chainsaw tables in Cloudbase support a wide range of fast queries and analytics
TOP SECRET//SI//REL TO USA, FVEY
22
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Unstructured Data Exploitation
SHARKFINN
- Selector extraction, normalization, and enrichment
- Flexible free-text query interface
- Graph, text, and spreadsheet output formats
SKS Structured Knowledge Space
- Entity extraction (people, organizations, times, geos)
- Keyword, faceted, and people search
- Document clustering
- Arabic name expansion
sharkfury
- Integrated data ingest using Niagara Files (NiFi)
- SharkQuery: search by selector, entity, location, and keyword
- SharkDocs: query, sharing, and collaboration on user uploaded documents
- Visualization of results in query overview, table, graph, and map
- Cloudbase and HDFS for scalable text analytics platform
TOP SECRET//SI//REL TO USA, FVEY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Target Development - SharkFinn / SKS
SKS
"OBJ SMITHERS" AND OBJECTIVES: OBJ TOPPER
Previous 30 Days
1.Perform keyword search with seed OBJ on report library
OBJECTIVES
OBJ SMITHERS (19)
OBJ TOPPER (19)
OBJ SPRINGFIELD (12)
OBJ APU (6)
OBJ GROUNDSMAN WILLY (6)
OBJ WIGGUM (4)
OBJ MAYFIELD (3)
OBJ NEVIS (1)
OBJ TRAJAN (1)
- Finding co-occurring objectives
SELECTORS
[REDACTED]
[REDACTED]
[REDACTED]
[REDACTED]
[REDACTED]
[REDACTED]
[REDACTED]
3.Selector co-occurrence
4.Additional Search & Filtering
SELECTOR
1 [REDACTED]
2 [REDACTED]
3 [REDACTED]
4 [REDACTED]
5 [REDACTED]
6 [REDACTED]
7 [REDACTED]
8 [REDACTED]
9 [REDACTED]
10 [REDACTED]
11 [REDACTED]
12 [REDACTED]
13 [REDACTED]
- Correlate found selectors with Octave/UTT and Panopticon
- Export selector & Report Graph
TOP SECRET//SI//REL TO USA, FVEY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Meeting Geolocation Challenges
Past Analysts manually correlated locations using map viewers or spreadsheets, aggregating data from multiple sources
Now Analytics and alerts push target information by subscription
412.001.00078.27161
UCELLIDS
412.001.00076.11402
/01
/10
/16
/22
/23
/24
/25
/26
/27
/28
/29
3/04
3/05
3/06
3/07
3/08
3/09
3/10
3/11
3/12
3/13
3/14
3/15
Day
RT-RG pattern of life analytic for target location cues
tio field
First Event
Last Event
MAHIM
Alükh
Qal'eh-ye
Adam Khan
Vazil
Niazi
Shiwak
Shiv
Analysts are notified by alerts, based on:
- Geospatial NAI*
- Tasking status
- SMS content
- Selectors, callsigns, frequencies
- ...
Op ATTENTION INS RC-C Kabul
(PAN):UNID ASW SUICIDE VEST KABUL 3
Thu, 10 May 2012 17:48:12 Zulu
TOP SECRET//COMINT//REL TO USA, AUS,
CAN, GBR, NZL//20320108
ASSET: LOPERS
EVENT TYPE: comms.gsm/call.orig
SIGAD: US-3309
PDDG: K2
SSD: L2
POEI: (Missing)
CASN: GCCA0153L5C0001
SELECTION MODE: STANDARD
COLLECTION AUTHORITY: EO12333
Target Location
Approx. Geo: 42S WD 21111 18789
UCELLID: [REDACTED]
Geo Data Source(s): GSM Geo-Sigdev
4 Item(s)
25
TOP SECRET//SI//REL TO USA, FVEY
- NAI = Named Area of Interest
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Target Geolocation with Bed Down
Find the most consistent location of the day's first/last event
First Events
+
Last Events
=
Bed Down
Manual Process - One Selector At a Time
- Query all events for your selector.
- Mark first and last events manually.
- OR
- Enter in a tool like CheekyMonkey to view gaps in activity.
- Slow process to do one selector at a time
Cloud Process- All Tasked Selectors
- Pre-calculates first and last events in local time for ALL selectors.
- Will calculate estimated Bed Down at query time.
- Can query multiple selectors in seconds, find common overlap.
TOP SECRET//SI//REL TO USA, FVEY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Target Geolocation with LayerCake
LayerCake - find geospatial overlap of a set of targets.
Manual Process
- Query all events for your selectors.
- Display events spatially on mapping software (impossible to view polygon overlaps).
- Rasterize and do "raster math” to determine max overlap. (very complex and expensive task in most GIS tools).
Cloud Process
- Pre-calculates unique locations visited for ALL selectors.
- Raster heat maps drawn at query time.
TOP SECRET//SI//REL TO USA, FVEY
TOP SECRET//SI// REL USA,FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Spatial Visualization of Data
Using MapReduce, analytics can count and aggregate over large number of daily events to give a multi-resolution spatial visualization of the data
Google Earth
Cloudbase Tables for Geo Data
MapReduce for Binning, Counting
Event Data
Analytic Data Flow
Synoptic View
TS//SI//REL
Call volume map of RTRG (AF1) collection (A day in Dec, 2011)
Fewer DNR events
Fewer events
28
TOP SECRET//SI// REL USA, FVEY
TOP SECRET//SI // REL USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Spatial Visualization of Data
Cloud enabled analytic allows viewing the spatial data at many levels-of-detail
“Meso-scale” View
Detail View – “Mashup” with heatmap
Kabul
Peshawar
Rawalpindi
Hyderabad
Karachi
SEDB
Faisalabad
Amritsar
Lahore
Srinagar
Heatmap of activity of a set of targets
[REDACTED]
ASSOC OF CH
[REDACTED]
NDS D124 DEP CHIEF[REDACTED]
ΜΟΙ ΖΟΝΕ 101 AS
KARZAI
AIDE TO MO
OCT 829
UPS AND INDIV
MOI CT OFFI
وز محشر شفا خواهی تمام مومنین است.
We will arrive on saturday
محمد در عرب سردار دین است چراغ آسمان
شمع زمین است. به فردای قیامت و
And Z the day of judgment
healing desiring all believers is.
We+will+arrive+on+saturday+
[REDACTED]
commander religion is: light
heaven/sky candle land is: to the
resurrection-day R
Directions: To here - From here
MFA
MOD SECRETARIAT
MP
DEVELOPMENT ICW IRGC QF DEPT 409
KHALEQ
MP
Using Cybertrans translation service on SMS messages, integrated with heatmaps
Fewer calls
More calls
TOP SECRET//SI // REL USA, FVEY
29
UNCLASSIFIED//FOR OFFICIAL USE ONLY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Outline
- RTRG Mission Overview
- RTRG System: Today and Tomorrow
- Target-Centric and Network-Centric Cloud Analytics
- Future Work
30
UNCLASSIFIED//FOR OFFICIAL USE ONLY
TOP SECRET//SI//REL TO USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
RTRG Tomorrow
- Improved DNI capabilities; focus on convergence
- Integrating active SIGINT capabilities
- Increased CT and expeditionary capabilities
- Better tools for faster analytic development
- Incorporation of content analysis and HLT capabilities
- Improved integration between target and population analytics
31
TOP SECRET//SI//REL TO USA, FVEY
TOP SECRET//SI// REL USA, FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
RTRG Planned Cloud Instances
NSA/CSS GEORGIA
FORT GORDON
gmSeminole
NSA Georgia
NAVY INFORMATION
OPERATIONS COMMAND BAHRAIN
NETWARCOM
gmGulf
NIOC-Bahrain
NCR AFGHANISTAN
SEMPER VIGILANS SEMPER PARATOS SEMPER VOLENS
gmZilla
Kabul
TOP SECRET//SI// REL USA, FVEY
32
TOP SECRET//SI//REL USA,FVEY
NATIONAL SECURITY AGENCY
SIGDEV
SIGINT DEVELOPMENT
Summary
- RTRG has been a successful regional data store and exploitation system for COIN, CIED and other missions
- Moving to NSA Cloud infrastructure
- More historical data
- Deeper analysis using parallel programs
- Allows for more flexible deployments to IC, DoD service installations
- Continuing to support advanced analytics for current and future operations
TOP SECRET//SI//REL USA, FVEY
33