Elegant Chaos: collect it all, exploit it all
Open PDF +1 other copyDocument date: -
Pages: 23
Genre: slides
Banner: SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
Classification: SECRET
Dissemination: REL TO USA, AUS, CAN, GBR, NZL
This document outlines a classified project, "Elegant CHAOS," focused on advanced signals intelligence (SIGINT) collection, processing, and analysis using cutting-edge technologies like automated surveys and modems. It highlights goals such as real-time data analysis, collaboration with international partners, and integration with cloud-based platforms to manage massive data. See also the document with notes (by GCHQ): "Elegant-Chaos-collect-it-all-exploit-it-all-plus"
My notes
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
Elegant CHAOS
Project Leads: [REDACTED]
Code Support: [REDACTED]
For more up-to-date information please visit this URL:
[REDACTED]Elegant_Chaos
For recent developments please contact the authors.
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
Field Site Responsibilities
Sniff it All
Know it All
Collect it All
Process it All
Exploit it All
Partner it All
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
For Better Or Worse...
Collection access is increasing,
- software modems (ASPHALT/A-PLUS)
- new hardware/software solutions (STORMFORCE modems and DARKQUEST auto-survey)
- new physical capacity (TORUS antennae)
Processing capabilities are increasing,
- JCE, TINT, XKS Deep Dive
But size of analytic workforce is not!
- more resources = more resource management
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
New Collection Posture
Sniff it All
Torus increases physical access
Know it All
Automated FORNSAT survey - DARKQUEST
Collect it All
Increase volume of signals: ASPHALT/A-PLUS
Process it All
Scale XKS and use MVR techniques
Exploit it All
Analysis of data at scale: ELEGANTCHAOS
Partner it All
Work with GCHQ, share with Misawa
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
ELEGANTCHAOS Goals
- Goal: perform basic, time-sensitive analysis on all of MHS collection
- Goal: create a prioritized list of signals (case notations) in our viewing arc
- Goal: use this list to automatically drive collection as collection capabilities increase
- Offshoot goal: create a product that analysts and collection managers can use to see into the system
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
ELEGANTCHAOS + Cloud
The MHS Cloud provides an excellent platform for this project:
- data ingest, normalization, tagging
- access to SIGINT data from various processors, from sustained mission + survey
- access to a huge body of enrichment data
- processing, storage, and web-hosting
- considering decoupling these parts...
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
ELEGANTCHAOS In Context
COLLECTION
FRONT END RF Collection
STORM FORCE
DARK QUEST
ASPHALT
Collection Distribution
WC
XKS
METADATA
CLOUD
TURMOIL
METADATA
CLOUD
TARMAC
PPF
SLR
CLOUD
CLOUD
Multiple QFDs
EC GUI
ENRICHMENT
Open Source
Target Network
Service
IP Geo
System Status
Σ
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
EC System Overview
1 Data
SIGINT
MHS CLOUD SERVERS
enrichment
2 EC Code
QFD
questions
QFD
score log
QFD
QFD
QFD
QFD
QFD
QFD
3 GUI
MySql tables and code
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
Data Sources (May 2011)
SIGINT Feeds
- XKEYSCORE
- ASDF (Turmoil LIVE)
- SLR (TARMAC)
- POPQUIZ (Turmoil DEV)
- WEALTHYCLUSTER2
Enrichment Feeds
- IPGeoTrap
- TRAVELLINGWAVE Scores
- BILBOBADGER Daily Summaries
- Target Network Service list + CNO Target list
- DRINKYBIRD monitoring info
- GLOBETROTTER OH Geo
- MASTERSHAKE Geo
- Quantumable Case Notation list
Event counts over a 12-hour period.
Total events: 335,663,981
TU/Live
TU/Dev
SLR
WC2
XKS
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
Analytic Questions (May 2011)
Target
- Dictionary hits
- Target Networks
- TNS, TW, CNO
- PLUS Reports, CRNs, etc.
Technology
- [REDACTED]
- VPNs
- Twitter, Facebook, VoIP
- CNO behavior
Miscellany
- Modem Capacity
- Paired Links
- Quantumable
Location
- IP-based
- MAC-based
- Geo-based
- Surge Countries
- Libya, Egypt, Afghanistan,
Syria, Yemen, Ivory Coast,
etc.
- Libya, Egypt, Afghanistan,
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
Questions & Scoring
- Each question is represented by a SQL query applied to one or more QFDs
- QFDs are case notation-based repositories of signal information
- eg, IPs and registries for all case notations
- eg, category hits for all case notations
- eg, GLOBETROTTER geos for all case notations
- All questions are asked once per day across all case notations
- Points are assigned to each question based on current analytic priorities
- Points for any particular question are "active" for a window of time (eg, 1 day, 7 days, 30 days)
- The sum of "active" points for a case notation, across all questions, forms the score
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
Interfaces
Different interfaces for different customers
- ELEGANTCHAOS GUI
- made for analysts to examine scores and the impact of the different questions
- eventually, control over the algorithms may reside here
- REST interface
- made for programmatic query, precursor to auto tasking
- DRINKYBIRD GUI
- made for collection personnel to determine if resources are available, easy to view what's on cover
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
EC GUI: Case Notation View
Elegant CHAOS
All Questions || All CASNs || Survey CASNs
Show 10 entries [Refresh]
| casen_id | casen | score | direction | notes | first_heard | last_heard | number_of_day |
|---|---|---|---|---|---|---|---|
| 1 | [REDACTED] | 157040 | forward | [REDACTED] | 2010-12-26 | 2011-03-17 | 25 |
| 101 | [REDACTED] | 55500 | forward | [REDACTED] | 2010-12-26 | 2011-03-17 | 25 |
| 5275 | [REDACTED] | 33160 | forward | [REDACTED] | 2011-03-16 | 2011-03-17 | 0 |
| 5288 | [REDACTED] | 31370 | unknown | [REDACTED] | 2011-03-17 | 2011-03-17 | 0 |
| 5285 | [REDACTED] | 26890 | unknown | [REDACTED] | 2011-03-17 | 2011-03-17 | 0 |
| 1110 | [REDACTED] | 17450 | return | [REDACTED] | 2011-01-03 | 2011-03-17 | 17 |
| 5292 | [REDACTED] | 15330 | unknown | [REDACTED] | 2011-03-17 | 2011-03-17 | 0 |
| 5277 | [REDACTED] | 14210 | unknown | [REDACTED] | 2011-03-17 | 2011-03-17 | 0 |
| 5289 | [REDACTED] | 12930 | unknown | [REDACTED] | 2011-03-17 | 2011-03-17 | 0 |
| 5283 | [REDACTED] | 12630 | unknown | [REDACTED] | 2011-03-17 | 2011-03-17 | 0 |
Showing 1 to 10 of 3,888 entries
First Previous 1 2 3 4 5 Next Last
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
EC GUI: Case Notation View
Mozilla Firefox
File Edit View History Bookmarks Tools Help
bucket elegantchaos
Tag This!! $$ MHS AM Notes Puzz Tiny Base Java API 1.6 cheat MySQL MySQL Syntax Tools misc RB EChaos Cloud Web Prog Mett
(U) XKS ... http...desc GM ROY... XKEYSC... FORESIG... (U) Whiz... [REDACTED] cl4r2hd2... Safari Bo... Private N... Matatu ht...sc DRINKYBI... Mass Tra...
TOP SECRET // COMINT // REL TO USA, FVEY
Elegant CHAOS
Not seeing any results?
All Questions || All CASNs || Survey CASNs
Show 10 entries Refresh
| casen_id | casen | score | direction | notes | first_heard | last_heard | number_of_day |
|---|---|---|---|---|---|---|---|
| 1 | [REDACTED] | 157540 | forward | [REDACTED] | 2010-12-26 | 2011-03-17 | 25 |
| 103 | [REDACTED] | 30000 | forward | [REDACTED] | 2010-12-20 | 2011-03-17 | 23 |
| 5275 | [REDACTED] | 33660 | forward | [REDACTED] | 2011-03-16 | 2011-03-17 | 0 |
| 5288 | [REDACTED] | 31870 | unknown | [REDACTED] | 2011-03-17 | 2011-03-17 | 0 |
| 5285 | [REDACTED] | 27398 | unknown | [REDACTED] | 2011-03-17 | 2011-03-17 | D |
| 1110 | [REDACTED] | 18950 | return | [REDACTED] | 2011-01-03 | 2011-03-17 | 17 |
| 5292 | [REDACTED] | 15830 | unknown | [REDACTED] | 2011-03-17 | 2011-03-17 | 0 |
| 5277 | [REDACTED] | 14710 | unknown | [REDACTED] | 2011-03-17 | 2011-03-17 | D |
| 5289 | [REDACTED] | 13430 | unknown | [REDACTED] | 2011-03-17 | 2011-03-17 | 0 |
| 5283 | [REDACTED] | 13130 | unknown | [REDACTED] | 2011-03-17 | 2011-03-17 | 0 |
Showing 1 to 10 of 4,106 entries
First Previous 1 2 3 4 5 Next Last
| topic | question weight | question score | first contributed |
|---|---|---|---|
| Egypt | 1000 | 7000 | 2011-02-18 |
| Qatar | 100 | 200 | 2011-02-27 |
| CNO Targets | 500 | 500 | 2011-03-17 |
| Paired Link | 50 | 200 | 2011-02-23 |
| Priority 3 CategoryHits | 50 | 2400 | 2011-03-16 |
| Jordan | 1000 | 1000 | 2011-03-17 |
| Afghanistan | 50 | 50 | 2011-03-04 |
| Libya | 1000 | 7000 | 2011-02-18 |
| Priority 4 CategoryHits | 20 | 600 | 2011-03-26 |
Points earned over time
Select a case notation
Observe questions which affected its score
x Find 192 Previous Next Highlight all Match case Phrase not found
elegantchaos.bucket.r6.r.nsa FoxyProxy Patterns
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
EC GUI: Question View
TOP SECRET//SI/TK//REL TO USA, AUS, CAN, GBR, NZL
Elegant CHAOS
Not seeing any results?
All Questions || All CASNo || Survey CASNO
Show 111 entries [Refresh]
| question_id | topic | weight | days_between_run | days_before_points_expire | description | active_score | total_score | expired_score | first_active_ |
|---|---|---|---|---|---|---|---|---|---|
| 16 | 1000 | 1 | 30 | 9000 | 16000 | 6000 | 2011 02 18 | ||
| 1 | TRAFFICTHIEF lit | 1000 | 1 | 7 | 1000 points given each day for each case notation with a priority 1 (TRAFFICTHIEF) tip seen | 28000 | 116000 | 118000 | 2011-03-15 |
| 17 | Libya | 1000 | 1 | 30 | 1000 points for each case notation with a Libyan IP address on the significant side of a link | 117000 | 216000 | 99000 | 2011-02-17 |
| 13 | Egypt | 1000 | 1 | 30 | Gives case notation 1000 points if the known significant/GAT side of the link has an IP address that resolves to Egypt | 62000 | 172000 | 110000 | 2011-02-16 |
| 14 | FORNSAT CRNo | 500 | / | / | 500 points for each case with a current FORNSAT CRN | 185000 | 185000 | 145000 | 2011-01-16 |
| 12 | Tasked Active Users | 150 | 1 | 7 | 150 points given for a case notation if it has Marina Daily Active Counts greater than 0 in the BILBOBADGER Daily Summary | 1350 | 16050 | 14700 | 2011-03-15 |
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
EC GUI: Question View
Mozilla Firefox
File Edit View History Bookmarks Tools Help
bucket elegantchaos
Tag This!! $$ MHS AM Notes Puzz Tiny Base Java API 1.6 cheat MySQL MySQL Syntax Tools misc RB EChaos Cloud Web Prog Mett
(U) XKS ... http...desc GM ROY... XKEYSC... FORESIG... (U) Whiz... WhizBan... WhizBan... cl4r2hd2... Safari Bo... Private N... Matatu ht...sc DRINKYBI... Mass Tra...
TOP SECRET // COMINT // REL TO USA, FVEY
Elegant CHAOS
All Questions || All CASNs || Survey CASNs
Show 10 entries
Refresh
| question_id | topic | weight | days_between_run | days_before_points_expire | description | active_score | total_score | expired_score | first_ac |
|---|---|---|---|---|---|---|---|---|---|
| CNO | |||||||||
| 23 | CNO Targets | 500 | 1 | 7 | Gives case notation 500 points for each day (up to 7 most recent days) in which target traffic hit on CNO XXS fingerprints | 44500 | 44500 | 2011-03- |
Showing 1 to 1 of 1 entries
First Previous 1 Next Last
Question 23
| Points | Score | Date |
|---|---|---|
| 500 | 2590 | 2011-03 |
| 500 | 3080 | 2011-03 |
| 500 | 15830 | 2011-03 |
| 500 | 18950 | 2011-03 |
| 500 | 13430 | 2011-03 |
| 500 | 500 | 2011-03 |
| 500 | 820 | 2011-03 |
| 500 | 8210 | 2011-03 |
| 500 | 500 | 2011-03 |
| 500 | 1580 | 2011-03 |
| 500 | 13130 | 2011-03 |
| 500 | 600 | 2011-03 |
| 500 | 2730 | 2011-03 |
| 500 | 9080 | 2011-03 |
| 500 | 820 | 2011-03 |
| 500 | 3400 | 2011-03 |
| 500 | 1640 | 2011-03 |
| 500 | 570 | 2011-03 |
Points earned over time
Filter based on a topic; select a question
Observe case notations which scored
x Find 192 Previous Next Highlight all Match case Phrase not found
elegantchaos.bucket.r6.r.nsa FoxyProxy Patterns
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
Focus Areas: Custom Views?
Home || All Questions || All CASNs || Survey CASNs || ASPHALT CASNs || Custom SQL Queries
Show 50 entries [Refresh]
| id | name | notes | active_score | first_run |
|---|---|---|---|---|
| 2 | geo | focus on geographical area | 885195 | 2011-02-23 |
| 3 | target | focus on target network or bad actor (via alert or dictionary hit) | 528017 | 2011-01-19 |
| 5 | other | focus on signal parameters, resource availability, casn type, etc | 33050 | 2011-03-22 |
| 1 | technology | focus on application or protocol | 0 | 2011-03-30 |
| 4 | compound | questions with multiple focus areas | 0 | 2011-03-30 |
Showing 1 to 5 of 5 entries
First Previous 1 Next Last
geo
| question | topic | active score | weight | first contributed | last contributed |
|---|---|---|---|---|---|
| 17 | Libya | 420000 | 1000 | 2011-02-23 | 2011-05-02 |
| 13 | Egypt | 187000 | 1000 | 2011-01-31 | 2011-05-02 |
| 25 | Jordan | 100000 | 1000 | 2011-03-17 | 2011-05-02 |
| 26 | Syria | 67000 | 1000 | 2011-03-21 | 2011-05-02 |
| 33 | Syria GLOBETROTTER | 42301 | 1000 | 2011-04-05 | 2011-05-02 |
| 6 | Afghanistan | 35900 | 50 | 2010-12-29 | 2011-05-02 |
| 32 | Libya GLOBETROTTER | 12661 | 100 | 2011-04-04 | 2011-05-02 |
| 24 | Yemen | 7000 | 1000 | 2011-03-17 | 2011-05-02 |
| 20 | Qatar | 6000 | 100 | 2011-03-03 | 2011-05-02 |
| 18 | Bahrain | 5100 | 100 | 2011-03-03 | 2011-05-02 |
| 34 | Yemen GLOBETROTTER | 1433 | 100 | 2011-04-05 | 2011-05-02 |
| 19 | Oman | 600 | 100 | 2011-03-03 | 2011-05-02 |
| 22 | Algeria | 100 | 100 | 2011-03-03 | 2011-05-02 |
| 21 | Morocco | 100 | 100 | 2011-03-03 | 2011-05-02 |
| 30 | Libya Facebook | 0 | 0 | 2011-03-30 | 2011-04-01 |
| 29 | Libya Twitter | 0 | 0 | 2011-03-30 | 2011-04-01 |
| 31 | Libya TOR | 0 | 0 | 2011-03-30 | 2011-04-01 |
| 9 | Ivory Coast | null | 200 | 2010-01-05 | 2011-05-02 |
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
REST: “Auto-tasking”
- ASPHALT
- Updated list of prioritized casns
[REDACTED]
VENUSAFFECT
- Scores in DRINKYBIRD
- Using modem tasking sheet
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
DRINKYBIRD
Elegant Chaos tasking priority
| Survey Casen | Casen | Satellite | Frequency | Polarity | Feed | Rasin | Data Rate | EC Priority | On Cover? |
|---|---|---|---|---|---|---|---|---|---|
| [REDACTED] | [REDACTED] | BB | 12551 | VER | K3V | IDIRECTOC | 19994.703 | 7300 | no |
| [REDACTED] | [REDACTED] | -- | -- | -- | -- | -- | -- | 5000 | no |
| [REDACTED] | [REDACTED] | BB | 12625.52 | VER | K3V | IDIRECTOC | NaN | 4350 | no |
| [REDACTED] | [REDACTED] | BB | 12563.008 | VER | K3V | WC1A | 2048 | 4000 | no |
| [REDACTED] | [REDACTED] | -- | -- | -- | -- | -- | -- | 3000 | no |
| [REDACTED] | [REDACTED] | BB | 12653.745 | VER | K3V | IDIRECTOC | -- | 3000 | no |
| [REDACTED] | [REDACTED] | -- | -- | -- | -- | -- | -- | 3000 | no |
| [REDACTED] | [REDACTED] | -- | -- | -- | -- | -- | -- | 2200 | no |
| [REDACTED] | [REDACTED] | -- | -- | -- | -- | -- | -- | 2200 | no |
| [REDACTED] | [REDACTED] | BB | 12658.442 | VER | K3V | IDIRECTOC | 7466.537 | 2000 | no |
| [REDACTED] | [REDACTED] | -- | -- | -- | -- | -- | -- | 1200 | no |
| [REDACTED] | [REDACTED] | NF | 12580.063 | HOR | K3H | IDIRECTOC | 4525.582 | 12600 | yes |
| [REDACTED] | [REDACTED] | 2C | 11501.069 | HOR | K2H | IDIRECTOC | 3780 | 8200 | yes |
| [REDACTED] | [REDACTED] | 5B | 12666.495 | VER | K3V | DVBS | 21000 | 7050 | yes |
Survey Casen | Casen
| EC Priority | On Cover? |
|---|---|
| 7300 | no |
DRINKYBIRD – Tasking Priority View
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
Libya Surge
PROBLEM
Which of the 1000's of signals surveyed have Libyan / Egyptian / Afghan networks on the VSAT-side?
SOLUTION
- Pre-run analytics determine “significance”
- Quick identification of 25 'LY' / 11 'EG' / 10 'AF' candidate signals
- Combine other analytics: target hits, pairing, etc.
- (Repeat for next country)
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
AMULET STELLAR
PROBLEM
Which case notations have traffic on IPs of interest that matches AST fingerprints?
SOLUTION
- Create IP target set; create whitelist and blacklist of XKS fingerprints
- Use Cloud capabilities to bridge between the set of all SIGINT events with matching fingerprints, and the target set
- Add the scoring question to EC
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
Ongoing Work
- New data feeds (FOGHORN, MATCHMAKER, ROADBED)
- More fields from XKS (HTTP language, NetStrings)
- XKS from MOONPENNY
- Fine tuning of GUI for Link Characterization Analysts
- NetStrings study
- Better use of Cloud resources (Link Direction) (CCDP)
- Detailed study of scoring methodology (math hire)
- Close the auto-tasking loop (RSE)
- Increase awareness and partnership with similar efforts
- Training
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108
Questions?
SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108