Elegant Chaos: collect it all, exploit it all

Open PDF +1 other copy
Creating agency: GCHQ Government Communications Headquarters (UK)
Document date: -
Pages: 23
Genre: slides
Banner:
Classification: SECRET
Dissemination: REL TO USA, AUS, CAN, GBR, NZL
the copy shown on this pageoriginal PDF ↗
The pages reprinted in the document appendix of No Place to Hide (appendix page 5).

This document outlines a classified project, "Elegant CHAOS," focused on advanced signals intelligence (SIGINT) collection, processing, and analysis using cutting-edge technologies like automated surveys and modems. It highlights goals such as real-time data analysis, collaboration with international partners, and integration with cloud-based platforms to manage massive data. See also the document with notes (by GCHQ): "Elegant-Chaos-collect-it-all-exploit-it-all-plus"

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

Elegant CHAOS

Project Leads: [REDACTED]
Code Support: [REDACTED]

For more up-to-date information please visit this URL:
[REDACTED]Elegant_Chaos
For recent developments please contact the authors.

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

Field Site Responsibilities

Sniff it All
Know it All
Collect it All
Process it All
Exploit it All
Partner it All

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

For Better Or Worse...

Collection access is increasing,

  • software modems (ASPHALT/A-PLUS)
  • new hardware/software solutions (STORMFORCE modems and DARKQUEST auto-survey)
  • new physical capacity (TORUS antennae)

Processing capabilities are increasing,

  • JCE, TINT, XKS Deep Dive

But size of analytic workforce is not!

  • more resources = more resource management

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

New Collection Posture

Sniff it All
   Torus increases physical access

Know it All
   Automated FORNSAT survey - DARKQUEST

Collect it All
   Increase volume of signals: ASPHALT/A-PLUS

Process it All
   Scale XKS and use MVR techniques

Exploit it All
   Analysis of data at scale: ELEGANTCHAOS

Partner it All
   Work with GCHQ, share with Misawa

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

ELEGANTCHAOS Goals

  • Goal: perform basic, time-sensitive analysis on all of MHS collection
  • Goal: create a prioritized list of signals (case notations) in our viewing arc
  • Goal: use this list to automatically drive collection as collection capabilities increase
  • Offshoot goal: create a product that analysts and collection managers can use to see into the system

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

ELEGANTCHAOS + Cloud

The MHS Cloud provides an excellent platform for this project:

  • data ingest, normalization, tagging
  • access to SIGINT data from various processors, from sustained mission + survey
  • access to a huge body of enrichment data
  • processing, storage, and web-hosting
    • considering decoupling these parts...

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

ELEGANTCHAOS In Context

COLLECTION
FRONT END RF Collection
STORM FORCE
DARK QUEST
ASPHALT

Collection Distribution
WC
XKS
METADATA
CLOUD
TURMOIL
METADATA
CLOUD
TARMAC
PPF
SLR
CLOUD

CLOUD

Multiple QFDs
EC GUI

ENRICHMENT
Open Source
Target Network
Service
IP Geo
System Status

Σ

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

EC System Overview

1 Data
SIGINT
MHS CLOUD SERVERS
enrichment

2 EC Code
QFD
questions
QFD
score log
QFD
QFD
QFD
QFD
QFD
QFD
3 GUI

MySql tables and code

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

Data Sources (May 2011)

SIGINT Feeds

  • XKEYSCORE
  • ASDF (Turmoil LIVE)
  • SLR (TARMAC)
  • POPQUIZ (Turmoil DEV)
  • WEALTHYCLUSTER2

Enrichment Feeds

  • IPGeoTrap
  • TRAVELLINGWAVE Scores
  • BILBOBADGER Daily Summaries
  • Target Network Service list + CNO Target list
  • DRINKYBIRD monitoring info
  • GLOBETROTTER OH Geo
  • MASTERSHAKE Geo
  • Quantumable Case Notation list

Event counts over a 12-hour period.
Total events: 335,663,981

TU/Live
TU/Dev
SLR
WC2
XKS

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

Analytic Questions (May 2011)

Target

  • Dictionary hits
  • Target Networks
    • TNS, TW, CNO
  • PLUS Reports, CRNs, etc.

Technology

  • [REDACTED]
  • VPNs
  • Twitter, Facebook, VoIP
  • CNO behavior

Miscellany

  • Modem Capacity
  • Paired Links
  • Quantumable

Location

  • IP-based
  • MAC-based
  • Geo-based
  • Surge Countries
    • Libya, Egypt, Afghanistan,
      Syria, Yemen, Ivory Coast,
      etc.

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

Questions & Scoring

  • Each question is represented by a SQL query applied to one or more QFDs
  • QFDs are case notation-based repositories of signal information
    • eg, IPs and registries for all case notations
    • eg, category hits for all case notations
    • eg, GLOBETROTTER geos for all case notations
  • All questions are asked once per day across all case notations
  • Points are assigned to each question based on current analytic priorities
  • Points for any particular question are "active" for a window of time (eg, 1 day, 7 days, 30 days)
  • The sum of "active" points for a case notation, across all questions, forms the score

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

Interfaces

Different interfaces for different customers

  • ELEGANTCHAOS GUI
    • made for analysts to examine scores and the impact of the different questions
    • eventually, control over the algorithms may reside here
  • REST interface
    • made for programmatic query, precursor to auto tasking
  • DRINKYBIRD GUI
    • made for collection personnel to determine if resources are available, easy to view what's on cover

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

EC GUI: Case Notation View

Elegant CHAOS

All Questions || All CASNs || Survey CASNs

Show 10 entries [Refresh]
casen_id casen score direction notes first_heard last_heard number_of_day
1 [REDACTED] 157040 forward [REDACTED] 2010-12-26 2011-03-17 25
101 [REDACTED] 55500 forward [REDACTED] 2010-12-26 2011-03-17 25
5275 [REDACTED] 33160 forward [REDACTED] 2011-03-16 2011-03-17 0
5288 [REDACTED] 31370 unknown [REDACTED] 2011-03-17 2011-03-17 0
5285 [REDACTED] 26890 unknown [REDACTED] 2011-03-17 2011-03-17 0
1110 [REDACTED] 17450 return [REDACTED] 2011-01-03 2011-03-17 17
5292 [REDACTED] 15330 unknown [REDACTED] 2011-03-17 2011-03-17 0
5277 [REDACTED] 14210 unknown [REDACTED] 2011-03-17 2011-03-17 0
5289 [REDACTED] 12930 unknown [REDACTED] 2011-03-17 2011-03-17 0
5283 [REDACTED] 12630 unknown [REDACTED] 2011-03-17 2011-03-17 0
Showing 1 to 10 of 3,888 entries
First Previous 1 2 3 4 5 Next Last

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

EC GUI: Case Notation View

Mozilla Firefox
File Edit View History Bookmarks Tools Help
bucket elegantchaos
Tag This!! $$ MHS AM Notes Puzz Tiny Base Java API 1.6 cheat MySQL MySQL Syntax Tools misc RB EChaos Cloud Web Prog Mett
(U) XKS ... http...desc GM ROY... XKEYSC... FORESIG... (U) Whiz... [REDACTED] cl4r2hd2... Safari Bo... Private N... Matatu ht...sc DRINKYBI... Mass Tra...

TOP SECRET // COMINT // REL TO USA, FVEY

Elegant CHAOS

Not seeing any results?

All Questions || All CASNs || Survey CASNs

Show 10 entries Refresh
casen_id casen score direction notes first_heard last_heard number_of_day
1 [REDACTED] 157540 forward [REDACTED] 2010-12-26 2011-03-17 25
103 [REDACTED] 30000 forward [REDACTED] 2010-12-20 2011-03-17 23
5275 [REDACTED] 33660 forward [REDACTED] 2011-03-16 2011-03-17 0
5288 [REDACTED] 31870 unknown [REDACTED] 2011-03-17 2011-03-17 0
5285 [REDACTED] 27398 unknown [REDACTED] 2011-03-17 2011-03-17 D
1110 [REDACTED] 18950 return [REDACTED] 2011-01-03 2011-03-17 17
5292 [REDACTED] 15830 unknown [REDACTED] 2011-03-17 2011-03-17 0
5277 [REDACTED] 14710 unknown [REDACTED] 2011-03-17 2011-03-17 D
5289 [REDACTED] 13430 unknown [REDACTED] 2011-03-17 2011-03-17 0
5283 [REDACTED] 13130 unknown [REDACTED] 2011-03-17 2011-03-17 0
Showing 1 to 10 of 4,106 entries

First Previous 1 2 3 4 5 Next Last
topic question weight question score first contributed
Egypt 1000 7000 2011-02-18
Qatar 100 200 2011-02-27
CNO Targets 500 500 2011-03-17
Paired Link 50 200 2011-02-23
Priority 3 CategoryHits 50 2400 2011-03-16
Jordan 1000 1000 2011-03-17
Afghanistan 50 50 2011-03-04
Libya 1000 7000 2011-02-18
Priority 4 CategoryHits 20 600 2011-03-26
Points earned over time

Select a case notation
Observe questions which affected its score

x Find 192 Previous Next Highlight all Match case Phrase not found
elegantchaos.bucket.r6.r.nsa FoxyProxy Patterns

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

EC GUI: Question View

TOP SECRET//SI/TK//REL TO USA, AUS, CAN, GBR, NZL

Elegant CHAOS

Not seeing any results?

All Questions || All CASNo || Survey CASNO

Show 111 entries [Refresh]
question_id topic weight days_between_run days_before_points_expire description active_score total_score expired_score first_active_
16 1000 1 30 9000 16000 6000 2011 02 18
1 TRAFFICTHIEF lit 1000 1 7 1000 points given each day for each case notation with a priority 1 (TRAFFICTHIEF) tip seen 28000 116000 118000 2011-03-15
17 Libya 1000 1 30 1000 points for each case notation with a Libyan IP address on the significant side of a link 117000 216000 99000 2011-02-17
13 Egypt 1000 1 30 Gives case notation 1000 points if the known significant/GAT side of the link has an IP address that resolves to Egypt 62000 172000 110000 2011-02-16
14 FORNSAT CRNo 500 / / 500 points for each case with a current FORNSAT CRN 185000 185000 145000 2011-01-16
12 Tasked Active Users 150 1 7 150 points given for a case notation if it has Marina Daily Active Counts greater than 0 in the BILBOBADGER Daily Summary 1350 16050 14700 2011-03-15

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

EC GUI: Question View

Mozilla Firefox
File Edit View History Bookmarks Tools Help
bucket elegantchaos
Tag This!! $$ MHS AM Notes Puzz Tiny Base Java API 1.6 cheat MySQL MySQL Syntax Tools misc RB EChaos Cloud Web Prog Mett
(U) XKS ... http...desc GM ROY... XKEYSC... FORESIG... (U) Whiz... WhizBan... WhizBan... cl4r2hd2... Safari Bo... Private N... Matatu ht...sc DRINKYBI... Mass Tra...

TOP SECRET // COMINT // REL TO USA, FVEY

Elegant CHAOS

All Questions || All CASNs || Survey CASNs

Show 10 entries
Refresh
question_id topic weight days_between_run days_before_points_expire description active_score total_score expired_score first_ac
CNO
23 CNO Targets 500 1 7 Gives case notation 500 points for each day (up to 7 most recent days) in which target traffic hit on CNO XXS fingerprints 44500 44500 2011-03-
Showing 1 to 1 of 1 entries
First Previous 1 Next Last

Question 23
Points Score Date
500 2590 2011-03
500 3080 2011-03
500 15830 2011-03
500 18950 2011-03
500 13430 2011-03
500 500 2011-03
500 820 2011-03
500 8210 2011-03
500 500 2011-03
500 1580 2011-03
500 13130 2011-03
500 600 2011-03
500 2730 2011-03
500 9080 2011-03
500 820 2011-03
500 3400 2011-03
500 1640 2011-03
500 570 2011-03
Points earned over time

Filter based on a topic; select a question
Observe case notations which scored

x Find 192 Previous Next Highlight all Match case Phrase not found
elegantchaos.bucket.r6.r.nsa FoxyProxy Patterns

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

Focus Areas: Custom Views?

Home || All Questions || All CASNs || Survey CASNs || ASPHALT CASNs || Custom SQL Queries

Show 50 entries [Refresh]
id name notes active_score first_run
2 geo focus on geographical area 885195 2011-02-23
3 target focus on target network or bad actor (via alert or dictionary hit) 528017 2011-01-19
5 other focus on signal parameters, resource availability, casn type, etc 33050 2011-03-22
1 technology focus on application or protocol 0 2011-03-30
4 compound questions with multiple focus areas 0 2011-03-30
Showing 1 to 5 of 5 entries
First Previous 1 Next Last

geo
question topic active score weight first contributed last contributed
17 Libya 420000 1000 2011-02-23 2011-05-02
13 Egypt 187000 1000 2011-01-31 2011-05-02
25 Jordan 100000 1000 2011-03-17 2011-05-02
26 Syria 67000 1000 2011-03-21 2011-05-02
33 Syria GLOBETROTTER 42301 1000 2011-04-05 2011-05-02
6 Afghanistan 35900 50 2010-12-29 2011-05-02
32 Libya GLOBETROTTER 12661 100 2011-04-04 2011-05-02
24 Yemen 7000 1000 2011-03-17 2011-05-02
20 Qatar 6000 100 2011-03-03 2011-05-02
18 Bahrain 5100 100 2011-03-03 2011-05-02
34 Yemen GLOBETROTTER 1433 100 2011-04-05 2011-05-02
19 Oman 600 100 2011-03-03 2011-05-02
22 Algeria 100 100 2011-03-03 2011-05-02
21 Morocco 100 100 2011-03-03 2011-05-02
30 Libya Facebook 0 0 2011-03-30 2011-04-01
29 Libya Twitter 0 0 2011-03-30 2011-04-01
31 Libya TOR 0 0 2011-03-30 2011-04-01
9 Ivory Coast null 200 2010-01-05 2011-05-02

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

REST: “Auto-tasking”

  • ASPHALT
    • Updated list of prioritized casns

[REDACTED]

VENUSAFFECT

  • Scores in DRINKYBIRD
  • Using modem tasking sheet

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

DRINKYBIRD

Elegant Chaos tasking priority

Survey Casen Casen Satellite Frequency Polarity Feed Rasin Data Rate EC Priority On Cover?
[REDACTED] [REDACTED] BB 12551 VER K3V IDIRECTOC 19994.703 7300 no
[REDACTED] [REDACTED] -- -- -- -- -- -- 5000 no
[REDACTED] [REDACTED] BB 12625.52 VER K3V IDIRECTOC NaN 4350 no
[REDACTED] [REDACTED] BB 12563.008 VER K3V WC1A 2048 4000 no
[REDACTED] [REDACTED] -- -- -- -- -- -- 3000 no
[REDACTED] [REDACTED] BB 12653.745 VER K3V IDIRECTOC -- 3000 no
[REDACTED] [REDACTED] -- -- -- -- -- -- 3000 no
[REDACTED] [REDACTED] -- -- -- -- -- -- 2200 no
[REDACTED] [REDACTED] -- -- -- -- -- -- 2200 no
[REDACTED] [REDACTED] BB 12658.442 VER K3V IDIRECTOC 7466.537 2000 no
[REDACTED] [REDACTED] -- -- -- -- -- -- 1200 no
[REDACTED] [REDACTED] NF 12580.063 HOR K3H IDIRECTOC 4525.582 12600 yes
[REDACTED] [REDACTED] 2C 11501.069 HOR K2H IDIRECTOC 3780 8200 yes
[REDACTED] [REDACTED] 5B 12666.495 VER K3V DVBS 21000 7050 yes

Survey Casen | Casen

EC Priority On Cover?
7300 no

DRINKYBIRD – Tasking Priority View

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

Libya Surge

PROBLEM

Which of the 1000's of signals surveyed have Libyan / Egyptian / Afghan networks on the VSAT-side?

SOLUTION

  • Pre-run analytics determine “significance”
  • Quick identification of 25 'LY' / 11 'EG' / 10 'AF' candidate signals
  • Combine other analytics: target hits, pairing, etc.
  • (Repeat for next country)

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

AMULET STELLAR

PROBLEM

Which case notations have traffic on IPs of interest that matches AST fingerprints?

SOLUTION

  • Create IP target set; create whitelist and blacklist of XKS fingerprints
  • Use Cloud capabilities to bridge between the set of all SIGINT events with matching fingerprints, and the target set
  • Add the scoring question to EC

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

Ongoing Work

  • New data feeds (FOGHORN, MATCHMAKER, ROADBED)
  • More fields from XKS (HTTP language, NetStrings)
  • XKS from MOONPENNY
  • Fine tuning of GUI for Link Characterization Analysts
  • NetStrings study
  • Better use of Cloud resources (Link Direction) (CCDP)
  • Detailed study of scoring methodology (math hire)
  • Close the auto-tasking loop (RSE)
  • Increase awareness and partnership with similar efforts
  • Training

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108


SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108

Questions?

SECRET//REL TO USA, AUS, CAN, GBR, NZL//20320108