Target Detection Identifiers

Open PDF
Creating agency: GCHQ Government Communications Headquarters (UK)
Document date: 2009-03
Pages: 18
Genre: slides
Banner:
Classification: UK SECRET STRAP 2 COMINT ORCON
Dissemination: -

A GCHQ presentation from March 2009 introducing the concept of Target Detection Identifiers — the "fundamental atom of the Internet age" — describing how cookies and other unique persistent identifiers extracted from bulk internet traffic at 2,500 per second are used to build the surveillance databases, with a live AutoTDI discovery table showing cookies extracted from Facebook, Yahoo, Google, Reuters, YouPorn, Bebo and other sites.

Long Description:
This is a UK SECRET STRAP2 COMINT ORCON GCHQ Applied Research presentation dated March 2009. The document provides the foundational technical description of GCHQ's Target Detection Identifier framework — the conceptual and technical basis for the agency's entire internet identity tracking infrastructure. A TDI is defined as "a definite indicator of presence, that is unique and persistent for a user/machine" — described as the "fundamental atom of the Internet age" answering the questions Who, When, Where and What. The concept evolved from telephony signalling, where a phone number indicates a user is online, but applied to the much more complex world of internet traffic where identifiers must be extracted from raw packets rather than read from signalling fields. At time of writing 70 distinct TDI types had been discovered, extracted at 2,500 per second from a single 10Gbps link — equivalent to 200 million per day — with each TDI type costing approximately 250 analyst hours to develop and an automated discovery prototype under development. The TDI sources listed include Google, Yahoo, Facebook, MSN Hotmail, YouTube, Reddit, WordPress, Amazon, Google Earth, CNN, BBC, Hi5, Maktoob, Bebo, Orkut, Paltalk, Cloob and vBulletin — covering the major English and Arabic language internet platforms of the period. The table of TDI types shows both user-level identifiers such as Yahoo-Y-Cookie and machine-level identifiers such as Yahoo-B-Cookie and MS-MUID-Cookie, reflecting the distinction between tracking an individual person and tracking a specific device. The AutoTDI discovery report — a live sample from October 2008 — shows cookies being automatically extracted and ranked by frequency across multiple bearers, with Facebook c_user cookies, Yahoo B cookies, Google ID and TM cookies, Reuters tracking cookies, YouPorn session and analytics cookies, and Bebo username and email cookies all appearing in the top results. The YouPorn entries appear prominently — five separate YouPorn cookie types in the top 20 — which became the basis for the Intercept article's headline about GCHQ tracking porn site visits. The presence of Reuters and CNN cookies in the same table as YouPorn illustrates the indiscriminate nature of the bulk collection. The TDI applications section names the full suite of bulk stores fed by TDI extraction: MUTANT BROTH, AUTO ASSOC, KARMA POLICE, INFINITE MONKEYS, SOCIAL ANIMAL, MEMORY HOLE, MARBLED GECKO and HRMAP. The MUTANT BROTH interface screenshot shows the database containing 18.41 billion rows from December 2007 to June 2008, searchable by identifier, IP address or password, with all queries logged for audit and requiring a MIRANDA number and JIC priority code as justification. The MUTANT BROTH results screenshot shows the system returning a Hotmail address associated with a target whose Hi5 cookie was seen accessing the platform from Istanbul, Ankara, Chicago and Damascus across a two-day period in June 2008 — demonstrating the geolocation capability embedded in the TDI framework through IP geolocation enrichment. Additional bulk event applications described include file transfer signature detection for proof-of-life videos, internet profile-based dead letter drop detection, Yahoo webcam image collection and airline reservation confirmation email interception.
Programs:
MUTANTBROTH, AUTOASSOC, KARMAPOLICE, INFINITEMONKEYS, SOCIALANIMAL, MEMORYHOLE, MARBLEDGECKO, HRMAP, AUTOTDI

People mentioned:
-

Countries:
France

Tags:
Applied Research, TDI, AutoTDI, cookies, bulk collection, internet surveillance, Facebook, Yahoo, Google, YouPorn, Reuters, Bebo, Hotmail, Hi5, Maktoob, Orkut, PalTalk, Cloob, vBulletin, geolocation, IP addresses, pattern-of-life

Identifier:
tdi-introduction

PDF SHA-256:
86718e5878a258de03726d7d783913b738afc33f3c2b99c1459f4ad8118d234c

APPLIED RESEARCH

Target Detection Identifiers

[REDACTED]

March 2009

© This information is exempt under the Freedom of Information Act 2000 (FOIA) and may be exempt under other UK information legislation. Refer any FOIA queries to [REDACTED].

Slide 1

APPLIED
RESEARCH


UK SECRET STRAP2 COMINT ORCON

High-Speed Internet Processing

TCP SYN

GET /
User-Agent: Mozilla 4.1, IE5
Host:www.google.com
Cookie:ik=xzxsrzczccZ

TCP FIN

...
09:28:01 2008-10-13 [REDACTED] 7776 80 GET / Cookie: ik= qyzwww.....
09:28:13 2008-10-13 [REDACTED] 3456 80 GET / Cookie:ik=xzxsrzczccz
...

Google™

Event data sent to bulk store

© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

UK SECRET STRAP2 COMINT ORCON
Slide 2


UK SECRET STRAP2 COMINT ORCON

High-Speed Internet Processing

  • Bulk events key to SIGINT success on Internet
  • Event types that are valuable for Intelligence change (quickly)
    • 2000 SMTP/POP3
    • 2001 Webmail
    • ...
    • 2007 vBulletin
    • 2008 Social Networks,...,?
  • GCHQ's Applied Research are pioneering ways of dealing with this:
    • Presence Events (TDI)
    • Very large scale high speed flat file storage to bulk store TDIs
    • Just enough data marts

© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

UK SECRET STRAP2 COMINT ORCON
Slide 3


UK SECRET STRAP2 COMINT ORCON

IP Packet Information

  • Many possible types of information
  • Many techniques available
  • HTTP Get requests dominate cutting edge techniques
  • To get Intelligence value Information must relate to a person or device... a TDI

© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

UK SECRET STRAP2 COMINT ORCON
Slide 4


UK SECRET STRAP2 COMINT ORCON

TDI ...?

;

© Crown Copyright. A [REDACTED] other UK information legislation. Refer disclosure requests [REDACTED] under
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

UK SECRET STRAP2 COMINT ORCON
Slide 5


UK SECRET STRAP2 COMINT ORCON

TDI ...?

;

© Crown Copyright. [REDACTED]
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

UK SECRET STRAP2 COMINT ORCON
Slide 6


UK SECRET STRAP2 COMINT ORCON

TDI

Target
Detection
Identifier

© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

UK SECRET STRAP2 COMINT ORCON
Slide 7


UK SECRET STRAP2 COMINT ORCON

TDI

Target
Detection
Identifier

Who
When
Where
(doing) What

© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

UK SECRET STRAP2 COMINT ORCON
Slide 8


UK SECRET STRAP2 COMINT ORCON

TDI

Target
Detection
Identifier

Who
When
Where
(doing) What

Fundamental atom of the Internet age.

© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

UK SECRET STRAP2 COMINT ORCON
Slide 9


UK SECRET STRAP2 COMINT ORCON

Target Detection Identifiers

  • DEFINITION
    • TDIs are definite indicators of presence, that are unique and persistent for a user/machine.
  • Built on the familiar
    • Telephony +44 – international phone code
    • Signalling tells us this phone user is 'online'
  • Target Detection Identifiers
    • Started with the Internet, mobile networks too.
    • TDI is a 'SIGINT standardised code'.
    • Not a standard managed by the ITU/ETSI.
    • Extraction from packets much more complex.

© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

UK SECRET STRAP2 COMINT ORCON
Slide 10


UK SECRET STRAP2 COMINT ORCON

TDI sources

hi5
BBC
YAHOO!
msn Hotmail
YouTube
reddit
Windows Live
MAKTOOb
First Arabic /English e-mail
facebook
Ask
Google
Windows
amazon.com
yellow
vBulletin
WORDPRESS
Google Earth
CNN

© Crown Copyright. All rights re [REDACTED] disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

UK SECRET STRAP2 COMINT ORCON
Slide 11


UK SECRET STRAP2 COMINT ORCON

Target Detection Identifiers

  • 70 distinct TDI types discovered.
  • 2500 TDIs/sec (GET, de-duplicated)
  • => 200 Million per day per 10Gbps
  • De-dupe rate ???
  • Cost – 250 hours per TDI
  • Automated discovery prototype
TDI Type TDI Location User/Machine
Yahoo-Y-Cookie Cookie User
Yahoo-B-Cookie Coookie Machine
Google-IK Request-URI User
Paltalk-Nickname Request-URI User
MS-MUID-Cookie Cookie Machine
Google-SID-Cookie Cookie Machine
Maktoob-MEUser-Cookie Cookie User
Orkut-PREFID-Cookie Cookie User
Cloob-Username Cookie User

© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Fr [REDACTED] other UK information legislation. Refer disclosure requests to [REDACTED]
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

UK SECRET STRAP2 COMINT ORCON
Slide 12


SECRET

[REDACTED] Search GCHQ
CHQ Homepage | B AutoTDI
GET data

Report is for the 7-day period from 04/10/08 (16:20) to 11/10/08 (16:20).
A cross-subnet threshold of ≤ 10% has been applied.
A mean user transmission threshold of ≥ 10 has been applied.
Putative selectors http://, photos/, images/, files/ are blacklisted from this plot.
Click to sort by a given column.
Domain Context Technology Selector Example value Bearer count Observation count Mean user transmission frequency Cross-/16 percentage
facebook Cookie datr= [REDACTED] 8 671 12.98 3.51
facebook Cookie c_user= [REDACTED] 8 651 12.09 3.51
facebook Cookie __utmz= [REDACTED] 7 609 12.14 4.25
facebook Cookie __utma= [REDACTED] 7 609 12.44 3.56
facebook Cookie h_user= [REDACTED] 7 601 12.37 3.74
facebook Cookie __qca= [REDACTED] 6 364 10.38 4.97
reuters Cookie lv= [REDACTED] 6 336 10.67 0.24
facebook Cookie next_path= [REDACTED] 6 323 18.63 9.18
live Cookie MUID= [REDACTED] 7 321 10.81 3.24
reuters Cookie id= [REDACTED] 6 312 21.59 0.45
google URI q= [REDACTED] 7 311 15.02 5.81
reuters Cookie ss= [REDACTED] 6 309 16.83 0.39
yahoo Cookie B= [REDACTED] 7 307 10.76 2.76
yahoo Cookie d= [REDACTED] 6 306 10.60 7.79
youporn Cookie sid= [REDACTED] 5 290 24.90 1.96
youporn Cookie __utma= [REDACTED] 5 282 24.23 1.65
youporn Cookie __utmz= [REDACTED] 5 281 22.92 4.60
reuters Cookie anonId= [REDACTED] 6 279 16.22 0.46
youporn Cookie __qca= [REDACTED] 5 277 24.40 1.69
yahoo URI p= [REDACTED] 7 277 31.18 6.89
bebo Cookie bdaysession= [REDACTED] 7 275 27.19 2.06
google Cookie LM= [REDACTED] 7 272 16.85 7.31
google Cookie ID= [REDACTED] 7 271 16.80 3.73
google Cookie TM= [REDACTED] 7 270 17.07 6.57
bebo Cookie Username= [REDACTED] 7 268 27.18 2.21
bebo Cookie Email= [REDACTED] 7 268 27.67 2.24
yahoo Cookie I_s= [REDACTED] 4 264 78.61 3.00
google Cookie S= [REDACTED] 6 264 39.35 3.82
yahoo Cookie ip= [REDACTED] 3 253 14.24 2.54
yieldmanager Cookie uid= [REDACTED] 7 251 66.03 1.01
reuters Cookie RaptTracker= [REDACTED] 6 242 14.24 0.48
yahoo Referer p= [REDACTED] 4 242 17.85 7.19

UK SECRET STRAP2 COMINT ORCON

TDI Applications

  • Bulk store of all TDIs seen in last 6 months [MUTANT BROTH]
  • Bulk store TDI correlations (6 months) [AUTO ASSOC]
  • Bulk store TDI <-> website correlations (6 months) [KARMA POLICE]
  • Bulk store TDI vBulletin activity [INFINITE MONKEYS]
  • Bulk store TDI Social Networking Site activity [SOCIAL ANIMAL]
  • Bulk store web search requests [MEMORY HOLE]
  • Bulk store Google Earth requests [MARBLED GECKO]
  • Bulk store of Host.Referer references [HRMAP]

© Crown [REDACTED] Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

UK SECRET STRAP2 COMINT ORCON
Slide 14


SECRET

Edit View History Bookmarks Tools Help
B [REDACTED] Search GCHQ
s wiki 10gR2 fsrf PE ct blog /em/ im AR b13sw eci wiki ft wikipedia

MUTANT BROTH | 401 Authorization Required

UTANT BROTH

Identifier Search | IP Address Search | Password Search

Welcome

Logged in as [REDACTED] all queries logged for audit.
Database currently contains identifiers from the period Tue Dec 25 16:26:40 2007 to Fri Jun 20 22:13:19 2008 (18.41 billion rows of as 07-JUN-08).
Warning: data for period(s):-
*   Fri Jun 20 22:13:21 2008 - Tue Jun 24 09:33:20 2008
is loaded, but currently unavailable for query due to index building. The rest of the database can be queried as usual during the rebuild.

Search for Identifiers
*   If allow wildcards is ticked, % and _ are multi-character (bob%) and single-character (b_b@hotmail.com) wildcards.
*   Queries are always case-sensitive (bob@hotmail.com ≠ BOB@hotmail.com ≠ BOB@HOTMAIL.COM). There is an option to automatically convert to lowercase.
*   For bulk queries, paste in a list of identifiers separated by newlines (one per line).
*   You can enter a minimum/maximum date for the search: default is to search all available selectors

[REDACTED]

MIRANDA 20135
JIC 1
Purpose NS
Reason demo
Execute

Allow wildcards
Convert to lowercase before searching

Matching Identifiers

The following identifiers have been found in the MUTANT BROTH database.
Select those that match your target(s) to generate a summary of target activity.
TDI type TDI value
Chat-MS-Messenger [REDACTED]@hotmail.fr

SECRET

Edit View History Bookmarks Tools Help
B [REDACTED] Search GCHQ
s wiki 10gR2 fsrf PE ct blog /em/ im AR b13sw eci wiki ft wikipedia

MUTANT BROTH | 401 Authorization Required

UTANT BROTH

Identifier Search | IP Address Search | Password Search

Welcome

Logged in as [REDACTED] all queries logged for audit.
Database currently contains identifiers from the period Tue Dec 25 16:26:40 2007 to Fri Jun 20 22:13:19 2008 (18.41 billion rows of as 07-JUN-08).

Warning: data for period(s):-
*   Fri Jun 20 22:1
is loaded, but curre rebuild.

(18.41 billion rows of as 07-JUN-08).

Search for Ident
*   If allow wildcards is
*   Queries are always case-sensitive (bob@hotmail.com ≠ BOB@hotmail.com ≠ BOB@HOTMAIL.COM). There is an option to automatically convert to lowercase.
*   For bulk queries, paste in a list of identifiers separated by newlines (one per line).
*   You can enter a minimum/maximum date for the search: default is to search all available selectors

[REDACTED]

MIRANDA 20135
JIC 1
Purpose NS
Reason demo
Execute

Allow wildcards
Convert to lowercase before searching

Matching Identifiers

The following identifiers have been found in the MUTANT BROTH database.
Select those that match your target(s) to generate a summary of target activity.
TDI type TDI value
Chat-MS-Messenger [REDACTED]@hotmail.fr

SECRET

WHEN
WHERE
WHAT
WHO

Edit View History Bookmarks Tools Help
B [REDACTED] Search GCHQ
wiki 10gR2 fsrf PE ct blog /em/ im AR b13sw eci wiki ft wikipedia

MUTANT BROTH | 401 Authorization Required

[REDACTED] Chat Identifiers), the source IP Geo-Location identifies the other end of the communications link to the

Save as CSV
Date Time Source IP HHFP Source IP Geo Identifier Type Identifier Value Passw
17/06/2008 17:08:44 [REDACTED] 6de32bb0 41.02;28.96;ISTANBUL; TR; 5MMM Hi5-Email-Cookie [REDACTED]@hotmail.com
17/08 [REDACTED] 6de32bb0 41.02[REDACTED] M Hi5-Email-Cookie [REDACTED]@hotmail.com
17/06/2008 16:55:21 6de32bb0 41.9022;-87.6726; CHICAGO; US;5 [REDACTED] [REDACTED]@hotmail.com
17/06/2008 16:55:16 [REDACTED] 6de32bb0 41.02;28.96; ISTANBUL; TR; 5MMM Hi5-Email-Cookie [REDACTED]@hotmail.com
17/06/2008 16:54:47 6de32bb0 41.9022;-87.6726; CHICAGO; US;5MMM Hi5-Email-Cookie [REDACTED]@hotmail.com
17/06/2008 16:52:13 [REDACTED] 6de32bb0 39.94;32.86; ANKARA; TR; 5MMM Hi5-Email-Cookie [REDACTED]@hotmail.com
15/06/2008 19:20:33 [REDACTED] de8bdc48 33.5;36.3; DIMASHQ;SY; 5MLV Hi5-Email-Cookie [REDACTED]@hotmail.com

UK SECRET STRAP2 COMINT ORCON

Other Bulk Event Applications

  • Most events that can be associated back to TDIs:
  • File Transfer Signature (eg proof of life videos)
  • Detection by Internet profile – eg 'Dead Letter Drop'.
  • Yahoo webcam images
  • Airline reservation confirmation emails

© Crown Copyright. All rights reserved. This information is exempt from disclosure under the Freedom of Information Act 2000 and may be subject to exemption under other UK information legislation. Refer disclosure requests to [REDACTED]
Contains Intellectual Property owned and/or managed by GCHQ. The material may be disseminated throughout the recipient organisation, but GCHQ permission must be obtained for dissemination outside the organisation.

UK SECRET STRAP2 COMINT ORCON
Slide 18