Finding and Querying Document Metadata
Open PDFDocument date: 2009-04-01
Pages: 24
Genre: slides
Banner: TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20291123
Classification: TOP SECRET COMINT
Dissemination: REL TO USA, AUS, CAN, GBR, NZL
This 2009 NSA training guide, created by Booz Allen Hamilton, explains how analysts can use the XKEYSCORE system to track targets by searching for "non-traditional selectors" embedded within documents, such as author names, company information, or even the hash values of company logos.
My notes
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20291123
KEYSCORE
Finding and Querying on Document Metadata
Booz Allen Hamilton
Sigint Development Support / SIGINT Technical Analysis (SDS/STA)
April 2009
Derived From: NSA/CSSM 1-52
Dated: 20070108
Declassify On: 20291123
DERIVED FROM: NSA/CSSM 1-52
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20291123
08
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Document Metadata Agenda
- Why to Query on Document Metadata
- How to Find Document Metadata
- e.g. File -> Properties
- How to Create Queries in XKS
- XKEYSCORE Document Metadata and PDF Metadata
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Document Metadata Analysis
- What?: Use non-traditional selectors to find and track targets sending/receiving documents of interest
- How? It targets documents by Author, Organization, or embedded images (logos)
- Why? We don't always know WHO is sending the documents, but they are "guilty-by-association" if they send/receive the document. So, who are THEY?
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Finding Document Metadata
- We find "Document Metadata" in File Properties
XKEYSCORE_Terms.doc - Microsoft Word TO
File Edit View Insert Format Tools Table Window Help SecretAgent Classif
New...
Open...
Close
Save
Save As...
Save as Web Page...
Remove Hidden Data...
File Search...
Versions...
Web Page Preview
Page Setup...
Print Preview
Print...
Send to
Properties
1 U:\private\andrew.doc
2 U:\...\XKEYSCORE Tips and Tricks 4 ap...
3 U:\private\Présentation.doc
4 U:\private\...\NIA Cross Training.doc
5 U:\private\NIA/MSRs\April 2009.doc
6 C:\Documents and Settings\...\IMEI.doc
7 U:\...\NIA\RITCHIE_DNI_IWPS(revised).doc
8 U:\...\XKS_kmkeith_tips7April.doc
9 U:\...\Zwakenberg, Garrit Trey IPWS.doc
Exit
STM4, STM16, STM64, multip
s.pdf
XKEYSCORE_Terms.doc Properties
General | Summary | Statistics | Contents | Custom
Title: XKeyscore Terms
Subject:
Author: Joe BaggaDonuts
Manager:
Company: ZMFA Zendian MFA
Category:
Keywords:
Comments:
Hyperlink base:
Template: Normal.dot
Save preview picture
OK Cancel
If unique, these Document Properties can be targeted
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Document Metadata Analysis
- How do you find document metadata?
- Passive Collection: Collected Documents already contain data
- Active Collection: CNE "Categorized Collection" from TUNINGFORK Data or Pinwale Queries on "US-3101"
- Open Source: Google Hacking
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Finding Document Metadata
| ID | To | From | CO BC Date | Subject | Size(K) | Type |
|---|---|---|---|---|---|---|
| 48 | [REDACTED]o.com" <[REDACTED]com> | "DESTOCKPRO" <des[REDACTED]> | 5/15/2008 6:31:35 PM | ARRIVAGE G STAR DOLCE&GABBANA DIESEL | 16 | text/html |
| 49 | [REDACTED]yahoo.com" [REDACTED]noo.com> | "Target Center" <[REDACTED]> | 5/16/2037 9:36:13 PM | Confirmation: Target Card | 2 | text/html |
| 51 | [REDACTED]75[REDACTED]> | 5/15/2008 11:14:32 PM | 1 | text/html | ||
| 41 | [REDACTED]<azedi[REDACTED]il.com>, "abu zubeer alyamani" | "khalid" <o[REDACTED]> | 5/15/2008 3:05:25 PM | Las Villas de Dubai | 3898 | application/octet-stre |
| 32 | [REDACTED]<ke[REDACTED] | [REDACTED] | 5/15/2008 11:15:21 PM | skriv ut | 452 | application/msword |
| 51 | [REDACTED] | 5/15/2008 11:14:32 PM | 452 | application/msword | ||
| 50 | application/msword |
3898 application/octet-strear bas
452 application/msword bas
50 application/msword bas
| Display original | Raw | SMTP header | Properties | Control | C2C | Trailer | Collected Doc | Search Kwd |
Dictionary Kwd | SRI |
Document Properties
Category:
Company:
HiddenSlideCount: 0
LineCount: 29
LinksUpToDate: False
Manager:
MMClipCount: 29
NoteCount: 29
ParagraphCount: 8
PresentationTarget:
ScaleCrop: False
SlideCount: 8
AppName: Microsoft Word 10.0
Author: GoGo
CharacterCount: 3582
Comments:
DateCreated: 5/12/2008 3:13:00 AM
SecurityLevel: none
Keywords:
LastAuthor: [REDACTED]
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Finding Document Metadata
- Active Collection: CNE “Categorized Collection” from TUNINGFORK Data
TUNINGFORK
Collection
No EP user information found.
Raw Project Details[s3115 only]
Mailbox Collection
Last Collection [limit 3 dates listed]: 2008-08-29 2008-08-27 2008-07-19
List All
Categorized Collection
Cipher (8) | MicroSoft (277) | Multimedia (17) | Mail (35) | Inst Msgr (9) | VOIP (1642) | HTM
Cipher
Access (0)
Excel (2)
Execs (4)
Ini files (2)
Other Office (5)
Powerpoint (0)
Thumbs.db (12)
Word (252)
| Show Pat | Collected | Size |
|---|---|---|
| 81e0bd 21-4a68af648ec5 | 2008-07-19 | 388 |
| b850ea 1d-3dffb4d38926 | 2008-07-19 | 388 |
| 0c6527 80-c1ed1756266f | 2008-07-13 | 388 |
To find Document Metadata in TUNINGFORK, you must view each Document in Categorized Collection (manual intensive)
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Finding Document Metadata
- Using XKEYSCORE to query on CNE data
Fields | Advanced Features | Show Hidden Search Fields | Clear Search Values | Reload Last Search Values
Search: Document Metadata
Input Source: FOXACID\* or FOXBASE*
[x] Selected implant exfils from active collection (xks-cne.corp.nsa.ic.gov:xs_web_db)
This query in XKS
Produced these results
| Filename | Extension | Author | Title | Input Source |
|---|---|---|---|---|
| \System Admin CV.doc | doc | Authorised User | [REDACTED]E4 | |
| C:\Downloads\Physical_Layer_in_RPR_0402.pdf | [REDACTED]ER12 | |||
| C:\Documents and Settings\Guest\Desktop\05070807_excelbook.pdf | Center For Excellence | (Microsoft Word - \307\341\343\336\3) | ||
| C:\Documents and Settings\user\Desktop\desktop icons\yementrack.doc | doc | user | [REDACTED]DER12 | |
| C:\Downloads\ins-3-overview.pdf | [REDACTED]DER12 | |||
| C:\Documents and Settings\user\Desktop\desktop icons\servers_exp | doc | results | الأخ مدير عام المشتريات والمخازن المحترم | [REDACTED]R12 |
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Finding Document Metadata
- Open Source: Google Hacking
Google
Advanced Search
Search Tip
Help
site:comsats.net.pk filetype:doc Google Search
Or....
filetype:xls filetype:pdf
- Search by domains
- "site:comsats.net.pk"
- Search by file types
- "filetype:pdf" or "filetype:doc"
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Document Metadata Analysis
How to find Document Metadata when you have NEVER collected a document
Employee List
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Document Metadata Analysis
Take Client’s (Active User) IP address and query on it in XKEYSCORE
Active User:
[REDACTED]@yahoo.com
| ACTIVE USER | ACTIVE_USER_IP |
|---|---|
| [REDACTED]<yahoo> | 89[REDACTED] |
| [REDACTED]<yahoo> | 89[REDACTED] |
Search: Document Metadata
Extension: ppt or doc or pdf or xls
IP Address: 89[REDACTED] Either
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Targeting Document Metadata
- Use XKEYSCORE to Find Who Else is sending the files?
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Document Metadata Analysis
Take "File Properties" information and fill-in query
XKEYSCORE_Terms.doc Properties
General | Summary | Statistics | Contents | Custom |
Title: XKeyscore Terms
Subject:
Author: Joe BaggaDonuts
Manager:
Company: ZMFA Zendian MFA
Category:
Keywords:
Comments:
Hyperlink base:
Template: Normal.dot
Save preview picture
OK Cancel
Search: Document Metadata
Document Type:
Encrypted?:
Corrupted?:
Filename:
Extension:
*Subject*:
*Creation Time*:
*Last Modified Time*:
*Unique ID* [fulltext]:
Author: Joe BaggaDonuts
Last Author:
Organization: ZMFA Zendian MFA
Title:
Language:
*Comment* [fulltext]:
File/Embedded Image Hash [fulltext]:
Metadata Name:
Metadata Value [fulltext]:
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Document Metadata Analysis
Sample Query
Sample Query:
Organization = PTCL
To/From Country = Pakistan
Search: Document Metadata
Organization: PTCL
Title:
Language:
Comment [fulltext]:
File/Embedded Image Hash [fulltext]:
Metadata Name:
Metadata Value [fulltext]:
IP Address: From
IP Address: To
Port: From
Port: To
Country: PK Either
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Document Metadata Analysis
Sample Query (Results)
Previous Slide produces these results
| Filename | Organization |
|---|---|
| Instructions to Kunar province bidders community midwifery.doc | PTCL |
| Instructions to Kunar province bidders community midwifery.doc | PTCL |
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Embedded Images
- Turn a logo into a selector
YemenNet
The Gateway of Yemen
= SIGINT VALUE
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Embedded Images
- XKEYSCORE parses out logos from within documents (PDFs, DOCs, Outlook Emails, etc) embedded as images
Logo/Image 32-character hash can be parsed out and queried.
YemenNet
The Gateway of Yemen
image-6b71557c23ce00a03a662d6dd1eb1c1b.jpeg
1 / 2 80.6% Find
المحترم
الخازن
الموضوع : طلب مراسلة للشركات المقدمة في مناقصة توسعة سرفرات خدمات الانترنت
بالاشارة الى الموضوع أعلاه نرجوا مخاطبة الشركات المذكورة بالملاحظات الخاصة بكل منها
الصنف |اسم الشركة الملاحظات
MDS لم تذكر : MB L2 On Chip Cache per Processor 2+ System Controller Card 1+ Solaris 10 03/05 HW1 Operating System Preinstalled.
Sun Fire V490 Server
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Embedded Images
Files often contain embedded images, such as company logos.
Step 1: Identify if a document HAS an image in it
تطورات خدمات الانترنت بالاشاره الى الموضوع اعلاه برده
YemenNet
النواحي التوجيهيه إلى من يلزم تفعيل خدمة ADSL الأخ
بال ١٥٢٥٦ الادارية و آآ را ت ا ا و دارات ١٥٢٥
VPN Clients Configuration Example I
لسنوي لادارة الانترنت للعام 2007 12/1/2008 8
YemenNet
by ITE. 316) & Abdullah Mohammed
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Embedded Images
| Datetime | Case Notation | From IP |
|---|---|---|
| 2009-03-26 15:54:50 | YM.PGQXXXABDDTC |
Step 2: Open Document and click on "Full Session"
Session | Header (3) | Attachments (6) | Meta (3)
Formatter: AUTO
Send to: Download Session | Mode: Full Session | Options |
Quick Clicks
* Session
* Attachments
* sigint
* image_summary_mont
* image_summary_m
* document_meta
* c:_documents and s
* unknown
* text
* document_body.رسید
* document_body.ی (ره
* image
* jpeg
* b3d7853e4bfde7087
* office
* pdf
* C:\Documents and S
* One-Click Searches
* Find opposite side of sessi
* :0->
* :0
* Find More Docs with Same
* 635ed0657cfe25b7790f
* b3d7853e4bfde70874cf
Retrieving Attachment...
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Embedded Images
Session | Header (3) | Attachments (6) | Meta (3)
Formatter: AUTO
Send to: Download Session | Mode: Full Session | Options |
Quick Clicks
* Session
* Attachments
* sigint
* image_summary_mont
* image_summary_m
* document_meta
* c:_documents and s
* unknown
* text
* document_body.رسید
* document_body.ی (ره
* image
* jpeg
* b3d7853e4bfde7087
* office
* pdf
* C:\Documents and S
* One-Click Searches
* Find opposite side of sessi
* :0->
* :0
* Find More Docs with Same
* 635ed0657cfe25b77901
* b3d7853e4bfde70874cf
b3d7853e4bfde70874cf402a3d6cfe10.jpg
Virus scan results
Using IMAGE format
b3d7853e4bfde70874cf402a3d6cfe10.jpg
G.T.I
General Telecommunication Institute" and المعهد العام للاتصالات
Step 3: In left-side menu bar, select an image and copy/paste the 32-character name (without the extension)
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Embedded Images
Step 4: Paste the 32-character name into the "File/Embedded Image Hash" Field in the Document Metadata query
Fields | Advanced Features | Show Hidden Search Fields | Clear Search Values | Reload Last Search Values
Search: Document Metadata
File/Embedded Image Hash [fulltext]: b3d7853e4bfde70874cf402a3d6cfe10
Step 5: Select all of your good collection sites + SUBMIT!
* Classic A-M
* ASF and WMV Metadata
* Alert
* BlackBerry
* CNE
* Call Logs
* Category DNI
* Cellular DNI
* Cisco Passwords
* Document Metadata
* Document Tagging
Search Databases
Clear Checks
Reset Checks
* [x] (xks-central.corp.nsa.ic.gov:qsummary)
* [x] Australian sites (xkcentral2.dsd:xs_web_db)
* [x] CARBOY (carboy-proxy.r1.r.nsa:carboy_web_db)
* [x] CARDAMON (xkey-dsd.r1.r.nsa:xs_web_db)
Submit Cancel
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Embedded Images
Or... You can one-click query to create a new query
Session | Header (3) | Attachments (6) | Meta (4)
Formatter: AUTO
Send to: Down
Quick Clicks
* image_summary_montage.jpeg
* document_meta
* c:_documents and settings_usuari
* unknown
* text
* ?document_body.SOLICITANTE .txt
* office
* word
* C:\Documents and Settings\usuari
* One-Click Searches
* Find opposite side of session
* :0->
* :0
* Find More Docs with Same hash
* a97d82d06aaa9017cacbe5fe4b12f15c
* f4c6353ebd01ba02b7c087a91bdf29c4
Banco Nacional de Vivienda y Hábitat
BANAVIH
a533e3a6e83d50ea629ba899f9b9091
a99d3f1acf45e5f466d6ed99e484d877
* Find email address
* zakimoussa@hotmail.com
Search: Document Metadata
Query Name: One-click search on document hash: f4c6353
Justification: One-click search to find more documents with
Additional Justification:
Miranda Number:
Language:
\*Comment* [fulltext]:
File/Embedded Image Hash [fulltext]: f4c6353ebd01ba02b7c087a91bdf29c4
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Embedded Images
- Stand-alone files can be uploaded into XKS and images parsed out
- Useful for TAO collection that didn't get into XKS (non United Rake)
- https://xks-central.corp.nsa.ic.gov/general/view_file.php
Employee List
This system is aud
CLASSIFICATION: TO
XKEYS
You can upload SOTF and D-124 files, as well as just random files (.doc, .ppt, etc.)
Upload File Browse...
Upload
This system is aud
CLASSIFICATION: TO
- To task the hex values for images in CADENCE or Query in PINWALE, contact The Xtreme Target Pursuit Team [REDACTED] S2I7 and [REDACTED] S3114
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL
Embedded Images
- Questions on any of these tools or techniques, contact:
- [REDACTED]
- [REDACTED]
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL