The Unofficial XKEYSCORE User Guide

Open PDF +2 other copies
Creating agency: NSA National Security Agency (US)
Document date: -
Pages: 27
Genre: guide
Banner:
Classification: TOP SECRET COMINT
Dissemination: REL TO USA, AUS, CAN, GBR, NZL

A TOP SECRET XKeyscore user guide written by a Booz Allen Hamilton consultant, providing step-by-step instructions for 32 classic search types — including email address queries, password extraction from mail server IPs, phone number extraction, HTTP activity analysis, document metadata fingerprinting and workflow automation — with worked examples targeting counterterrorism subjects in Afghanistan, Pakistan, Iran and Sweden, viewed through an Agility instance bearing the ECI RAGTIME classification banner confirming the tool was accredited for FISA-authorised collection.

Long Description:
This is a TOP SECRET//COMINT//REL TO USA, FVEY document authored by a consultant from Booz Allen Hamilton (unit E92-ADET), published by The Intercept on 1 July 2015 as part of the XKeyscore document release. The document is a practical analyst training guide covering XKeyscore's query interface, search types and workflow system, written in an accessible tutorial style with annotated screenshots. The document's most operationally significant content is the Logins and Passwords section, which describes how analysts can extract username and password pairs from bulk internet collection by querying on a foreign mail server's IP address — with the explicit note that "Logins and passwords are valuable tools to enable Tailored Access Operations (TAO)." The guide describes using TUNINGFORK (including FoggyBottom) as a source of passwords, alongside content of emails and text messages and previous XKeyscore queries. The worked example queries on an Afghanistan network mail server IP to retrieve all login credentials passing through it. The query decision flowchart explicitly lists "Logins and Passwords for the server for TAO" as a primary use case alongside email addresses, phone numbers and file attachments. The Email Addresses query section demonstrates searching on a target's email username and domain, but also shows how querying an IP address returns all email addresses seen on that IP — effectively mapping all users of a foreign mail server. A constraint noted is that the IP must be hosted outside Five Eyes countries and the domain must be foreign owned. The HTTP Activity section shows analysts querying for all visitors to a specific website — the worked example uses al-hisbah.com with Sweden as the country filter — or querying all web activity from a specific IP block to discover unknown sites visited by a target. The Document Metadata section demonstrates tracking document authors across intercepted file attachments using Author and LastAuthor fields from Microsoft Office and PDF metadata, enabling identification of target-associated documents even without knowing the sender's email address. The Document Metadata section contains a significant classification disclosure. The Agility viewer interface displays the classification banner TOP SECRET//COMINT-ECI RAGTIME///NOFORN//20291123. RAGTIME is an Exceptionally Controlled Information compartment associated with FISA-authorised domestic collection programmes. The banner indicates the Agility tool itself is accredited to handle RAGTIME-compartmented data — meaning analysts using this tool were working within a FISA collection environment where all viewed content carried the RAGTIME classification regardless of individual document sensitivity. The two email subjects visible in the Agility viewer — "raporti UN la kurdistan" and "Impotence meds at best prices" — illustrate the nature of bulk collection, with a substantive human rights document sitting alongside pharmaceutical spam in the same intercept stream. The PDF properties identify the document as "Microsoft Word - HR Report April or 2007 AR" created with Acrobat Distiller 8.0.0 — this is the UNAMI (United Nations Assistance Mission for Iraq) Human Rights Report for 1 April to 30 June 2007, Arabic edition, whose contents include a section on freedom of expression in the Kurdistan Region. The title parses as Human Rights Report April-June 2007 Arabic, and the email subject "raporti UN la kurdistan" corresponds to the report's Kurdistan content. The Workflow section describes standing periodic queries that run automatically on a schedule — bridging the gap between selector discovery and tasking delivery to UTT/Cadence dictionaries, and enabling sustained monitoring without manual query repetition. Workflows are submitted to the XKeyscore team for review but are not reviewed for USSID-18 compliance, which is left entirely to the analyst. The site selection step explicitly notes that NOFORN selectors require deselecting second and third party sites. The workflow list visible in the My Workflows screenshot shows active and inactive queries targeting French MFA documents, Iraqi telecommunications, Pakistani IP blocks, Swedish phone numbers and a phone-number query named HERNANDEZION. The tips and tricks section reveals operational details about XKeyscore's query syntax — underscore wildcard behaviour, IP range regex construction, and Boolean operators — alongside a practical limitation: geolocation-based country filtering captures only 50-60% of actual traffic from a given country due to IP geolocation inaccuracy. The document's existence as an unofficial guide written by a private contractor employee (Booz Allen Hamilton, the same firm that employed Edward Snowden) is itself significant — confirming that commercial contractors had top secret access to ECI RAGTIME-accredited analytical tools processing FISA-authorised collection.
Programs:
XKEYSCORE, PINWALE, RAGTIME, AGILITY, METAFRAME, TIMBERLINE, YAKIMA, REDMINE, TUNINGFORK, FOGGYBOTTOM, FOXTRAIL, CADENCE, UTT, NKB, AIRGAP, DOPHONE

People mentioned:
bin Laden, Osama

Countries:
Afghanistan, Iran, Pakistan, Sweden, France, Iraq, Russia

Tags:
-

Identifier:
unofficial-xks-user-guide
Also known as:
EFF_-_20131211-svt-xkeyscore_slide_with_swedish_example
nsa-se-fra-xkeyscore-slide

PDF SHA-256:
a415f2f8e87868a167c19e9295483b05f1e153af906a6a491f151395d2d12aa4

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

The Unofficial XKEYSCORE User Guide

[REDACTED]
E92 - ADET
Consultant, Booz Allen Hamilton

The Unofficial XKEYSCORE User Guide...................................................................1
Creating Queries .....................................................................................2
Classic Queries.......................................................................................2
Multisearch: .........................................................................................2
Classic Searches (A-Z) ...............................................................................6
Creating a WorkFlow ..................................................................................18
Searching - Tips and Tricks ..........................................................................24
Which Query Is best for Me? ..........................................................................25

Derived From: NSA/CSSM 1-52
Dated: 20070108
Declassify On: 20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Creating Queries

Clicking on Search at the top of the screen will bring up a list of searches in the Navigation Menu:

Home Users Workflow Central Search Results Statistics

Navigation Menu

Search
  Classic
    MultiSearch
      IP Addresses
      Mac Addresses
      Username
    Classic A-M
      ASF and WMV Metadata
      Alert
      BlackBerry
      CNE
      Call Logs
      Category DNI
      Cellular DNI
      Cisco Passwords
      Document Metadata
      Document Tagging
      Email Addresses
      Extracted Files
      Full Log DNI
      HTTP Activity
      IRC Cafe Geolocation
      Logins and Passwords
    Classic N-Z

Help

Fields Advanced Features S

Search: Full Log

Query
Justif
Additional Justif
Miranda N
Da
Client IP (X-Forwarde
Use
Attribu
IP A
IP A

The Search screen has cascading menus of different Searches: Classic, Common, Dictionary Hits, File Transfer, Multisearch, Network Management, User Activity, VoIP, and Wireless.

Classic Queries:

Within the Classic Menu there are three folders: MultiSearch, Classic A-M, and Classic N-Z.

Multisearch:

Expand the Multisearch folder by clicking on the plus sign:

2
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Navigation Menu

Search
  Classic
    MultiSearch
    Classic A-M
    Classic N-Z

Search
  Classic
    MultiSearch
      IP Addresses
      Mac Addresses
      Username

Multisearch IP Address:

The Multisearch IP Address query allows you to search on an IP address into seven different searches. Think of it as a federated query using an IP address. The Multisearch IP Address query searches on:

  • User Activity
  • Phone Number Extractor
  • Email Addresses
  • Extracted Files
  • HTTP Activity
  • Full Log
  • Web Proxy

Refer to some of the individual searches below for more information about specific queries

Creating a MultiSearch IP Address Query:

When you have filled in your query name, justified it, entered an IP address, selected your search engines and sites the last thing is to submit the query. If you select "Merge Results", then all of your individual queries will be merged into one consolidated result.

"Why would I want to merge my results?"
If you wanted to see all of the activity together to get a 'big picture' look at the IP address, regardless of the activity or application that is on the IP. The New GUI's results screens allow you to filter your results easily which may make viewing your results more intuitive. See "Viewing Your Results" in this Guide.

"What would I want to NOT merge my results?"
Viewing the results individually allows you to focus on a particular activity or result (e.g. Documents or email addresses).

3
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Multisearch MAC Address:

The Multisearch MAC Address query is exactly the same as the IP address query except it only allows you to search on a MAC address. Follow the same instructions as the Multisearch IP Address query above but replace the IP address with your MAC address(es).

Fields Show Hidden Search fields Clear Search Values Reload Last Search Values

Multiple Search: Mac Address

Query Name:
Justification:
Additional Justification:
Miranda Number:
Datetime: 1 Day Start: 2009-01-22 00:00
Mac Address:
Content Must Exist:
Max Results for a Single DB:

Search Forms
Clear

User Activity
Phone Number Extractor
Email Addresses
Extracted Files
HTTP Activity
Full Log
Web Proxy

Save in my Favorites No
Load From my Favorites

Multisearch MAC Address looks just like the Multisearch IP Address query except you must now search on a MAC

4
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Multisearch Username:

As you may have guessed, the Multisearch Username query is exactly the same as the IP Address Query and the MAC Address except it only allows you to search on a target's Username. Follow the same instructions as the Multisearch IP Address query above but replace the IP address with your Username(s).

Fields Show Hidden Search fields Clear Search Values Reload Last Search Values

Multiple Search: Username

Query Name: abujihad
Justification: c: targetin afganistan
Additional Justification:
Miranda Number:
Datetime: Custom Start: 2009-01-22 00:00
Username: abujihad
Domain: yahoo.com
Content Must Exist:
Max Results for a Single DB:

Search Forms
Clear

User Activity
Email Addresses
Full Log
Logins and Passwords

Save in my Favorites No
Load From my Favorites

xks-central.corp.nsa.ic.gov:q0

Type in the username and domain (without the "@" symbol)

"What is a Username?"
A "Username" in XKEYSCORE queries is the portion before the "@" symbol in an email address.

For example:
Abujihad@hotmail.com:
Username = abujihad
Domain = yahoo.com

5
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Classic Searches (A-Z)

There are 32 different searches between the A-M and N-Z searches. This guide will cover some of the most common searches. You will notice that most of the fields of the searches are the same and each individual query will be unique because based on its query name. For example, the Extracted Files search has fields that are only applicable to file attachments (e.g., file names, file extensions) and the Email Addresses query has fields for email addresses (e.g., username and domains). All of the Classic queries will have common fields like Ports, IP addresses, Countries, SIGADS, and CaseNotations that you can use to

Fields Advanced Features Show Hidden Search Fields Copy Search Values Reload Last Search Values

Search: Email Addresses

Query Name: [REDACTED]
Justification:
Additional Justification:
Miranda Number:
Datetime: 1 Month Start: 2009-02-24 00:00 Stop: 2009-
Email Username:
@Domain:
Subject:
IP Address: From
IP Address: To
Port: From
Port: To
Country:
Country:

Fields Advanced Features Show Hidden Search Fields Clear Search

Search: Phone Number Extractor

Query Name: [REDACTED]
Justification:
Additional Justification:
Miranda Number:
Datetime: Month Start: 2008-12-24 00:00 Stop: 2009-01-23
Phone Number:
Number Type:
Country Code:
Area:
IP Address: From
IP Address: To
Port: From
Port: To
Country: From
Country: To

Here are two Classic queries: Email Addresses and Phone Number Extractor.

The fields between Datetime and the IP Addresses are the plug-ins unique to each query.

The Email Address query is catered to querying on email addresses.

The Phone Number Query has phone number fields.

6
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Email Addresses Query:

One of the most common queries is (you guessed it) an Email Address Query searching for an email address. To create a query for a specific email address, you have to fill in the name of the query, justify it and set a date range then you simply fill in the email address(es) you want to search on and submit.

That would look something like this...

Fields Advanced Features Show Hidden Search Fields Clear Search Values Reload Last Search Values

Search: Email Addresses

Query Name: abujihad
Justification: ct target in n africa
Additional Justification:
Miranda Number:
Datetime: 1 Month Start: 2008-12-24 00:00
Email Username: abujihad
@Domain: yahoo.com

NOTE: You DO NOT have to know an email address to use the Email Address Query. You can also search on an IP address*, domain name**, country, port, casenotation, protocol, SIGAD, MAC address, PID and more. If you search on something other-than an email address (e.g., an IP address), your results will be all of the email addresses seen on those IPs.

* The IP must be hosted OUTSIDE 5-eyes countries
** The Domain MUST be foreign owned. Check WHOIS and NSLOOKUP for more info on your domain before-hand

7
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Extracted Files Query:

  1. To find a specific file (i.e., if you already know the file name): For example, if you noticed a file name in your target's inbox and you never actually got the file attachment. This is VERY common for webmail collection because the attachment is often not put into PINWALE with the email.
Fields Advanced Features Show Hidden Search Fields Clear Search Values Reload Last Search Values

Search: Extracted Files
Query Name: ctFilesearch
Justification: Iranian Nuke files
Additional Justification:
Miranda Number:
Datetime: Custom Start: 2008-01-24 00:00 Stop 2009-0
Extracted Filename: nuclearindustry2343.pdf
Extension:
File Type (MIME Type):
Is Obfuscated (yes/no):
Obfuscated Real File Extension:
  1. To search for all files or specific file types on a particular area or on a network. (E.g., IP address). This is a GREAT query if you have a foreign mail server and want to see what files are collected on that IP address.
Fields Advanced Features Show Hidden Search Fields Clear Search Values Reload Last Search Values

Search Extracted Files
Query Name: ctFilesearch
Justification: Iranian Nuke files
Additional Justification:
Miranda Number:
Datetime: Custom Start 2008-01-24 00 00
Extracted Filename:
Extension:
File Type (MIME Type):
Is Obfuscated(yes/no):
Obfuscated Real File Extension:
File Size on Disk:
File Created:
File Last Access:
File Last Modified:
IP Address: 2 05 227 Other
IP Address To:

If you leave the Extracted Filenames field blank, you are wildcarding the search to look for ALL files names

The IP Address of the mail server you found using NSLookup in Foxtrail or your non-attrib Airgap account goes in the "IP Address" field

8
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Logins and Passwords

  1. If you already know the login and/or password.
If you know the logins or passwords,
query on them as long as they are unique
and will comply with USSID-18.

Fields Advanced Features Show Hidden Search Fields Clear Search Values Reload Last Search Values
Search: Logins and Passwords

Query Name: LoginsAfghanIP
Justification: Afghanistan network mail server password
Additional Justification:
Miranda Number:
Datetime: 1 Month Start: 2008-12-24 00:00 Stop 2009-01-23
User Name: admin
Password: w2cik[REDACTED]
Domain:
IP Address: From
IP Address: To
Port: From
Port: To

"Where would I find passwords to use in this query?"
Passwords can be found in TUNINGFORK (e.g., FoggyBottom), passed in the content of emails or text messages, or from previous XKEYSCORE queries.

  1. Trying to discover logins and passwords on a network? NOTE: Logins and passwords are valuable tools to enable Tailored Access Operations (TAO).

"What tools would I use to get the network information like a Mail Server, or Name Server?"
NS Lookups tools on NSA net such as FOXTRAIL and Open Source tools such as robtex.com, centralops.net, and network-tools.com are a GREAT START. They provide you with IP addresses for domains. You can then query on the foreign-hosted IP addresses.

9
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Fields Advanced Features Show Hidden Search Fields Clear Search Values Reload Last Search Values
Search: Logins and Passwords

Query Name: LoginsAfghanIP
Justification: Afghanistan network mail server password
Additional Justification:
Miranda Number:
Datetime: 1 Month Start: 2008-12-24 00:00 Stop 2009-01-23
User Name: admin
Password: w2cik[REDACTED]
Domain:
IP Address: 210[REDACTED]
IP Address:

If you are trying to FIND logins and passwords
and you know the IP address for the network,
then search on the IP!! Your results will be....LOGINS
and PASSWORDS!

Phone Number Extractor

The Phone Number Extractor query looks through the content of an email for phone numbers. This is very similar to a PINWALE DoPhone query except the traffic that XKEYSCORE finds may be survey (i.e., unselected, non-tasked data) and might not be in PINWALE. XKEYSCORE may be your only hope at finding an email address for a target where you only have their phone number as lead information.

  1. Already have a phone number? If all you have to start with as lead information is a phone number, you may find it useful to query on that phone number and see if anyone sent an email with that number in the signature line.
Fields Advanced Features Show Hidden Search Fields Clear Search Values Reload Last Search Values
Search: Phone Number Extractor

Query Name: Afghan
Justification: Afghanistan phone number ct target
Additional Justification:
Miranda Number:
Datetime: 1 Month Start: 2008-12-24 00:00 Stop 2009-01-23
Phone Number: [REDACTED]
Number Type:
Country Code:
Area:
IP Address: From
IP Address: To

10
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

  1. Looking for any phone numbers on a network? Quite often you know the mail server IP address and could use some telephone numbers to task?
Fields Advanced Features Show Hidden Search Fields Clear Search Values Reload Last Search Values
Search: Phone Number Extractor

Query Name: Afghan
Justification: Afghanistan phone number ct target
Additional Justification:
Miranda Number:
Datetime: 1 Month Start: 2008-12-24 00:00 Stop 2009-01-23 23:59
Phone Number:
Number Type:
Country Code:
Area:
IP Address: 210[REDACTED] From
IP Address: To

11
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

  1. Looking for a phone number without the country code (non-normalized)? It's possible a target will pass their phone number without the country code (e.g. a signature line with "Tel: 5354658"). In that case, XKEYSCORE will not find the number with the country code so you must create a query that looks for fewer digits but still complies with USSID-18. This is not a 100% solution* but ANDing your query with a country or IP address would certainly be more compliant. See example below:
Phone Number: [REDACTED]
Number Type:
Country Code:
Area:
IP Address: From
IP Address: To
Port: From
Port: To
Country: PK Either
Or
Phone Number: [REDACTED]
Number Type:
Country Code:
Area:
IP Address: 210[REDACTED] Either
IP Address: To

The number you enter here isn't
normalized because you expect to see
it in traffic without the country code.
To make this USSID-18 Compliant
you must AND this with something
like a country or IP address.

This example shows traffic in/out of
Pakistan.

This example shows traffic
in/out of a particular
network/IP Address

*If you ask XKEYSCORE to give you all Pakistani traffic, it's doing an NKB lookup on all Pakistani registered IP addresses. Geolocation of IP addresses is not 100% accurate at this time. Unofficial estimates say asking for all of Country X's traffic will find between 50-60% of the actual traffic. (That's more than 0%, though, right?)

12
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

HTTP Parser

The HTTP Parser query looks for web activity (remember, HTTP = web) on a particular link. This query is useful for several reasons. Firstly, if you know a particular website and want to see if a foreign target visits it (e.g. an extremist web forum URL, or maps.google.com). Secondly, this query enables you to query on a network IP(s), casenotation, or country and see what websites we don't know about (survey-type query).

Here are two examples

  1. If you know the particular website the target visits. For this example, I'm looking for everyone in Sweden that visits a particular extremist web forum.
Search HTTP Activity
Query Name: HTTP_in_Sweden
Justification: SwedishExtremist website visitors
Additional Justification:
Miranda Number:
Datetime: 1 Week Start: 2009-01-20 00
HTTP Type:
Host: *al-hisbah.com

Scroll down to enter a country code (Sweden is selected

Country: SE Either
Country: To

The website URL (aka "host) is
entered in with a wildcard to
account for "www" and "mail"
other hosts.

To comply with USSID-18 you
must AND that with some
other information like an IP or
country

13
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

  1. If you don't know the website but you know the network information (IP). For this example, I'm querying on a network IP block to see all of the websites the target visits.
Search: HTTP Activity
Query Name: WebUseIRaniano
Justification: Websites For Iranian University
Miranda Number:
Datetime: 1 Week Start: 2009-01-20 00:00 Stop: 2009-01-27
HTTP Type:
Host:
URL Path:
URL Args:
Search Terms:
*Language*:
Character Encoding:
Content Start:
Content Stop:
Content total:
Referer:
X-Forwarded-For:

The website URLs (hosts) are
left blank to wildcard those
fields.

To comply with USSID-18 you AND that with some other information like an IP or country

IP Address: 210.[REDACTED] Either
IP Address: To
Port: From
Port: To

Results from an HTTP Parser query

This shows what the results from a query look like for an HTTP Parser query:

Help Reports View
State ID Datetime DatetimeEnd HTTP Type Host URL Path Url Args Search Terms
1 1 2009-01-27 11:44:25 2009-01-27 11:44:25 post f-gaming.com /s/stat.php
2 2 2009-01-27 11:13:43 2009-01-27 11:13:43 post mambau.ru i/stat.php
3 3 2009-01-27 11:55:45 2009-01-27 11:55:49 post candy-country.com .newspapka/s
4 4 2009-01-27 11:39:58 2009-01-27 11:39:08 post mambau.ru /i/stat.php
5 5 2009-01-27 11:25:12 2009-01-27 11:25:14 post candy-coyntry.com .newspapka/s
6 6 2009-01-27 11:09:41 2009-01-27 11:09:42 post mambau.ru /i/stat.php
7 7 2009-01-27 11:29:47 2009-01-27 11:29:51 post mambau.ru /i/stat.php
8 8 2009-01-27 11:06:32 2009-01-27 11:06:32 response

Example 1 above shows a person was visiting www.f-gaming.com/s/stat.php
Host = f-gaming.com
URL Path = /s/stat/php

14
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Document Metadata

Document Metadata query allows you to search on document authors, organization, encryption*, and many other things about a document. This is extremely helpful if you have found a file attachment from a target (e.g. Brick-and-mortar targets, person, or Organization) and you want to see all of the other files they have sent. With the Document Metadata query you don't have to know the email address of the person sending the document, you just have to know the document's properties.

*Most Microsoft Office allows uses to encrypt files by clicking Tools -> Options -> Security and password protecting the files. The Document Metadata query looks for that type of encryption. It doesn't look for PGP or other 3rd party encryption.

"How do I find a document's properties?"
The easiest way to see this is to open a MS Office document and click on File -> Properties. To find the document properties for a file you target sent, the easiest way is to view the file in Agility and click on Properties.

Finding your target's file properties

If you can view the target's document in Agility, click on the Properties tab to show the target's Organization and/or Author. If the fields are unique or random enough you can query on the term itself. If the Organization or Author aren't enough to comply with USSID-18, then you must AND that query with supporting information (IP or Country).

15
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Displaying MS Word document in Agility:

Agility-MetaFrame Presentation Server Client
File Edit Tools Help
TOP SECRET//COMINT-ECI RAGTIME///NOFORN//20291123
[REDACTED] [REDACTED]
[REDACTED] [REDACTED]
[REDACTED] [REDACTED]
[REDACTED] [REDACTED]
[REDACTED] [REDACTED]

3898   application/octet-stream   bas
 462   application/msword         bas
  50   application/msword         bas

Display signal | Raw | SMTP Header | Properties | Control | CSD | Trailer | Collected Doc
Document Properti
Category
Company
HiddenSlideCount      0
LineCount             29
LinksUpToDate         False
Manager
MMClipCount           29
NoteCount             29
ParagraphCount        8
PresentationTarget
ScaleCrop             False
SlideCount            8
AppName               Microsoft Word 10.0
Author                [REDACTED]
CharacterCount        3582
Comments
DateCreated           6/12/2006 3:13:00 AM
SecurityLevel         none
Keywords
LastAuthor            [REDACTED]
LastPrinted           12/30/1999 12:00:00 AM

Author      = [REDACTED]
Last Author = [REDACTED]

To create a query in XKEYSCORE from this information:

Search: Document Metadata

Query Name: foriegndocuments
Justification: Swedish CT Target
Additional Justification:
Miranda Number:
Datetime: 1 Week Start: 2009-01-20 00 00 Stop: 2009-01-
Document Type:
Encrypted?:
Corrupted?:
Filename:
Extension:
"Subject":
*Creation Time*:
*Last Modified Time*:
*Unique ID* [fulltext]:
Author: GoGo
Last Author: [REDACTED]

16
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

View of document properties of PDFs in Agility:

Agility-MetaFrame Presentation Server Client
Agility Viewer - NewQuery_20071019121728.ago
File Edit Tools Help
SR1 | Mail | Web | Chat | Other
TOP SECRET//COMINT-ECI RAGTIME///NOFORN//20291123
To From Date Subject Type From
[REDACTED] [REDACTED] 10/19/2007 9:29:21 PM raporti UN la kurdistan
[REDACTED] [REDACTED] 10 2007 9:52:49 AM Impotence meds at best prices

Display original | Raw | SMTP Header | Properties | Control | CSD | Trailer | Collected Doc | Search Kwd | Dictionary Kwd | SBI

PDF Properties

ModDate       D:2
CreationDate  D:2
Title         Microsoft Word - HR Report April or 2007 AR
Creator       PScript5.dll Version 5.2
Producer      Acrobat Distiller 8.0.0 (Windows)
Author        [REDACTED]

Author = [REDACTED]

TOP SECRET//COMINT-ECI RAGTIME///NOFORN//20291123

To create a query in XKEYSCORE using this information:

Search: Document Metadata

Query Name: foriegndocuments
Justification: Swedish CT Target
Additional Justification:
Miranda Number:
Datetime: 1 Week Start: 2009-01-20 00:00 Stop: 2009-01-27
Document Type: pdf
Encrypted?:
Corrupted?:
Filename:
Extension:
*Subject*:
*Creation Time*:
*Last Modified Time*:
*Unique ID* [fulltext]:
Author: [REDACTED]

Author = [REDACTED]

17
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Creating a WorkFlow

Workflows are periodic queries you can set up that run at specified times. They are great for sustained targets because they query the database for you (e.g. every night) and you can easily view the recently collected traffic without having to create a new query each day. They are also very helpful if you are performing target discovery on a network and haven't seen much traffic yet on a selector. A workflow for an email address can bridge the gap between when you discover the selector (and you task it to UTT/Cadence) and when it actually makes it to the appropriate dictionaries).

It's important to understand that a normal (ad hoc) query is submitted when you hit Submit. Workflows, on the other hand, are created then submitted to the XKEYSCORE team for review. The XKEYSCORE team does not review it for USSID-18 compliance (that's up to you); they only review it to ensure your query won't strain the system with too complex a query.

The first step in creating a Workflow is click on Workflow Central:

XKEYSCORE
Home Users Workflow Central Search Results Statistics Preferences Help
Navigation Menu Histo :Query Type
Results Clear Selection

Then click on Request on the left to start the Workflow Request Wizard, and then click Next.

18
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Workflow Central Request Wizard

Welcome to the X KEYSCORE Workflow Request Wizard.

This wizard will guide you through the process of creating a Workflow.

A Workflow is an standing query that will execute at a specific time or interval.
In addition, a workflow can execute follow-on actions once the query has
completed. Follow-on actions are scripts that perform certain tasks such as
sending email or creating a summary of the results through specified SQL
queries.

Upon completion of this wizard the workflow will be submitted for approval.
Cancel Prev Next Submit

Next, select the search type you want to create from the pull-down menu. For this, I'm selecting an Extracted Files query. These queries are essentially the Classic A-M and N-Z queries you have seen in the Classic Search screens. The only difference is an Extracted Files workflow will start looking for extracted files in the future and an ad hoc Extracted Files query will search in past/previous collection.

Workflow Central Request Wizard
Please select a Search Type.
Alert
ASF ID Data
BlackBerry
Cal Logs
Category (DNI)
CELLULAR DNI
Cisco Passwords
CNE
Document Metadata
Email Addresses
Extracted Files
Full Log
Wireshark
HTTP Activity
IRC Cafe Geolocation

19
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Next, fill in the name of your query ("AfghanFiles"), the auditor-compliant justification, and how often you want the query to run. I recommend offsetting the time from the default of midnight (2400) by a few hours (before of after). For this, I'm selecting 0400. Then hit NEXT.

Workflow Central Request Wizard
Basic Information
Query Name: AfghanFiles
Query Justification: CT target in afghanistan
Additional Justification:
Miranda Number:
Run every: 24 Hours - Starting at 4:00 EST
Recurring Search One Time Search
Basic Features Help
Cancel Prev Next Submit

In the Add Search Fields window, you will select the search criteria that you want to search on. In this example, I'm looking for specific file attachment (DOC or PDF or XLS or PPT) on a specific Afghanistan IP address.

You must hit the green "+" symbol to enter the search criteria.

20
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Workflow Central Request Wizard
Add Search Fields
Search Field | Search Value | Remove
Extension | DOC or PDF or XLS or PPT
From IP Address OR To IP Address | 203[REDACTED]
Datetime
Single Field Search Multiple Field Search
Search Value Help
Cancel Prev Next Submit

Click Next

Single Field Search only searches in one field (e.g. File Extensions)

Multiple Field Search allows you to search on several fields (e.g., To IP AND From IP)

Add Search Fields
Search field
Extension
From IP Address OR To IP Address
File Last Modified
From IP Address
To IP Address
From Port
To Port
Single Field Search Multiple Field Search
Search Value Help

Next, you will select the sites where you want your query to run. Scroll down in this window to use the convenient "Select All" or "Uncheck All" buttons.

NOTE: If your selector is NOFORN, you must DESELECT sites that are 2nd/3rd party.

21
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Texas (m1vwal.tex.f.nsa:xs_web_db)
TIMBERLINE (timberline:xs_web_db)
xks voip 1 (xksvoip1:q0)
xks voip 2 (xksvoip2:q0)
Yakima mission system (keyscore2a.yrs.f.nsa:xs_web_db)
Content must exist
Check All
Uncheck All
Basic Features Help
Cancel Prev Next Submit

Click Next

Follow-on Actions tell XKEYSCORE to do things after it runs your query. For example, it can email you with the results, or it can send them to Agility, or any combination of the two. For this example, I want XKEYSCORE to email me telling me I have results and I want it to download my results to Agility. Make sure you select Send to Agility if you want the same.

Workflow Central Request Wizard
Follow-on Actions
Would you like to add any follow on actions
No
Yes
Script Arguments Add
User ID [REDACTED]
Email To [REDACTED]@nsa.ic.gov
Email Subject: You have new
Email Content: Go check Agility My men
ROWR: Return Only With Results
Filename:
Mail Trigraph:
GZIP: Compress Contents
Send To Agility: Send To Agility

Cancel Prev Next Submit

Click the Green Add symbol, and then click next when finished.
On the next screen, enter any comments you wish (optional) and click Next

22
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Lastly, click SUBMIT. Your query isn't active yet. The XKEYSCORE team will review it and you will have to check back later and turn the query ON or OFF as you wish.

Navigation Menu
Workflow Control
  Request
  My Workflows

My Workflows

Help Actions

Query Type | Query Name | Last Modified | State | Actions
phone_number | HERNANDEZION | 2009-01-19 21:47:40 | on (xks) |
phone_number | laras_phones | 2009-01-19 21:47:40 | on (xks) |
phone_number | swedishPhones_Cruz | 2009-01-19 21:47:40 | on (xks) |
phone_number | Fr_MFA_Doc_2008 | 2009-01-19 21:47:21 | off |
phone_number | Iraq_Neta2006_telecom | 2009-01-19 21:47:21 | off |
phone_number | MultinetPhones | 2009-01-19 21:47:21 | off |
http_parser | pakistanIP_XRI | 2009-01-19 21:47:21 | off |
http_parser | Webforum_2_0 | 2009-01-19 21:47:21 | off |
http_parser | web_forum2 | 2009-01-19 21:47:21 | off |
http_parser | web_forum3 | 2009-01-19 21:47:21 | off |
phone_number | ZTO_OPSUMMERCLEA | 2009-01-19 21:47:21 | off |

23
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Searching - Tips and Tricks

The Official XKEYSCORE Frequently Asked Questions page is located here:
http://xkeyscore.r1.r.nsa/redmine/wiki/xkeyscore/FAQ. Here are some other tips/tricks that may be useful

  1. Underscores in usernames:

If your selector has an underscore in it, you must precede the underscore with a backslash. For example: abu_jihad would become searched as abu\_jihad. If you leave the underscore in the query without the backslash, you are wildcarding a single character (see below).

To search on: abu_jihad@hotmail.com:
Bad query: Abu_jihad
Good query: Abu_jihad

If you search on "abu_jihad" (without the backslash), you could bring back "abu1jihad", "abuTjihad", "abuSjihad", "abu-jihad", etc... because you are wildcarding that character and therefore you would be pulling on an entirely different selector.

  1. To search on a range of IP addresses:

IP Address Range:
202.82.86.224 - 202.82.86.244

Becomes this XKEYSCORE Query (entered in the IP Address as To, From, or Either):
regex:202\.82\.86\.22[4-9] OR regex:202\.82\.86\.23[0-9] OR regex:202\.82\.86\.24[0-4]

  1. Boolean Search Descriptions (Wildcards, ANDs, ORs, etc):
OPERATOR DESCRIPTION USAGE
! Not Equal Comparison beginning of word (i.e. !joe and !sam)
or Logical OR (Search for multiple values) between words (i.e. osama or laden)
and Logical AND (Search for combination value matches) between words (ie. *osama* and *laden*) takes precedence over ORs
* Multiple Character Wildcard anywhere in word (i.e. *osam*bin*laden)
_ Single Character Wildcard anywhere in word (i.e. _sam__bin_laden)
> Greater Than Comparison beginning of word (i.e. >00080 and <00111)
< Less Than Comparison beginning of word (i.e. >00080)
regex: REGEX Expression (i.e. to retrieve only numbers: regex:[0-9]*)

24
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Which Query is best for me?

Quite often the most difficult part of using XKEYSCORE is deciding which query to use at which time. Here's a rough guide to help you decide.

Do you have an IP Address and want to learn more about that network

Which XKEYSCORE Query is Best for Me?
I have....
a/an
IP Address
for a
Mail/Web Server
(e.g. from a NS lookup)
and need to know
Which email addresses are seen on the network?
you'll use
The EMAIL ADDRESS Query and search on the IP
Have we seen any phone numbers (e.g. in sig lines) on the network?
you'll use
The PHONE NUMBER EXTRACTOR query and search on the IP
Have we collected any files or attachment on the IP/network?
you'll use
The EXTRACTED FILES query and search on the IP
Logins and Passwords for the server for TAO
you'll use
The LOGINS/PASSWORDS query and search on the IP (hint: Search or sys admin or valuable ports like 21, 23, 25)
Which websites people on the network visit (e.g. Google Earth, web forums)
you'll use
the HTTP ACTIVITY query and pull on the IP

25
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Do you have an Email Address or Foreign Domain And want to learn more about it?

Which XKEYSCORE Query is Best for Me?
I have.......
a/an
Email Address or Domain (foreign)
and need to know
Which IP addresses the target connects to
Which sites collect the user?
Are we collecting the user or domain?
you'll use
The EMAIL ADDRESS query and search on the email (in username) and domain (in domain)

26
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108

Do you have a phone number for your target and want to learn their email address?

Which XKEYSCORE Query is Best for Me?
I have.....
a/an
A Phone Number
and need to know
The target's email address
then use
The PHONE NUMBER EXTRACTOR query and search on the PHONE NUMBER(S).

27
TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108