Tag: metadata 142 documents
Search documents matching “metadata” →Free-text subjects the curators added. Unlike the other axes these are not necessarily from the documents' own vocabulary, so they are the loosest thing to browse by (and the most forgiving).
Graphs
By document date (11 years)
By Banner
By Classification
Refine (combine with another filter, via search)
Sort: Title
Documents
-
Data Acquisition Update SIDtoday [pdf]
SID's deputy director for Data Acquisition gives an update on what his department has been doing. He points out that the capture of terrorist Umar Baziyani was an example of seamless interaction between many diverse parts of NSA, an operation that included "cryptoanalytic exploitation using special purpose hardware." He outlines some of Data Acquisition's activities to build tools and infrastructure to make similar operations much easier in the future, including the establishment of the Global Network Forum, "a 5 Eyes construct to manage midpoint accesses on a community basis."
People: Baziyani, UmarThe Intercept ↗ (2017-04-24) -
Dealing With a 'Tsunami' of Intercept SIDtoday [pdf]
As part of NSAs efforts to implement Human Language Technology, prototype processors located at collection sites will soon be able to query content at "line speed" based on the language, it is written in.
People: Kulman, Anita H.The Intercept ↗ (2019-05-29) · “Inside NSA, Officials Privately Criticize "Collect It All" Surveillance” - The Intercept ↗ (2015-05-28) · Maass, Peter -
Decision Memorandum for the DNI on ICREACH [pdf]
This 2006 memorandum from the Director of the NSA, Keith B. Alexander, requests that the Director of National Intelligence authorize the NSA to create ICREACH, a new Intelligence Community-wide system for sharing communications metadata. The proposal designates the NSA as the executive agent for this new structure, which would replace the older, less scalable PROTON program and grant unprecedented data access to partner agencies.
People: Alexander, Keith B. -
Deep in the Heart of Texas with Cindy Farkus and 'Party' SIDtoday [pdf]
Deputy Director Cindy Farkus visits NSA/CSS Texas in San Antonio, holds a town hall meeting, and meets with Signals Intelligence Directorate leadership.
Programs: HIGHCASTLEThe Intercept ↗ (2018-08-15) -
Deployment of New System Improves Access to Iranian Communications SIDtoday [pdf]
A new collection system deployed to Kuwait is capable of collection of Iranian satellite ground stations used by the Revolutionary Guard, when previously only one side of the traffic. This has resulted in valuable intelligence on Iran's activities in Iraq, including its influence on the Iraqi government.
The Intercept ↗ (2018-08-15) -
Digint Imbalance [pdf]
This secret MI5 report highlights a growing imbalance between the massive scale of digital intelligence (DIGINT) collection and the agency’s limited capacity to analyze it, leaving much data unassessed and creating risks of intelligence failure. It identifies major challenges in staffing, tools, and infrastructure, and calls for improved analysis, better integration with other intelligence, and new capabilities to disrupt online threats.
Marking: SECRET -
Elegant Chaos: collect it all, exploit it all [pdf]
This document outlines a classified project, "Elegant CHAOS," focused on advanced signals intelligence (SIGINT) collection, processing, and analysis using cutting-edge technologies like automated surveys and modems. It highlights goals such as real-time data analysis, collaboration with international partners, and integration with cloud-based platforms to manage massive data. See also the document with notes (by GCHQ): "Elegant-Chaos-collect-it-all-exploit-it-all-plus"
Programs: ELEGANTCHAOS · ASPHALT · ASPHALT-A · ASPHALT-A-PLUS · STORMFORCE · DARKQUEST · TORUS · XKEYSCORE · MHS CLOUD · TURMOIL · TARMAC · ASDF · SLR · POPQUIZ · WEALTHYCLUSTER2 · TRAVELLINGWAVE · BILBOBADGER · DRINKYBIRD · GLOBETROTTER · MASTERSHAKE · REST · TRAFFICTHIEF · VENUSAFFECT · FOGHORN · MATCHMAKER · ROADBED · MOONPENNY · AMULETSTELLAR · CLOUD · WEALTHYCLUSTER · MVR · TINT · IPGEOTRAP · MARINA · PLUS -
Elegant Chaos: collect it all, exploit it all (plus notes) [pdf]
GCHQ/NSA presentation describing ELEGANT CHAOS — an automated system at the Menwith Hill Station satellite interception facility that scores and ranks thousands of intercepted satellite signals by intelligence value, automatically deciding which ones deserve collection resources based on criteria like geographic location, target network hits, technology type and Computer Network Operations interest — built to implement the station's directive to "Sniff it All, Know it All, Collect it All, Process it All, Exploit it All, Partner it All" as collection capacity scaled from 100 signals to a planned 3,000 simultaneous intercepts.
Programs: ELEGANTCHAOS · ASPHALT · ASPHALT-A · ASPHALT-A-PLUS · STORMFORCE · DARKQUEST · TORUS · XKEYSCORE · MHS CLOUD · TURMOIL · TARMAC · ASDF · SLR · POPQUIZ · WEALTHYCLUSTER2 · TRAVELLINGWAVE · BILBOBADGER · DRINKYBIRD · GLOBETROTTER · MASTERSHAKE · REST · TRAFFICTHIEF · VENUSAFFECT · FOGHORN · MATCHMAKER · ROADBED · MOONPENNY · WHIZBANG · CLOUDBASE · APLUS · MAILORDER · CROSSBONES · AMULET STELLAR · MVR · TINT · WEALTHYCLUSTER · IPGEOTRAP · MARINA · CLOUD · PLUS -
Email Address vs User Activity [pdf]
This 2009 NSA training presentation explains how to use the XKEYSCORE system to search raw SIGINT, detailing the differences between the "Email Address" query, which finds email addresses in full content, and the more flexible "User Activity" query, which can search for selectors like cookies, usernames, and phone numbers.
“XKEYSCORE: NSA’s Google for the World’s Private Communications” - The Intercept ↗ (2015-07-01) · “No Place to Hide” - Metropolitan Books ↗ (2014-05-13) · Marquis-Boire, Morgan · Greenwald, Glenn · Lee, Micah -
Event (SIGINT) [pdf]
5 p.This document explains that within GCHQ and NSA SIGINT operations, an “Event” refers to metadata about observable communications or user actions rather than message content, categorized into types like Communications, Presence, Social, and Geo-reference Events. These events are collected, stored in various databases, and analyzed through convergence and correlation processes, with standardized formats defined by Interface Control Documents such as INTERSTELLAR DUST (IDUST) and Actor Action (AA).
Programs: INTERSTELLAR DUST · HAUSTORIUM · SOCIAL ANTHROPOID · ACTOR-ACTION · SAMREF · GEOFUSION · HARD ASSOC · SALAMANCA · ASSOCIATION · BANYAN · MAINWAY · FASCIA · CULTWEAVE II · VOICESAIL · TEEDALE · PILBEAM · MUTANTBROTH · AUTOASSOC · MARBLEDGECKO · KARMAPOLICE · HRMAP · MEMORYHOLE · INFINITEMONKEYS · SOCIALANIMAL · AUTOTDI · SAMUELPEPYS · Mobile Applications Project · NGE · PPF · SLR -
Excerpt of May 2008 NSA Afghanistan Dataflow Analysis [pdf]
The document outlines how the NSA’s Real Time Regional Gateway (RT-RG) in Afghanistan progressively expanded from late 2007 through 2008 to ingest massive amounts of GSM, VHF, HF, and telephony metadata from dozens of locations across Afghanistan, supplemented by third-party “dirtbox” collection from allied nations such as the Netherlands, Denmark, Norway, Germany, and Spain. It details the specific collection sites, data types, and timelines (SPIN 9, 10, 11, etc.), illustrating how partner contributions and multiple signals sources fed into the RT-RG system to enable real-time intelligence.
Programs: FOURCORNERS · FORTRESS · WARRIOR · ECHIDNA · CASSIEOPIA · KARUS · SOUNDER · SNICK · TEMPOWOLFE · CALYPSO · TATTERAN · AIRBORNE · LEAVEBEHIND · UDEAUS · NELEUS · CROSSHAIR · SSO · DRT · RIVET JOINT · VOICESAIL“Mission Creep: How the NSA’s Game-Changing Targeting System Built for Iraq and Afghanistan Ended Up on the Mexico Border” - The Intercept ↗ (2019-05-29) · “Norges hemmelige krig i Afghanistan” - NRK ↗ (2019-05-29) · Moltke, Henrik · Skille, Øyvind Bye · Døvik, Olav · Hansen, Ståle Ellingsgard · Fjørtoft, Martin -
Exploiting Foreign Lawful Intercept (LI) Roundtable [pdf]
Top-secret PowerPoint presentation, "Exploiting Foreign Lawful Intercept Roundtable," highlights the agency’s strategy to exploit vulnerabilities in lawful intercept systems from telecom companies, such as Ericsson, Motorola, and Nokia, in countries like Mexico, Indonesia, and Egypt. The presentation also discusses NSA's use of various "Fingerprints" for identifying data and its collaboration with industry players and foreign lawful intercept systems.
“A Death in Athens” - The Intercept ↗ (2015-09-28) · “Americans and Greeks started the 2004 wiretaps together” - Kathimerini ↗ (2015-09-29) · Bamford, James · Petropoulos, Aggelos -
Exploiting US/UK/CAN Phone Numbers -- In Compliance with USSID-18 Policy SIDtoday [pdf]
The NSA can now classify certain U.S. phone numbers as foreign, allowing them to be targeted. Such targeting may occur on a U.S. number if the phone number is “identified on foreign links” and if it is associated with a “Pick Your Own Number” internet service such as Vonage.
The Intercept ↗ (2018-08-15) -
FAIRVIEW and STORMBREW: 'Live' - On the Net SIDtoday [pdf]
Early description of FAIRVIEW and STORMBREW, later known as Upstream. FAIRVIEW “is now forwarding more than 1 million emails a day to a keyword selection system.”
“NSA Spying Relies on AT&T’s ‘Extreme Willingness to Help’” - ProPublica ↗ (2015-08-15) · “AT&T Helped U.S. Spy on Internet on a Vast Scale” - The New York Times ↗ (2015-08-15) · The Intercept ↗ (2016-08-10) · Angwin, Julia · Larson, Jeff · Savage, Charlie · Moltke, Henrik · Poitras, Laura · Risen, James -
FAIRVIEW brief overview [pdf]
This document outlines the FAIRVIEW program, established in 1985, which leverages a corporate partner to exploit international communications through cables, switches, and routers. It is one of the NSA's top sources for metadata and serialized intelligence, supporting a wide range of targets including counter-terrorism, crime, and international security issues.
Programs: FAIRVIEW -
FAIRVIEW dataflow charts April 2012 [pdf]
This document outlines data flow diagrams that show how various types of sensitive communication, including voice, fax, and modem traffic, are handled through systems operated by the NSA and its partners. These partners include corporations, telecommunications companies, ISPs, and other entities involved in processing and managing the data through secure and classified channels. The document covers processing flows for various data types, metadata, and communications systems under NSA's control, detailing how data is routed, stored, and analyzed across different platforms and sites.
Programs: FAIRVIEW · SAGUARO · TURNSTILE · MESA · TITANPOINT · RIMROCK · MAILORDER · PINECONE · BLARNEY · STONEGATE · TINSEL · CONVEYANCE · FISHWAY · NUCLEON · PINWALE · SCISSORS · COURIERSKILL · DRIFTWOOD · ASDF · FALLOUT · MARINA · LOPERS · WEALTHYCLUSTER · PUPR · CLEARSIGHT · MIDFIELD · CADENCE · BUGCATCHER · WAYLAND · DESOTO · GATEKEEP · HIGHDECIBEL · XKEYSCORE · YANKEE · TOPROCK · FASCIA · DISHFIRE · SEAGULL · BIGDIPPER · CORPORATE · LILDIPPER · HAVASU · WEALTHYCLUSTER2 · RAGTIME · TUMULT · TATTOO · DARKTHUNDER · ICYTWINS · TML-STATUS · WATERFRONT · VULCANDEATHGRIP · STEELFLAUTA · UNITY · KEYCARD · UTT · BTUBE · ISLANDTRANSPORT · SSO · MAINWAY · SSDM · TUBE · TITANPOINTE · TURMOIL · VPCS“NSA Spying Relies on AT&T’s ‘Extreme Willingness to Help’” - ProPublica ↗ (2015-08-15) · “AT&T Helped U.S. Spy on Internet on a Vast Scale” - The New York Times ↗ (2015-08-15) · “Titanpointe: The NSA’s Spy Hub in New York, Hidden in Plain Sight” - The Intercept ↗ (2016-11-16) · Angwin, Julia · Larson, Jeff · Savage, Charlie · Moltke, Henrik · Poitras, Laura · Risen, James · Gallagher, Ryan -
FASCIA: The NSA's huge trove of location records [pdf]
FASCIA is the National Security Agency's enormous database containing trillions of metadata records, many with device locations, that are collected from a variety of sources. This document shows the volume and types of device-location data collected by the agency.
-
FASHIONCLEFT - Interface Control Document (15 June 2009) [pdf]
This 2009 NSA Interface Control Document provides comprehensive technical specifications for FASHIONCLEFT, the protocol that acts as a covert shipping system enabling TAO malware implants to send stolen network data back to NSA collection servers with minimal computer processing requirements—using multiple encryption layers (RSA-1024 and RC6-128) to disguise the traffic as it travels across the internet while allowing NSA's Common Data Receptor to reassemble and authenticate the original intercepted packets. The document details how implants make modified copies of captured network traffic (changing destination addresses to point toward NSA instead of legitimate recipients), apply various levels of encryption or scrambling ("munging") depending on how powerful the infected computer is, attach metadata instructions explaining how to reconstruct the original packets, and use pattern-matching algorithms to ensure NSA systems can identify which stolen packets belong to which collection operation.
Programs: FASHIONCLEFT · FLAXENPRECEPT · SHELLGREY · SURPASSPIN · SEAGULLFARO · ROOTKNOT · FIGBUILD · OPTICPINCH · VINYLSEAT · OLYMPUS · UNITEDRAKE · HOGTECH · SWEEPFORWARD · CONVEY · PINWALE · TREBLECLEF · FOGYNULL · FUNNELAPS · FLASHHANDLE -
Finding Genetic Sequences in SIGINT SIDtoday [pdf]
A cryptoanalysis intern describes her experience working for the Office of Tradecraft for Analysis, developing algorithms to answer specific questions from a haystack of metadata -- an entirely new type of problem than what cryptoanalysts typically tackle. The goal of her specific tour was to "gain general knowledge about genetic engineering research activity by foreign entities and to identify laboratories and/or individuals who may be involved in nefarious use of genetic research."
The Intercept ↗ (2017-04-24) -
FISA dataflow [pdf]
This document describes the technical SIGINT architecture for FISA data processing, specifically detailing how the RAGTIME/SQF partition and PALMCARTE system utilize unique case notations and metadata tools to route and track intercepted communications.