Tag: metadata analysis 41 documents
Search documents matching “metadata analysis” →Free-text subjects the curators added. Unlike the other axes these are not necessarily from the documents' own vocabulary, so they are the loosest thing to browse by (and the most forgiving).
Graphs
By Banner
By Classification
Refine (combine with another filter, via search)
Sort: Title
Documents
-
Analysis Conference — Call for Presentations SIDtoday [pdf]
The NSA will host a conference on analysis and is specifically looking for presentations about coordinating with new partners, recognizing and communicating about new and emerging "threats," and improving existing systems.
People: Mercurio, JoeThe Intercept ↗ (2017-09-13) -
Analysis Series Wrap-up: Continuous Learning SIDtoday [pdf]
Concludes the Analysis Series and lists resources for analysts to continue learning.
The Intercept ↗ (2016-08-10) -
AURORAGOLD Working Group [pdf]
The "AURORAGOLD Working Group" briefing is a highly classified SIGINT development presentation from June 2012 that outlines the NSA's progress in mapping the global mobile landscape (GSM/UMTS/LTE). It focuses on the project's "value proposition": providing an exhaustive database of network identifiers and technical configurations—often obtained by intercepting internal company documents—to support tactical military operations and long-term surveillance forecasting.
Programs: AURORAGOLD · UTT · XKEYSCORE · CADENCE · JUBILEECORONA · RONIN · NKB · GLOBALTIPPER · PINWALE -
Broadcast/Internet Radio Exploitation and Analysis, 6th November 2009, GCHQ [pdf]
This GCHQ report analyzes 6.68 million internet radio listening events collected over three months in 2009, detailing global broadcasting patterns, user behaviors, and specifically examining Islamic radio stations and Pakistan-UK connections for potential radicalization intelligence, while recommending integration with bulk surveillance databases like KARMA POLICE for enhanced user profiling.
-
Contact chaining [pdf]
A secret GCHQ Applied Research presentation describing how the agency's analysts use automated tools to map out a surveillance target's social network — starting with a known email address or phone number, then scoring and ranking every person that target communicates with to identify the most "suspicious" or strongly connected contacts, using twelve different relationship types including direct calls, shared locations, timing patterns, equipment sharing and calling profile similarity.
Programs: CARPAT · PRIME TIME · SNAPDRAGON · MoaG · SORTING FRIENDS · CHART BREAKER · MAMBA · CONTACTLENS · FLEXIMART · PSOUP · IMMINGLE -
CSEC Cyber Threat Capabilities [pdf]
This document details CSEC’s full-spectrum cyber operations architecture, integrating SIGINT and IT Security to conduct both defensive and offensive activities under the guise of “information superiority.” While early sections emphasize detection, anomaly analysis, and collaboration with Five Eyes partners, later portions reveal active measures including network infiltration (implant deployment), command-and-control tracking, traffic manipulation and redirection, DNS interference, and potential disruption or deception operations—illustrating CSEC’s capacity not only to defend Canadian networks but also to surveil, exploit, and influence foreign and domestic digital infrastructure.
Programs: EONBLUE · CYBERDMZ · SEEDSPHERE · PHOTONICPRISM · SPECIALSOURCE · SUPERDRAKE · DARKSPACE · QUANTUM · MANDATE · STREAMING SENTRY · SSO · TKB · COVENANT -
CSEC Cyber Threat Detection [pdf]
This document details the Communications Security Establishment’s (CSEC) cyber threat detection and intelligence infrastructure, including systems like EONBLUE, CROSSBOW, CRUCIBLE, and LODESTONE, which enable passive monitoring, anomaly detection, and botnet identification through metadata and deep packet inspection. It outlines how CSEC integrates SIGINT, cryptanalysis, and network forensics to detect and mitigate cyber intrusions using both SIGINT and domestic sensors, while collaborating with Five Eyes partners to enhance Canada’s cyber defense and situational awareness. The presentation is duplicated from page 7 onwards - one of the duplicated slide has internal CSE comments.
Programs: EONBLUE · SPECIALSOURCE · WARRANTEDACCESS · COVENANT · SEEDSPHERE · DIESELRATTLE · SIENNABLUE · DOWNGRADE · CROSSBOW · CRUCIBLE · LODESTONE -
Discovery SIGINT Targeting Scenarios and Compliance [pdf]
This document is a top-secret NSA guidance manual detailing SIGINT targeting scenarios, compliance procedures, and legal guidance for querying foreign actors, malware, and potentially U.S.-linked systems, including considerations for minimizing incidental collection of U.S. persons’ communications. It outlines specific rules, approvals, and exceptions for using SIGINT tools, foreign intelligence tasks, and cases involving ambiguous or dual-use selectors like WikiLeaks, The Pirate Bay, and foreign actors operating through U.S. infrastructure.
-
Events Analysis [pdf]
This GCHQ document explains authorization policies for querying metadata databases (SALAMANCA, HAUSTORIUM, THUGGEE, IMMINGLE), stating that metadata is "less intrusive than communications content" and that authorization is "not needed for individuals in the UK," though searches are logged and audited with necessity and proportionality considerations still applying.
-
For Media Mining, the Future Is Now! (conclusion) SIDtoday [pdf]
In the future, thanks to Human Language Technology (HLT), an analyst will be able to sort through millions on intercepted phone calls each day "and focus on only the small percentage that is relevant". He or she will also be able to get precise answers to queries such as "Where is the mayor of Baghdad?"
People: Picone, Joseph“How the NSA Converts Spoken Words Into Searchable Text” - The Intercept ↗ (2015-05-05) · The Intercept ↗ (2019-05-29) · Froomkin, Dan -
GCHQ – Making network sense of the encryption problem (2011) [pdf]
GCHQ’s Network Analysis Centre convened a 2011 roundtable on “making network sense” of encryption, pushing a metadata-first strategy to triage SSL/TLS, IKE, OpenVPN, and SSH flows, pivot from “seed” indicators, and zero in on targets and subnets worth decrypting. The playbook leans on enrichment, AS-level scoping (explicitly citing Belgacom AS6774 GRX VPNs), and “access optimisation” to collect data pre‑encryption—passively or actively.
Programs: SQUEAL · BEARDEDPIGGY · CLOUD · IRASCIBLEHARE · IRASCIBLERABBIT · IRASCIBLEMOOSE · IRASCIBLEEMITT“Operation Socialist: The Inside Story of How British Spies Hacked Belgium’s Largest Telco” - The Intercept ↗ (2014-12-13) · “Lees hier hoe de Britse geheime dienst GCHQ Belgacom aanviel” - NRC ↗ (2014-12-13) · “Britse geheime dienst bespioneerde jarenlang Belgacom-klanten” - De Standaard ↗ (2014-12-13) · Gallagher, Ryan · Modderkolk, Huib · Vanhecke, Nikolas · Eeckhaut, Mark -
Graph theory in the operational environment – GCHQ [pdf]
This document details GCHQ’s research on identifying “closed loops” of communication, where targets use a small set of phones to communicate exclusively among themselves, often with cheap, disposable handsets like the Nokia 1100. It explains the methodology for analyzing anonymized metadata, constructing contact graphs, and extracting non-giant, non-tree components to discover operationally relevant closed loops for intelligence and counterterrorism purposes.
Programs: VOICESAILPeople: Rumsfeld, Donald -
GRIMPLATE - First Steps Toward Identifying Adversarial Use of BitTorrent [pdf]
This document outlines the NSA's GRIMPLATE program, which was developed to monitor and analyze BitTorrent traffic on the Department of Defense's NIPRnet to identify potentially malicious activity. The program focuses on crawling the Distributed Hash Table (DHT) of BitTorrent networks to gather data on file-sharing nodes and peer connections, in order to detect and mitigate threats while building a case for halting unauthorized P2P usage on secure networks.
-
HIMR Data Mining Research Problem Book [pdf]
This document is a research roadmap created by GCHQ for the Heilbronn Institute for Mathematical Research (HIMR). Its purpose is to guide long-term mathematical and data-mining research that could improve signals intelligence (SIGINT) capabilities. It outlines key research problems, datasets, and collaboration plans between intelligence agencies and academic researchers. The main goal is to develop new algorithms and mathematical tools to analyze extremely large communications datasets produced by modern surveillance systems.
Programs: CLOUD · DISTILLERY · DISCOVER · PRESTON · LECKWITH · TERRAIN · BROADOAK · NGE · FIVEALIVE · CHARTBREAKER · FIRSTCONTACT · GEOFUSION · RADONSHARPEN-B · PRIMETIME · HIDDENOTTER · BAKERSDOZEN · CLASP · SALTYOTTER · BIRCH · SOLIDINK · FLUIDINK · MAMBA · GRINNINGROACH · PIRATECAREBEAR · AUTOASSOC · CROUCHINGSQUIRREL · HRMAP · KACHINA · SQUEAL · QUESTA · METEORSHOWER · SUNSTORM · GOLDMINE · HAGERAWEL · WOODY · BUZZ · SILVERLIBRARY · SILVERLINING · VALHALLA · MOUNTMCKINLEY · SEPANG · BLACKHOLE · SALAMANCA · HAGELAWEL · LUCKYSTRIKE · SPIKYROCK · MYOFIBRIL · INJUNCTION · PSYCHICSALMON · RAGINGBULLFROG · ROBOTICFISH · TIMIDTOAD · AURA · CARBONCOPY · CASK · CHORDAL · CRAN · INSTINCT · MADFORGE · SWAMP · VOLSUNGA · WHITERAVEN · MVR · PPF · JTRIG · MAPREDUCE · MAILORDERPeople: Tukey, John -
Is There a Sustainable Ops Tempo in S2? How Can Analysts Deal with the Flood of Collection? SIDtoday [pdf]
The NSA's Deputy Director for Analysis and Production discusses the unsustainable operational tempo caused by overwhelming data collection volumes, advocating for "Analytic Modernization" that would shift from treating SIGINT as "radioactive" NSA-only material to sharing raw intelligence with other agencies for collaborative exploitation.
Programs: A-SPACE -
June Analytic Showcase Scheduled SIDtoday [pdf]
Staff will present two papers on new techniques. One enables the NSA to spy on traffic from Iraqi internet cafes to identify unique users and "support SIGINT-enabled surveillance and rendition operations," while the other, THERAPYCHEATER, helps analysts pick new targets for surveillance by looking at activity patterns in the draft folders of web accounts.
People: Mercurio, JoeThe Intercept ↗ (2017-09-13) -
LEVITATION and the FFU Hypothesis [pdf]
This document describes LEVITATION, a top-secret Canadian Communications Security Establishment (CSE) project that monitors millions of daily file uploads and downloads from free file-sharing sites to identify potential extremist activity. It details the program’s data collection methods, analytical workflow, use of partner databases, and examples of successful discoveries related to terrorist materials and hostage videos.
-
Location Cookies [pdf]
These slides contain excerpts from an April 2013 National Security Agency presentation detailing signal surveillance techniques and successes. They reveal that the NSA and its British counterpart, Government Communications Headquarters (GCHQ), use a Google-specific tracking cookie to pinpoint targets for hacking.
-
Menwith initiatives maximizing our access [pdf]
11 p.This document details Menwith Hill Station’s initiatives to maximize and optimize SIGINT collection, processing, and exploitation, including large-scale monitoring of internet and communications traffic, metadata analysis, and cross-platform target tracking. It describes programs and techniques for surveying, processing, and exploiting signals, enabling analysts to identify, geolocate, and track targets across multiple networks and devices.
Programs: ASPHALT · BLACKTOP · TARMAC · GTE · INDEX · ROADBED · DARKQUEST · EVIL EYE · Spider · SHAREDVISION · QUEST · NOSEYPARKER · XKEYSCORE · GRANDPIANO · FOGHORN · BASSQUEST · APPARITION · MUTANT BROTH · KARMAPOLICE · SOCIALANIMAL · MEMORY HOLE · TURBULENCE · TELLURIAN · MAILORDER · AUTO ASSOC · SLR · BLACKHOLE · JTRIG · PPF · LECKWITH · SCEPTRE · CLOUDPeople: Alexander, Keith B.“GCHQ taps fibre-optic cables for secret access to world's communications” - The Guardian ↗ (2013-06-21) · “Inside Menwith Hill: The NSA’s British Base at the Heart of U.S. Targeted Killing” - The Intercept ↗ (2016-09-06) · MacAskill, Ewen · Borger, Julian · Hopkins, Nick · Davies, Nick · Ball, James · Gallagher, Ryan -
Metadata-Driven Drone Strikes SIDtoday [pdf]
These documents describe the NSA and CIA’s use of geolocation technology, particularly via UAV-mounted systems like GILGAMESH and VICTORYDANCE, to track mobile devices for targeted drone strikes, including operations that killed U.S. citizen Anwar Nasser Aulaqi. These programs rely heavily on signals intelligence rather than human confirmation, raising concerns about reliability, civilian casualties, and the accuracy of targeting. See also: SIDtoday/documents/2005/2005-01-18_SIDToday_-_New_Tactical_Collection_System_Joins_the_War_on_Terroris