Tag: metadata processing 7 documents
Search documents matching “metadata processing” →Free-text subjects the curators added. Unlike the other axes these are not necessarily from the documents' own vocabulary, so they are the loosest thing to browse by (and the most forgiving).
Graphs
By Banner
By Classification
Refine (combine with another filter, via search)
Sort: Title
Documents
-
GCHQ Hopscotch [pdf]
An internal note argues for consolidating “edges”—summarized contact pairs—into a single cloud to unify relationship graphs, streamline analytics, and simplify management of HOPSCOTCH. It’s a mundane-sounding data-migration pitch with sharp operational teeth: centralize the social graph so surveillance queries run faster and wider.
Programs: HOPSCOTCH“Operation Socialist: The Inside Story of How British Spies Hacked Belgium’s Largest Telco” - The Intercept ↗ (2014-12-13) · “Lees hier hoe de Britse geheime dienst GCHQ Belgacom aanviel” - NRC ↗ (2014-12-13) · “Britse geheime dienst bespioneerde jarenlang Belgacom-klanten” - De Standaard ↗ (2014-12-13) · Gallagher, Ryan · Modderkolk, Huib · Vanhecke, Nikolas · Eeckhaut, Mark -
Graph theory in the operational environment – GCHQ [pdf]
This document details GCHQ’s research on identifying “closed loops” of communication, where targets use a small set of phones to communicate exclusively among themselves, often with cheap, disposable handsets like the Nokia 1100. It explains the methodology for analyzing anonymized metadata, constructing contact graphs, and extracting non-giant, non-tree components to discover operationally relevant closed loops for intelligence and counterterrorism purposes.
Programs: VOICESAILPeople: Rumsfeld, Donald -
HIMR Data Mining Research Problem Book [pdf]
This document is a research roadmap created by GCHQ for the Heilbronn Institute for Mathematical Research (HIMR). Its purpose is to guide long-term mathematical and data-mining research that could improve signals intelligence (SIGINT) capabilities. It outlines key research problems, datasets, and collaboration plans between intelligence agencies and academic researchers. The main goal is to develop new algorithms and mathematical tools to analyze extremely large communications datasets produced by modern surveillance systems.
Programs: CLOUD · DISTILLERY · DISCOVER · PRESTON · LECKWITH · TERRAIN · BROADOAK · NGE · FIVEALIVE · CHARTBREAKER · FIRSTCONTACT · GEOFUSION · RADONSHARPEN-B · PRIMETIME · HIDDENOTTER · BAKERSDOZEN · CLASP · SALTYOTTER · BIRCH · SOLIDINK · FLUIDINK · MAMBA · GRINNINGROACH · PIRATECAREBEAR · AUTOASSOC · CROUCHINGSQUIRREL · HRMAP · KACHINA · SQUEAL · QUESTA · METEORSHOWER · SUNSTORM · GOLDMINE · HAGERAWEL · WOODY · BUZZ · SILVERLIBRARY · SILVERLINING · VALHALLA · MOUNTMCKINLEY · SEPANG · BLACKHOLE · SALAMANCA · HAGELAWEL · LUCKYSTRIKE · SPIKYROCK · MYOFIBRIL · INJUNCTION · PSYCHICSALMON · RAGINGBULLFROG · ROBOTICFISH · TIMIDTOAD · AURA · CARBONCOPY · CASK · CHORDAL · CRAN · INSTINCT · MADFORGE · SWAMP · VOLSUNGA · WHITERAVEN · MVR · PPF · JTRIG · MAPREDUCE · MAILORDERPeople: Tukey, John -
MILKWHITE Enrichment Service (MES) Programme [pdf]
This March 2011 GCHQ document details the MILKWHITE program, which is enhancing domestic interception and data analysis capabilities. It outlines expanded funding for advanced analytics, bulk data storage, and service scaling to meet growing demand from multiple law enforcement and intelligence agencies, including SOCA, HMRC, MI5, and SIS, thereby broadening access to large volumes of internet-derived metadata for operational use.
-
Sensitive Metadata Analytic Collaboration (SMAC) - Concept of Operations, October 2006 [pdf]
This document outlines the Sensitive Metadata Analytic Collaboration (SMAC) initiative, a system developed by the Five Eyes intelligence agencies to facilitate the sharing and analysis of sensitive metadata to support SIGINT (signals intelligence) missions. While the concept aims to improve collaboration and data sharing, its success is hindered by ongoing policy, legal, and technical barriers, as well as concerns about trust and participation among the agencies involved. The document suggests a complex, somewhat opaque process of metadata sharing that risks undermining privacy protections.
Programs: SMAC -
SKIDROWE [pdf]
This document reveals the extensive surveillance capabilities of the NSA's SKIDROWE program at TITANPOINTE, specifically designed to intercept and analyze satellite communications and internet data. It highlights how intercepted data is processed and filtered through advanced systems like XKEYSCORE, which enables real-time intelligence gathering by sifting through massive amounts of digital content, including emails, calls, and internet activity.
Programs: SKIDROWE · WEALTHYCLUSTER2 · WEALTHYCLUSTER · TURNWEALTHY · VPCS · XKEYSCORE · GENESIS · CONTROLAND · ATM · FITBOY · DARKTHUNDER · FIP · METTLESOME · SKYFRAME · PVOICE · XXPY0012 · AGITATOR · STARPROC · SPELLBINDER · BLACKPEARL · BLACKALCHEMY · BLACKGOLD · BLACKFIRE · MONKEYMIND · BCD · EVVIPER · QUANTUM · MISTYPEAK · VALIANTSURF · GALLANTWAVE · SWINE · BLUESNORT · TOSSSWUNG · GNOMEVISION · GNOMEGOGGLES · HIDDENSALAMANDER · TURMOIL · PSC -
SSO GHOSTMACHINE Analytics [pdf]
This document describes the NSA’s cloud analytics platform GHOSTMACHINE, used by Special Source Operations (SSO) to process and analyze massive volumes of data (e.g., from FASCIA) using Hadoop-based infrastructure. It’s essentially the backbone system for querying, correlating, and enriching surveillance data at scale — not just a single analytic, but the analytic/cloud environment powering many programs.
Programs: GHOSTMACHINE · FASCIA · CERF · POPTOP · DRT · SIGDASYS · JUGGERNAUT · SEADIVER · AST128 · TUSKATTIRE · ASDF · BROKENRECORD · REDHAWK · SSO