Tag: data reduction 5 documents
Search documents matching “data reduction” →Free-text subjects the curators added. Unlike the other axes these are not necessarily from the documents' own vocabulary, so they are the loosest thing to browse by (and the most forgiving).
Graphs
By Banner
By Classification
Refine (combine with another filter, via search)
Sort: Title
Documents
-
CSEC ITS/N2E Cyber Threat Discovery [pdf]
A CSEC IT Security presentation to the DISCOCON 2010 conference, given with speakers' notes, on how the agency's cyber threat discovery team hunts for intrusions in the traffic it collects from Government of Canada networks. It describes the PonyExpress email processing system, which the notes record as handling 400,000 emails a day, the 8Ball attachment scanner and the Stripsearch binary analyser, and reports experiments in reducing the volume of metadata analysts see — including a machine learning model that discards 85% of email before deep scanning.
Marking: TOP SECRETPrograms: PONYEXPRESS · 8BALL · STRIPSEARCH · POPQUIZ · SLIPSTREAM · FINDMASK · SAWUNEH · TROPPUSNU · N2E · DISCOCON -
HIMR Data Mining Research Problem Book [pdf]
This document is a research roadmap created by GCHQ for the Heilbronn Institute for Mathematical Research (HIMR). Its purpose is to guide long-term mathematical and data-mining research that could improve signals intelligence (SIGINT) capabilities. It outlines key research problems, datasets, and collaboration plans between intelligence agencies and academic researchers. The main goal is to develop new algorithms and mathematical tools to analyze extremely large communications datasets produced by modern surveillance systems.
Programs: CLOUD · DISTILLERY · DISCOVER · PRESTON · LECKWITH · TERRAIN · BROADOAK · NGE · FIVEALIVE · CHARTBREAKER · FIRSTCONTACT · GEOFUSION · RADONSHARPEN-B · PRIMETIME · HIDDENOTTER · BAKERSDOZEN · CLASP · SALTYOTTER · BIRCH · SOLIDINK · FLUIDINK · MAMBA · GRINNINGROACH · PIRATECAREBEAR · AUTOASSOC · CROUCHINGSQUIRREL · HRMAP · KACHINA · SQUEAL · QUESTA · METEORSHOWER · SUNSTORM · GOLDMINE · HAGERAWEL · WOODY · BUZZ · SILVERLIBRARY · SILVERLINING · VALHALLA · MOUNTMCKINLEY · SEPANG · BLACKHOLE · SALAMANCA · HAGELAWEL · LUCKYSTRIKE · SPIKYROCK · MYOFIBRIL · INJUNCTION · PSYCHICSALMON · RAGINGBULLFROG · ROBOTICFISH · TIMIDTOAD · AURA · CARBONCOPY · CASK · CHORDAL · CRAN · INSTINCT · MADFORGE · SWAMP · VOLSUNGA · WHITERAVEN · MVR · PPF · JTRIG · MAPREDUCE · MAILORDERPeople: Tukey, John -
SSO Collection Optimization [pdf]
This is another presentation on problems with the National Security Agency's overcollection of address books and buddy lists and its efforts to weed out useless content. See also EFF_-_20131104-wapo-optimization_google
-
Too Much of a Good Thing? SIDtoday [pdf]
NSA's UK Menwith Hill Station finds itself "bursting at the seams" with unencrypted internet data collected from commercial satellite networks. A new initiative aims to "thin out the data we store at site".
“Inside Menwith Hill: The NSA’s British Base at the Heart of U.S. Targeted Killing” - The Intercept ↗ (2016-09-06) · The Intercept ↗ (2019-05-29) · Gallagher, Ryan -
XKEYSCORE Helper Notes [pdf]
This GCHQ document describes new features in the XKEYSCORE Palantir helper tool, including summary/histogram import capabilities that reduce large datasets by aggregating connection data rather than creating individual events for each entry, and data sourcing features that track which XKEYSCORE queries imported specific data into Palantir. The document provides step-by-step screenshots showing how analysts can histogram network traffic by source IP, destination IP, and ports, then import aggregated results showing metadata like total session sizes and connection counts.
Programs: XKEYSCORE