Tag: HDFS 4 documents
Search documents matching “HDFS” →Free-text subjects the curators added. Unlike the other axes these are not necessarily from the documents' own vocabulary, so they are the loosest thing to browse by (and the most forgiving).
Refine (combine with another filter, via search)
Sort: Title
Documents
-
BOUNDLESSINFORMANT - Describing Mission Capabilities from Metadata Records [pdf]
This presentation on the Boundless Informant program (BOUNDLESSINFORMANT), which uses big data tools to analyze and visualize global SIGINT metadata collection in near real time. It explains how the system replaced older manual methods, enabling the NSA to track collection volumes, coverage, and trends for specific countries or sites through cloud-based infrastructure.
People: Patel, Vimesh“NSA-files repeatedly show collection of data «against countries» - not «from»” - Dagbladet ↗ (2013-11-22) · “Boundless Informant: the NSA's secret tool to track global surveillance data” - The Guardian ↗ (2013-06-08) · “Claves y giros del espionaje masivo en España” - El Mundo ↗ (2013-11-04) · Greenwald, Glenn · MacAskill, Ewen · Aranda, Germán -
BOUNDLESSINFORMANT – Frequently Asked Questions [pdf]
This document outlines the NSA’s internal metadata analysis tool, designed to provide real-time visualizations and statistics on global SIGINT collection by country, organization, and site. It explains the tool’s purpose, data sources, technical architecture, views, data limitations, and the process for requesting feature enhancements, highlighting its role in cataloging surveillance activity without human intervention.
Programs: BOUNDLESSINFORMANT · ASDF · GAO · RAMPART-A · SPINNERET · GM-PLACE · FALLOUT · TUSKATTIRE · FASCIA · MUSCULAR · Terrestrial RF · MachineShop · TurkeyTower · CloudBase · GXT · ExtGWT · JUGGERNAUT · LOPER · FLAWMILL · SSO · MAPREDUCE“Boundless Informant: the NSA's secret tool to track global surveillance data” - The Guardian ↗ (2013-06-08) · “NSA-files repeatedly show collection of data «against countries» - not «from»” - Dagbladet ↗ (2013-11-22) · “New NSA Revelations: Inside Snowden's Germany File” - Der Spiegel ↗ (2014-06-18) · Greenwald, Glenn · MacAskill, Ewen · Becker, Sven · Gude, Hubert · Horchert, Judith · Müller-Maguhn, Andy · Poitras, Laura · Reißmann, Ole · Rosenbach, Marcel · Schindler, Jörg · Schmid, Fidelius · Sontheimer, Michael · Stark, Holger -
GCHQ Analytic Cloud Challenges [pdf]
The 2012 GCHQ Analytic Cloud document outlines a sophisticated data processing infrastructure handling approximately 50 billion daily events across multiple sites, utilizing QFD databases and HADOOP clusters for various analytic capabilities targeting intelligence gathering and analysis. It describes architectural components, challenges in scaling and enabling analysts to cope with data complexity, and proposes future technology options including consolidation approaches and new cloud architectures to enhance capabilities.
Programs: EVOLVEDMUTANTBROTH · MUTANTBROTH · HARDASSOC · HRMAP · KARMAPOLICE · SOCIALANTHROPOID · AUTOASSOC · GEOFUSION · MVR · TERRAIN · SALAMANCA · BLACKHOLE · SAMUELPEPYS · BROKER · VAIL · GUIDINGLIGHT · FIRSTCONTACT · CLOUDYCOBRA · RUMOURMILL · EVERYASSOC · EVERYPOLICE · EVERYCREATURE · EVERYEAD · EVERYCIPHER · GREYFOX · SILVERFOX · GOLDENAXE · TRIBALCARNEM · PUBLICANEMONE · EPICFAIL · STERLINGMOTH · FOGHORN · LOOKINGGLASS · FIREENGINE · NGGEO · ALPHACENTAURI · CLOUDBASE · ACCUMULO · ASEM · SEM · HAKIM · STREAMING · HBASE · LAUREL · SKINNYCLOUD · HARDY · GCHQCORE -
SSO GHOSTMACHINE Analytics [pdf]
This document describes the NSA’s cloud analytics platform GHOSTMACHINE, used by Special Source Operations (SSO) to process and analyze massive volumes of data (e.g., from FASCIA) using Hadoop-based infrastructure. It’s essentially the backbone system for querying, correlating, and enriching surveillance data at scale — not just a single analytic, but the analytic/cloud environment powering many programs.
Programs: GHOSTMACHINE · FASCIA · CERF · POPTOP · DRT · SIGDASYS · JUGGERNAUT · SEADIVER · AST128 · TUSKATTIRE · ASDF · BROKENRECORD · REDHAWK · SSO