Tag: petabyte storage 4 documents
Search documents matching “petabyte storage” →Free-text subjects the curators added. Unlike the other axes these are not necessarily from the documents' own vocabulary, so they are the loosest thing to browse by (and the most forgiving).
Refine (combine with another filter, via search)
Sort: Title
Documents
-
GCHQ Analytic Cloud Challenges [pdf]
The 2012 GCHQ Analytic Cloud document outlines a sophisticated data processing infrastructure handling approximately 50 billion daily events across multiple sites, utilizing QFD databases and HADOOP clusters for various analytic capabilities targeting intelligence gathering and analysis. It describes architectural components, challenges in scaling and enabling analysts to cope with data complexity, and proposes future technology options including consolidation approaches and new cloud architectures to enhance capabilities.
Programs: EVOLVEDMUTANTBROTH · MUTANTBROTH · HARDASSOC · HRMAP · KARMAPOLICE · SOCIALANTHROPOID · AUTOASSOC · GEOFUSION · MVR · TERRAIN · SALAMANCA · BLACKHOLE · SAMUELPEPYS · BROKER · VAIL · GUIDINGLIGHT · FIRSTCONTACT · CLOUDYCOBRA · RUMOURMILL · EVERYASSOC · EVERYPOLICE · EVERYCREATURE · EVERYEAD · EVERYCIPHER · GREYFOX · SILVERFOX · GOLDENAXE · TRIBALCARNEM · PUBLICANEMONE · EPICFAIL · STERLINGMOTH · FOGHORN · LOOKINGGLASS · FIREENGINE · NGGEO · ALPHACENTAURI · CLOUDBASE · ACCUMULO · ASEM · SEM · HAKIM · STREAMING · HBASE · LAUREL · SKINNYCLOUD · HARDY · GCHQCORE -
Is There a Sustainable Ops Tempo in S2? How Can Analysts Deal with the Flood of Collection? SIDtoday [pdf]
The NSA's Deputy Director for Analysis and Production discusses the unsustainable operational tempo caused by overwhelming data collection volumes, advocating for "Analytic Modernization" that would shift from treating SIGINT as "radioactive" NSA-only material to sharing raw intelligence with other agencies for collaborative exploitation.
Programs: A-SPACE -
Operational Engineering November 2010 [pdf]
This internal GCHQ briefing from late 2010 documents the Events Product Centre's suite of bulk internet and communications surveillance tools — including IMMINGLE, SALAMANCA, SOCIAL ANTHROPOID, MUTANT BROTH, KARMA POLICE, SAMUEL PEPYS and GUIDING LIGHT — covering scaling targets, new data sources, and operational capabilities across telephony, webmail, instant messaging, social media, GPRS and VoIP collection.
Programs: EVENTS · IMMINGLE · SALAMANCA · GUIDINGLIGHT · B3M · HARDASSOC · MAINWAY · THUGGEE · FASCIA · HAUSTORIUM · FIRESTORM · TERRAIN · SHAREOWN · ESCHAR · BRIO · INFINITEMONKEY · KARMAPOLICE · MARBLEDGECKO · MEMORYHOLE · MUTANTBROTH · SAMUELPEPYS · SOCIALANIMAL · SOCIALANTHROPOID · HEARTBEAT · SANTHROPOID · LEAKYGATEWAYS · GTPTUNNEL · LUSTRE · GLASSBACK · COLLATERAL · HRMAP · SWORDPLAY · BROADOAK · REFORMER · ARTEMIS · MERLOT · CONTRAOCTAVE · CRYINGFOWL · CULTWEAVE · SAMDYCE · GCQUERY · BLAZZINGSADDLES · EREPO · JTRIG · SOUNDER -
QFD BLACKHOLE Technology Behind INOC [pdf]
A UK SECRET STRAP2 COMINT ORCON GCHQ Applied Research presentation from March 2009 documenting the BLACK HOLE bulk data repository's scale and structure, and the suite of analytical tools — MUTANT BROTH, KARMA POLICE, INFINITE MONKEYS and MARBLED GECKO — built on top of it to answer specific intelligence questions about online presence, browsing activity, forum usage and geolocation searches.
Programs: BLACKHOLE · MESME · NETEZZA · CLOUDS · MUTANTBROTH · KARMAPOLICE · INFINITEMONKEYS · MARBLEDGECKO · GOOB · AUTOASSOC · PSOUP · SOCIALANIMAL · SAMUELPEPYS · MEMORYHOLE · HRMAP · PSOUPALERT · LAUGHINGHYENA