Digint Imbalance
Open PDFDocument date: 2010-03-01
Pages: 2
Genre: report
Banner: SECRET
Classification: SECRET
Dissemination: -
This secret MI5 report highlights a growing imbalance between the massive scale of digital intelligence (DIGINT) collection and the agency’s limited capacity to analyze it, leaving much data unassessed and creating risks of intelligence failure. It identifies major challenges in staffing, tools, and infrastructure, and calls for improved analysis, better integration with other intelligence, and new capabilities to disrupt online threats.
-
People mentioned:
-
Countries:
-
Tags:
DIGINT, digital intelligence, intelligence failure, failure, data collection, data exploitation, analysis, raw data, unselected collection, shortages, challenges, intelligence gaps, agent targeting, data processing, broadband monitoring, metadata
Identifier:
Digint-Imbalance
PDF SHA-256:
eb62ef4c6a336dd99cc637cedeb9831c2a00b2b9fe0389f3ee5cfff0a9a46ab4
My notes
SECRET
Figure 2 - Approximate coverage of targets (July 2009)
Volume
500
400
300
200
100
0
P1/P2H
P2M
P3
Priority of Investigation
No. targets
Email accounts
Broadband intercepts
Remote access
2010/11 investigative challenges
In addition to the strategic DSO coverage requirements, the investigative branches of the Service have identified a set of specific business challenges for 2010/11:
- Increasing assurance levels from 50% towards 70% for high priority investigations
- Addressing new cross border terrorist planning and commissioning threats
- Reducing the proportion of new casework with no pre-existing intelligence to below 30%
- Launching coverage of P4 targets, and providing minimum acceptable levels of coverage for non-Islamist investigations
- Addressing current workload backlogs
- Mitigating the relative lack of experience of investigative staff
- Reducing the risk of intelligence black-outs due to infrastructure failure
Capability requirements
In addition to target coverage requirements, a number of issues have been identified that demand improvements in the Service's existing DIGINT capabilities:
- There is an imbalance between collection and exploitation capabilities, resulting in a failure to make effective use of some of the intelligence collected today. With the exception of the highest priority investigations, a lack of staff and tools means that investigators are presented with raw and unfiltered DIGINT data. Frequently, this material is not fully assessed because of the significant time required to review it, particularly when pre-filtered intelligence from other sources (such as summarised transcriptions of voice interceptions) is likely to yield more value for less effort.
- The Service's processes for requesting DIGINT coverage and managing its collection are fragmented and can add administrative burden that reduces the time available for investigators to
SECRET
SECRET
- assess intelligence. Furthermore, in some circumstances the processes do not help to maintain appropriate levels of coverage of individual targets over time.
- The capacity of the Service to maintain and develop new collection capabilities is limited: working at its maximum capacity it cannot meet all of the requests placed upon it.
- Across teams and amongst individual investigators, there is an inconsistent understanding of the capabilities currently available for the collection and exploitation of DIGINT, and there is no mechanism for sharing knowledge of evolving online behaviours that might indicate a threat or provide a disruption opportunity.
- Mechanisms for agent targeting and verification are small in scale and poorly resourced.
- The Service does not have adequate capabilities to disrupt on-line operational activities.
- The provision of business continuity and disaster recovery is inadequate for the operational systems used to receive and technically process incoming DIGINT.
To address these issues and to deliver increased assurance (in addition to significantly increasing the number of targets with DIGINT coverage), the business requirements of the Service are therefore to:
- ensure that all of the intelligence already being collected is analysed or assessed to at least an acceptable minimum level;
- exploit collected intelligence more completely and effectively, and integrate it with intelligence from other sources to derive additional value;
- develop collection mechanisms to capture intelligence that is not available from fixed and mobile broadband intercept or from the collection of customer records data, to prevent a degradation of current collection penetration;
- secure current operational IT systems (as opposed to corporate systems) by providing improved infrastructure for scalability and resilience;
- improve support for the targeting of agents and verification of agent reporting, and build new capabilities to allow disruption of on-line operational activities;
- build and maintain a workforce that understands and can keep up with the emerging threats and opportunities of the internet age.
SECRET
Published in the Same Report
(8 documents)
-
Communications Capability Development Programme [pdf]
The Communications Capability Development (CCD) Programme (formerly the Intercept Modernisation Programme, IMP) was a Home Office–led effort to maintain the U.K.’s lawful interception and communications data capabilities amid changing technologies, emphasizing that law enforcement faced a “significant decline” without new measures. GCHQ supported CCD with technical expertise and interim services like MILKWHITE Enrichment, while stressing that CCD was politically contentious, technically complex, and required legislation to mandate data retention by U.K. network operators.
-
Digint Narrative [pdf]
This Digint Programme (2010 draft) outlines MI5’s plan to strengthen its ability to collect and exploit digital intelligence, addressing the growing challenge of massive internet-based data and the risk of missing critical insights. It emphasizes collaboration with GCHQ, NTAC, and other agencies to build strategic capabilities, improve exploitation of collected data, and expand coverage for counterterrorism and national security.
Marking: UK SECRETPrograms: IMP -
MILKWHITE Enrichment Service (MES) Programme [pdf]
This March 2011 GCHQ document details the MILKWHITE program, which is enhancing domestic interception and data analysis capabilities. It outlines expanded funding for advanced analytics, bulk data storage, and service scaling to meet growing demand from multiple law enforcement and intelligence agencies, including SOCA, HMRC, MI5, and SIS, thereby broadening access to large volumes of internet-derived metadata for operational use.
-
Mobile Apps Checkpoint Meeting Archives [pdf]
This GCHQ document tracks project checkpoints from December 2009 to July 2010, focusing on the rollout and operationalization of mobile apps, TERRAIN system upgrades, GTP/GRX access, and converged event processing. Key themes include coordinating multi-team dependencies, addressing technical gaps in handset and app data characterization, and ensuring timely delivery of milestones like MIP-1.
Programs: TERRAIN · PRESTON · XKEYSCORE · BEGAL · TAMING PASTRIES · MILKWHITE · MVR · SOCIAL ANTHROPOID · INTEGER SPIN · MUTANT BROTH · GEO FUSION · MARBLED GEKO · BLACKHOLE · ACTOR ACTION · TRAFFICMASTER · HARD ASSOC · MIRTF · HEARTBEAT · SILVER LINING · GREENHEART · EREPO · MAGLITE · POKERFACE · BOSTROM · MIDDLESEX GREEN · ENCHANTERSS · HAUSTORIUM · SALAMANCA · BROADOAK · MONKEY PUZZLE · TACHO · EREPO GAMMA · NINJA · PEBBLE BED · GOLDEN EYE · STRAPROC · STARPROC · LOOKING GLASS · SORCERER · SOCIAL ANIMAL · GTE · NGE · PPF · IIB · UDAQ · MES · TERRAINS · CCDF · SINO · GDR · OSDS · LECKWITH -
NTAC Overview [pdf]
This document outlines the structure, mission, and operations of the U.K.’s National Technical Assistance Centre (NTAC), a GCHQ-controlled unit responsible for enabling and managing lawful interception of communications and decryption of data across national networks. It details NTAC’s integration into GCHQ, its key role in programs like MILKWHITE, its extensive interception capacity, and the technical and resource challenges it faces in adapting to evolving telecommunications technologies.
Marking: SECRET -
PRESTON Business Processes [pdf]
The PRESTON Business Processes document outlines GCHQ’s internal workflows for authorizing, enabling, managing, and terminating communications interception, as well as producing intelligence from intercepted data. It defines the roles, responsibilities, and technical processes that support lawful interception operations within the PRESTON system.
-
PRESTON Architecture [pdf]
The PRESTON architecture, centered on GCHQ’s PRESTON Processing system, is detailed across multiple views showing how the agency intercepts, processes, and stores communications. It outlines a sophisticated surveillance framework that integrates lawful interception, data processing, and analyst access, emphasizing covert monitoring and management of intercepted data.
Programs: PRESTON · FAST GROK · BOXSTER · MARMION · GENTIAN · TERRAIN · BLACKNIGHT · CORINTH · FARNDALE · SAMDYCE · B3M · VSLT · SAMBOK · HAUSTORIUM · FUNFAIR · Cheltenham · SALAMANCA · IIB · LOCHNVAR · MONACO · HOTLINE · Benhall · GDR · CE · BRIGHTON · ARSENIDE · SPQR · TACHO · TRAFFIC MASTER · KEYCARD · XKEYSCORE · COURIERSKILL · MONKEYPUZZLE · PILBEAM · GREENHEART · CCDF · GTE · CADENCE · TERRAINS · GCNET · HENCE -
PRESTON Study [pdf]
This GCHQ document details the PRESTON surveillance program, revealing that over a six-month period, 97% of intercepted communications—including calls, texts, and internet data—were not viewed by analysts despite being fully warranted, highlighting systemic challenges in processing and exploiting the vast volume of collected intelligence. It underscores the reliance on traditional PRESTON sources for events like the London G20, while noting gaps in data feeds from other repositories, reflecting both operational scale and limitations in effective oversight.
Related Documents
-
Digint Narrative [pdf]
This Digint Programme (2010 draft) outlines MI5’s plan to strengthen its ability to collect and exploit digital intelligence, addressing the growing challenge of massive internet-based data and the risk of missing critical insights. It emphasizes collaboration with GCHQ, NTAC, and other agencies to build strategic capabilities, improve exploitation of collected data, and expand coverage for counterterrorism and national security.
Marking: UK SECRETPrograms: IMP -
PRESTON Study [pdf]
This GCHQ document details the PRESTON surveillance program, revealing that over a six-month period, 97% of intercepted communications—including calls, texts, and internet data—were not viewed by analysts despite being fully warranted, highlighting systemic challenges in processing and exploiting the vast volume of collected intelligence. It underscores the reliance on traditional PRESTON sources for events like the London G20, while noting gaps in data feeds from other repositories, reflecting both operational scale and limitations in effective oversight.
-
SSO dictionary relevant entries [pdf]
This document provides a detailed list of relevant entries related to NSA's Special Source Operations (SSO), highlighting various NSA surveillance and data collection systems used to monitor global communications. It includes descriptions of programs such as BIGBIRD and DISHFIRE, which focus on cable and SMS interception, as well as tools like LOPERS and COURIERSKILL for processing telephony metadata and refining target selection. The entries emphasize the extensive and automated nature of the NSA's intelligence-gathering operations, particularly in relation to undersea cables and telecommunication networks, and underscore the secretive and expansive scope of SSO's global surveillance activities.
Programs: ARTIFICE · BIGBIRD · FAIRVIEW · SSO · POORWILL · PINECONE · BLACKBELT · CADENCE · BLACKNIGHT · SHIPMASTER · COURIERSKILL · THEORYMASTER · WEALTHYCLUSTER · CLIFFSIDE · DISHFIRE · MARINA · MAINWAY · NUCLEON · PINWALE · FASCIA · BANYAN · LAMPSHADE · ASSOCIATION · HOMEBASE · SEDB · FRIAR · KEYCARD · TURMOIL · TURBULENCE · UTT · IRISHBEAUTY · OCTAVE · KOZYKOVE · LITHIUM · BLARNEY · LOPERS · MAILORDER · LUMBERYARD · MERLIN · NODDY · SAGUARO · NUTHATCH · CONTRAOCTAVE · XKEYSCORE · PLANK · RODEOSTAR · SAGURA · SERENADE · STARGATE · SILVERCOLLAM · SEALION · SEAGULL · SLIVER · SORA-2 · TITANPOINTE · TUBE · VPCS · STONEGATE“NSA Spying Relies on AT&T’s ‘Extreme Willingness to Help’” - ProPublica ↗ (2015-08-15) · “AT&T Helped U.S. Spy on Internet on a Vast Scale” - The New York Times ↗ (2015-08-15) · “The Wiretap Rooms: The NSA’s Hidden Spy Hubs in Eight U.S. Cities” - The Intercept ↗ (2018-06-25) · Angwin, Julia · Larson, Jeff · Savage, Charlie · Moltke, Henrik · Poitras, Laura · Risen, James · Gallagher, Ryan -
Access: Vision 2013 [pdf]
This July 2010 GCHQ strategic document outlines the agency's vision for 2013 to create "the world's biggest SIGINT engine" with access to 800 10-gigabit connections (up from existing capacity), expanding overseas operations to 60% while reducing UK-based to 40%, and developing new capabilities including computer network exploitation (CNE), exfiltration, beaconry, geolocation, and collection of "Internet Pocket Litter" like cookies and browsing data. The strategy emphasizes "scale and pace," "balance," and "agility" while claiming to operate within legal frameworks of "necessity, proportionality and authorisation," and highlights partnerships with second and third parties to access "intractable and hard targets."
Marking: SECRET -
NTAC Overview [pdf]
This document outlines the structure, mission, and operations of the U.K.’s National Technical Assistance Centre (NTAC), a GCHQ-controlled unit responsible for enabling and managing lawful interception of communications and decryption of data across national networks. It details NTAC’s integration into GCHQ, its key role in programs like MILKWHITE, its extensive interception capacity, and the technical and resource challenges it faces in adapting to evolving telecommunications technologies.
Marking: SECRET