How Is Human Language Technology (HLT) Progressing?
Open PDFDocument date: 2011-09-06
Type: SIDtoday
Pages: 2
Genre: newsletter
Classification: SECRET SI
Dissemination: REL
This document provides an operational update on the deployment of "HLT Labs"—a specialized suite of voice analytics designed to automate the discovery of intelligence "nuggets" within massive volumes of audio collection. The document highlights successful field applications in Afghanistan, Georgia, Texas, and Latin America, noting that these tools are becoming "force multipliers" that allow linguists to bypass manual listening in favor of automated keyword searching and speaker recognition.
HLT, SPIRITFIRE, TRANSX, DISTILLERY, CLOUDBASE, GHOSTMACHINE
People mentioned:
-
Countries:
Afghanistan, France, Spain, Cuba
Tags:
media mining, speech-to-text, keyword search, speaker identification, Special Collection Service, VHF, cloud computing, drug trafficking, Cuban official, voice analytics, linguists
Identifier:
sidtoday-is-hlt-progressing-final
PDF SHA-256:
00e9b1a43f1ef11dca4a6eea47f4453898a37f03e05f3695bce7b14a7c969bbd
My notes
(S//SI//REL) How Is Human Language Technology (HLT) Progressing?
FROM: (U//FOUO) [REDACTED]
Language Analysis Modernization Lead (S2)
Run Date: 09/06/2011
(S//SI//REL) Editor's intro: At the SID town hall meeting of February 2011, [REDACTED] (pictured) briefed on Human Language Technology, i.e., tools that sort through SIGINT voice collection and automatically find the most promising nuggets, thereby saving linguists countless hours. What's happened with HLT since that time?
(S//SI//REL) In 2011 we deployed HLT Labs to Afghanistan, NSA Georgia, Latin American SCS sites, and NSA Texas.
(U) Afghanistan-area targets
(S//SI//REL) Afghan Regional Operating Cryptologic Center (AROCC) analysts started using HLT Labs to track their targets in April, and when the analytics were successfully used to find new information, the mission was expanded to include international teams.* The Afghanistan deployment boasts some technological firsts associated with cloud computing** and includes the full suite of analytics with Pashto speech-to-text (STT). Recently French analysts in the ARC were able to find target speakers on new selectors using speaker recognition.
(S//SI//REL) Our deployment to NSA Georgia enables us to partner with analysts to assess the performance of our newest STT models: Pashto and Farsi. These languages have limited training data which creates challenges for STT, and we have been focused on finding applications that are beneficial even for these low-resource languages. NSA-Georgia traffic includes noisy VHF collections which seriously degrade analytic performance; however, analysts can still find target speaker cuts on unknown frequencies.
(U) Spanish-speaking targets
(S//SI//REL) Spanish is the most mature of our speech-to-text analytics, and has higher keyword-search accuracy than other deployed STT models. We've had great success searching for Spanish keywords at NSA Texas and Latin America SCS sites.
(S//SI//REL) For example, in early August a new NSA Texas user applied keyword search the morning after his training to find a previously unreported cut from a drug trafficking target. Likewise, the OIC of one of the Latin American SCS sites recently reported he was able to find foreign intelligence regarding a Cuban official in a fraction of the usual time. His comment: This same example could be used over and over by many that have to go over countless voice cuts to finally dig that gold nugget that will turn into a report.
(U) Development work continues
(U//FOUO) The R6 research team is working to add new applications, improve keyword search capability, enhance analytics, add new languages, and refine the user interface. Recently the Summer Camp for Applied Language Exploration (SCALE) -- a joint NSA Johns Hopkins University exercise -- investigated new ways to use the results of HLT analytics from existing targets to find new targets. Research is also working closely with the SPIRITFIRE (voice analytics)
and TransX (translation, transcription and transliteration) efforts to ensure HLT Labs capabilities are included in the corporate solution for enterprise deployment in 2012.
(U//FOUO) More information about HLT Labs is available here.
(U//FOUO) See a related SIDtoday article about HLT here.
(U) Notes:
* (S//REL) The international teams were from the Analysis and Research Cell (ARC), Task Force 310, and Combined Joint Special Operations Task Force (CJSOTF).
** (S//SI//REL) Specifically, the Afghan deployment is the first use of DISTILLERY and CLOUDBASE on a GHOSTMACHINE platform.
Published in the Same Report
(10 of 29 documents)
-
For Media Mining, the Future Is Now! SIDtoday [pdf]
A number of new "media mining" tools allow analysts to search intercepted content in near-real time, no matter if text, audio or visual. This includes retroactive word-search in intercepted communications.
People: Picone, Joseph“How the NSA Converts Spoken Words Into Searchable Text” - The Intercept ↗ (2015-05-05) · “Finding Your Voice: Forget About Siri and Alexa — When It Comes to Voice Identification, the “NSA Reigns Supreme”” - The Intercept ↗ (2018-01-19) · The Intercept ↗ (2019-05-29) · Froomkin, Dan · Kofman, Ava -
For Media Mining, the Future Is Now! (conclusion) SIDtoday [pdf]
In the future, thanks to Human Language Technology (HLT), an analyst will be able to sort through millions on intercepted phone calls each day "and focus on only the small percentage that is relevant". He or she will also be able to get precise answers to queries such as "Where is the mayor of Baghdad?"
People: Picone, Joseph“How the NSA Converts Spoken Words Into Searchable Text” - The Intercept ↗ (2015-05-05) · The Intercept ↗ (2019-05-29) · Froomkin, Dan -
“Black Budget” — FY 2013 Congressional Budget Justification/National Intelligence Program, p. 262 (February 2012) [pdf]
This excerpt from the Fiscal Year 2013 Congressional Budget Justification (the "Black Budget") details the NSA's financial and performance metrics for "Analytics Modernization," specifically focusing on the transition to high-volume, automated speech-to-text systems. It records the decommissioning of legacy voice processing systems (VoiceRT) in favor of more robust, scalable capabilities aimed at rendering millions of intercepted voice files searchable via text queries and paired dialogue transcription.
Programs: VoiceRT -
“Black Budget” — FY 2013 Congressional Budget Justification/National Intelligence Program, pp. 360-364 (February 2012) [pdf]
The "Human Language Technology (HLT) Research" section of the FY 2013 "Black Budget" details a specialized research project focused on creating "force multipliers" for the NSA workforce through automated speech-to-text, machine translation, and content extraction. With a requested budget of $29.4 million, the project aims to bridge the gap between foundational academic research (via Johns Hopkins University) and the practical exploitation of the volume and velocity of global SIGINT.
Programs: HLT -
Classification Guide for Human Language Technology (HLT) Models (May 18, 2011) [pdf]
This document is an NSA administrative policy document that establishes the rules for what information regarding speech analytics remains public versus what must be kept secret. It explicitly confirms that the NSA possesses automated tools for gender, language, dialect, and speaker recognition, as well as speech-to-text and phonetic tokenization. While the existence of these capabilities is unclassified, the specific data used to "train" the models and the actual identity of targeted speakers are protected at the SECRET level.
Programs: HLT -
RT10 Overview [pdf]
This RT10 overview document is a high-level briefing on the implementation of a real-time SIGINT architecture designed to modernize intelligence operations in the Iraq theater, specifically Baghdad. The project aims to replace legacy, "manual-intensive" processes with a "SIGINT Brain"—a distributed database and automated analytic framework capable of processing over 50 million GSM metadata events per day.
Programs: RT10 · OBELISK · WISPYKNIT · VICTORYUNIFORM · JUGGERNAUT · COBRAFOCUS · MATTERHORN · EXPIATION · VoiceRT · GREENDRAGON · TAREX · KEYCARD -
Finding Nuggets -- Quickly -- in a Heap of Voice Collection, From Mexico to Afghanistan SIDtoday [pdf]
A 2011 article by an NSA Texas analyst describing the deployment of Human Language Technology analytics — including speech-to-text, speaker identification, gender recognition and language identification — to Special Collection Service sites in the Americas and to the Afghanistan Remote Operations Cryptologic Center, with operational examples ranging from tunnel detection in Tijuana to intelligence support in Kandahar.
Programs: HLT -
Coming Soon! A Tool that Enables Non-Linguists to Analyze Foreign-TV News Programs SIDtoday [pdf]
This document introduces eViTAP (Enhanced Video Text and Audio Processing), a fully automated system designed to monitor and translate international television broadcasts. Primarily aimed at analysts who do not possess specific foreign language skills, the tool leverages automated processing to transform open-source media into actionable intelligence.
-
SIRDCC Speech Technology WG assessment of current STT technology [pdf]
This secret document from the SIRDCC Speech Technology Working Group provides a technical assessment of Speech-to-Text (STT) capabilities for the Security Service (MI5) and GCHQ. The report acknowledges that while automated transcription of intercepted speech remains highly challenging, the technology offers immediate "force multiplier" benefits for specific tasks like identifying spoken telephone numbers or keywords. The Working Group recommends continued investment in bespoke training and a transition toward a common system—specifically the BBN Byblos engine—to facilitate the sharing of classified linguistic models between UK and US agencies.
Programs: BYBLOS -
RT10 Initiative Overview [pdf]
Overview document for RT10, "Real-time, distributed architecture for mobile (tactical) and fixed (national) operational environments"
Programs: RT10 · COBRA FOCUS · GREENDRAGON · GOLDMINER · JUGGERNAUT · MATTERHORN · OBELISK · SORTINGHAT · INFOVIS · SHARPFOCUS2 · TOGA · SORTINGLEAD · GLOBALREACH · VOICERT · QUADRASPECTRE · RENOIR · ANALYSTSNOTEBOOK · OCTAVE · HLT · DRT · GEOT · InnoVisions
See all 10 documents published with “How the NSA Converts Spoken Words Into Searchable Text” →
Related Documents
-
Classification Guide for Human Language Technology (HLT) Models (May 18, 2011) [pdf]
This document is an NSA administrative policy document that establishes the rules for what information regarding speech analytics remains public versus what must be kept secret. It explicitly confirms that the NSA possesses automated tools for gender, language, dialect, and speaker recognition, as well as speech-to-text and phonetic tokenization. While the existence of these capabilities is unclassified, the specific data used to "train" the models and the actual identity of targeted speakers are protected at the SECRET level.
Programs: HLT -
Finding Nuggets -- Quickly -- in a Heap of Voice Collection, From Mexico to Afghanistan SIDtoday [pdf]
A 2011 article by an NSA Texas analyst describing the deployment of Human Language Technology analytics — including speech-to-text, speaker identification, gender recognition and language identification — to Special Collection Service sites in the Americas and to the Afghanistan Remote Operations Cryptologic Center, with operational examples ranging from tunnel detection in Tijuana to intelligence support in Kandahar.
Programs: HLT -
Real-Time Regional Gateway Cloud Analytics for Forward Users [pdf]
This document outlines the NSA’s Real-Time Regional Gateway (RT-RG) program, a global cloud-based signals intelligence system designed to provide near real-time data analysis and targeting support for military and intelligence operations. It details the transition to scalable cloud architectures such as GHOSTMACHINE and Cloudbase, enabling massive data retention, exploitation, advanced geolocation, and automated analytics for tracking high-value targets across multiple theaters, including Afghanistan, Iraq, and the Horn of Africa.
Programs: RT-RG · GOLDMINER · GEOT · AGENTLOGIC · SHARKFINN · SKS · PANOPTICON · DRT · CIDNE · JUGGERNAUT · LOPERS · TALIS · MATTERHORN · AIRHANDLER · LYCANTHROPE · VOICESAIL · RETURNSPRING · SORTINGLEAD · GHOSTMACHINE · GMROYALE · GMREGAL · GMGEIST · GMIDEA · GMTOTE · GMPEN · GMPAC · GMCALM · GMMATE · GMAPPLE · GMCAVE · GMPEACH · GMDEN · GMMOUNT · GMALAMO · GMLIGHT · GMBALTIC · GMPLACE · GMPARK · GMHALO · GMCARBON · DRAGONFIRE · DOMEEXPOSE · GIANTSQUID · MAINWAY · FURIOUSCHAINSAW · SHARKFURY · NIAGARAFILES · SHARKQUERY · SHARKDOCS · OCTAVE · UTT · BEDDOWN · CHEEKYMONKEY · LAYERCAKE · CYBERTRANS · GMSEMINOLE · GMGULF · GMZILLA · HLT · JEMA · NEVIS · SEDB · CLOUDBASE · MAPREDUCE · CASPORT · WAVELEGAL“Mission Creep: How the NSA’s Game-Changing Targeting System Built for Iraq and Afghanistan Ended Up on the Mexico Border” - The Intercept ↗ (2019-05-29) · “Norges hemmelige krig i Afghanistan” - NRK ↗ (2019-05-29) · Moltke, Henrik · Skille, Øyvind Bye · Døvik, Olav · Hansen, Ståle Ellingsgard · Fjørtoft, Martin -
Human-Language Technology in Your Future SIDtoday [pdf]
A look into nine new Human Language Technologies that will likely assist NSA analysts in the near future. These include automated translation, transcription and language-based pattern recognition / identification.
Programs: HLT -
Human-Language Technology -- Everywhere SIDtoday [pdf]
The best Human Language Technology (HLT) functions "behind the desktop" making sure the analyst receives relevant, filtered information. In the near future, analysts will have more and better HLT at their disposal.
People: Kulman, AnitaThe Intercept ↗ (2019-05-29)