Publications

This work investigates methods for improving a pronunciation proficiency recognition system, both in terms of phonetic level posterior probability calibration, and in ordinal utterance level classification, for Modern Standard Arabic (MSA), Spanish and Russian. To support this work, utterance level labels were obtained by crowd-sourcing the annotation of language learners' recordings. Phonetic posterior probability estimates extracted using automatic speech recognition systems trained in each language were estimated using a beta calibration approach [1] and language proficiency level was estimated using an ordinal regression [2]. Fusion with language recognition (LR) scores from an i-vector system [3] trained on 23 languages is also explored. Initial results were promising for all three languages and it was demonstrated that the calibrated posteriors were effective for predicting pronunciation proficiency. Significant relative gains of 16% mean absolute error for the ordinal regression and 17% normalized cross entropy for the binary beta regression were achieved on MSA through fusion with LR scores.

Machine learning methods in Human Language Technology have reached a stage of maturity where widespread use is both possible and desirable. The MIT Lincoln Laboratory LLTools software suite provides a step towards this goal by providing a set of easily accessible frameworks for incorporating speech, text, and entity resolution components into larger applications. For the speech processing component, the pySLGR (Speaker, Language, Gender Recognition) tool provides signal processing, standard feature analysis, speech utterance embedding, and machine learning modeling methods in Python. The text processing component in LLTools extracts semantically meaningful insights from unstructured data via entity extraction, topic modeling, and document classification. The entity resolution component in LLTools provides approximate string matching, author recognition and graph-based methods for identifying and linking different instances of the same real-world entity. We show through two applications that LLTools can be used to rapidly create and train research prototypes for human language processing.

One of the greatest challenges for an adult learning a new language is gaining the ability to distinguish and produce foreign sounds. The US Government trains 3,600 enlisted soldiers a year at the Defense Language Institute Foreign Language Center (DLIFLC) in languages critical to national security, most of which are not widely studied in the U.S. Many students struggle to attain speaking fluency and proper pronunciation. Teaching pronunciation is a time-intensive task for teachers that requires them to give individual feedback to students during classroom hours. This limits the time teachers can spend imparting other information, and students may feel embarrassed or inhibited when they practice with their classmates. Given the demand for students educated in foreign languages and the limited number of qualified teachers in languages of interest, there is a growing need for computer-based tools students can use to practice and receive feedback at their own pace and schedule. Most existing tools are limited to listening to pre-recorded audio with limited or nonexistent support for pronunciation feedback. MIT Lincoln Laboratory has developed a new tool, Net Pronunciation Feedback (NetProF), to address these challenges and improve student pronunciation and general language fluency.