Publications
Estimating and evaluating confidence for forensic speaker recognition
January 1, 2005
Conference Paper
Published in:
Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, ICASSP, Vol. 1, 19-23 March 2005, pp. I-717 - I-720.
Topic:
R&D area:
Summary
Estimating and evaluating confidence has become a key aspect of the speaker recognition problem because of the increased use of this technology in forensic applications. We discuss evaluation measures for speaker recognition and some of their properties. We then propose a framework for confidence estimation based upon scores and metainformation, such as utterance duration, channel type, and SNR. The framework uses regression techniques with multilayer perceptrons to estimate confidence with a data-driven methodology. As an application, we show the use of the framework in a speaker comparison task drawn from the NIST 2000 evaluation. A relative comparison of different types of meta-information is given. We demonstrate that the new framework can give substantial improvements over standard distribution methods of estimating confidence.
Summary
Estimating and evaluating confidence has become a key aspect of the speaker recognition problem because of the increased use of this technology in forensic applications. We discuss evaluation measures for speaker recognition and some of their properties. We then propose a framework for confidence estimation based upon scores and metainformation...
READ MORE
New measures of effectiveness for human language technology
January 1, 2005
Journal Article
Published in:
Lincoln Laboratory Journal, Vol. 15, No. 2, 2005, pp. 341-345.
Topic:
R&D area:
Summary
The field of human language technology (HLT) encompasses algorithms and applications dedicated to processing human speech and written communication. We focus on two types of HLT systems: (1) machine translation systems, which convert text and speech files from one human language to another, and (2) speech-to-text (STT) systems, which produce text transcripts when given audio files of human speech as input. Although both processes are subject to machine errors and can produce varying levels of garbling in their output, HLT systems are improving at a remarkable pace, according to system-internal measures of performance. To learn how these system-internal measurements correlate with improved capabilities for accomplishing real-world language-understanding tasks, we have embarked on a collaborative, interdisciplinary project involving Lincoln Laboratory, the MIT Department of Brain and Cognitive Sciences, and the Defense Language Institute Foreign Language Center to develop new techniques to scientifically measure the effectiveness of these technologies when they are used by human subjects.
Summary
The field of human language technology (HLT) encompasses algorithms and applications dedicated to processing human speech and written communication. We focus on two types of HLT systems: (1) machine translation systems, which convert text and speech files from one human language to another, and (2) speech-to-text (STT) systems, which produce...
READ MORE
Parallel MATLAB for extreme virtual memory
January 1, 2005
Conference Paper
Published in:
Proc. of the HPCMP Users Group Conf., 27-30 June 2005, pp. 381-387.
Topic:
R&D group:
Summary
Many DoD applications have extreme memory requirements, often with data sets larger than memory on a single computer. Such data sets can be addressed with out-of-core methods, which use memory as a "window" to view a section of the data stored on disk at a time. The Parallel Matlab for eXtreme Virtual Memory (pMatlab XVM) library adds out-of-core extensions to the Parallel Matlab (pMatlab) library. The DARPA High Productivity Computing Systems' HPC challenge FFT benchmark has been implemented in C+MPI, pMatlab, pMatlab hand coded for out-of-core and pMatlab XVM. We found that 1) the performance of the C+MPI and pMatlab versions were comparable; 2) the out-of-core versions deliver 80% of the performance of the in-core versions; 3) the out-of-core versions were able to perform a 1 TB (64 billion point) FFT; and 4) the pMatlab XVM program was smaller, easier to implement and verify, and more efficient than its hand coded equivalent. We plan to apply pMatlab XVM to the full HPC challenge benchmark suite. Using next generation hardware, problems sizes a factor of 100 to 1000 times larger should be feasible. We are also transitioning this technology to several DoD signal processing applications. Finally, the flexibility of pMatlab XVM allows hardware designers to experiment with FFT parameters in software before designing hardware for a real-time, ultra-long FFT.
Summary
Many DoD applications have extreme memory requirements, often with data sets larger than memory on a single computer. Such data sets can be addressed with out-of-core methods, which use memory as a "window" to view a section of the data stored on disk at a time. The Parallel Matlab for...
READ MORE
Technology requirements for supporting on-demand interactive grid computing
January 1, 2005
Conference Paper
Published in:
Proc. of the HPCMP Users Group Conf., 27-30 June 2005, pp. 320-327.
Topic:
R&D group:
Summary
It is increasingly being recognized that a large pool of High Performance Computing (HPC) users requires interactive, on-demand access to HPC resources. How to provide these resources is a significant technical challenge that can be addressed from two directions. The first approach is to adapt existing batch queue based HPC systems to make them more interactive. The second approach is to start with existing interactive desktop environments (e.g., MATLAB) and design a system from the ground up that allows interactive parallel computing. The Lincoln Laboratory Grid (LLGrid) project has taken the latter approach. The LLGrid system has been operational for over a year with a few hundred processors and roughly 70 users, having run over 13,000 interactive jobs and consumed approximately 10,000 processor days of computation. This paper compares the on-demand and interactive computing features of four prominent batch queuing systems: openPBS, Sun GridEngine, Condor, and LSF. It goes on to briefly describe the LLGrid system, and how interactive, on-demand computing was achieved on it by binding to a resource management system. Finally, usage characteristics of the LLGrid system are discussed.
Summary
It is increasingly being recognized that a large pool of High Performance Computing (HPC) users requires interactive, on-demand access to HPC resources. How to provide these resources is a significant technical challenge that can be addressed from two directions. The first approach is to adapt existing batch queue based HPC...
READ MORE
Parallel VSIPL++: an open standard software library for high-performance parallel signal processing
January 1, 2005
Journal Article
Published in:
Proc. IEEE, Vol. 93, No. 2 , February 2005, pp. 313-330.
Topic:
R&D group:
Summary
Real-time signal processing consumes the majority of the world's computing power. Increasingly, programmable parallel processors are used to address a wide variety of signal processing applications (e.g., scientific, video, wireless, medical, communication, encoding, radar, sonar, and imaging). In programmable systems, the major challenge is no longer hardware but software. Specifically, the key technical hurdle lies in allowing the user to write programs at high level, while still achieving performance and preserving the portability of the code across parallel computing hardware platforms. The Parallel Vector, Signal, and Image Processing Library (Parallel VSIPL++) addresses this hurdle by providing high-level C++ array constructs, a simple mechanism for mapping data and functions onto parallel hardware, and a community-defined portable interface. This paper presents an overview of the Parallel VSIPL++ standard as well as a deeper description of the technical foundations and expected performance of the library. Parallel VSIPL++ supports adaptive optimization at many levels. The C++ arrays are designed to support automatic hardware specialization by the compiler. The computation objects (e.g., fast Fourier transforms) are built with explicit setup and run stages to allow for runtime optimization. Parallel arrays and functions in Parallel VSIPL++ also support explicit setup and run stages, which are used to accelerate communication operations. The parallel mapping mechanism provides an external interface that allows optimal mappings to be generated offline and read into the system at runtime. Finally, the standard has been developed in collaboration with high performance embedded computing vendors and is compatible with their proprietary approaches to achieving performance.
Summary
Real-time signal processing consumes the majority of the world's computing power. Increasingly, programmable parallel processors are used to address a wide variety of signal processing applications (e.g., scientific, video, wireless, medical, communication, encoding, radar, sonar, and imaging). In programmable systems, the major challenge is no longer hardware but software. Specifically...
READ MORE
Application of a Relative Development Time Productivity Metric to Parallel Software Development
January 1, 2005
Conference Paper
Published in:
SE-HPCS '05: Proceedings of the second international workshop on Software engineering for high performance computing system applications
Topic:
R&D area:
R&D group:
Summary
Evaluation of High Performance Computing (HPC) systems should take into account software development time productivity in addition to hardware performance, cost, and other factors. We propose a new metric for HPC software development time productivity, defined as the ratio of relative runtime performance to relative programmer effort. This formula has been used to analyze several HPC benchmark codes and classroom programming assignments. The results of this analysis show consistent trends for various programming models. This method enables a high-level evaluation of development time productivity for a given code implementation, which is essential to the task of estimating cost associated with HPC software development.
Summary
Evaluation of High Performance Computing (HPC) systems should take into account software development time productivity in addition to hardware performance, cost, and other factors. We propose a new metric for HPC software development time productivity, defined as the ratio of relative runtime performance to relative programmer effort. This formula has...
READ MORE
An analysis of wake vortex lidar measurements at LaGuardia Airport
November 30, 2004
Project Report
Published in:
Project Report ATC-318, MIT Lincoln Laboratory
Topic:
R&D area:
R&D group:
Summary
The majority of research into the wake vortex hazard has concentrated on the in-trail encounter scenario for arrivals. At LaGuardia Airport, wake vortex spacings are applied to arrivals on runway 22 following a heavy departure on the intersecting runway 31, resulting in delay and increased workload for controllers. Previous analysis of this problem led to a recommendation for a measurement campaign to collect data on the behavior of wake vortices generated by departing heavy aircraft. In April of 2004, MIT Lincoln Laboratory deployed its wake vortex lidar system to measure such wakes at LaGuardia. Additionally, wind speed and turbulence data were collected with the hope of correlating wake behavior with the local atmospheric conditions. Analysis of the lidar data indicates that the system was able to acquire and track vortices from departures, a task not proven prior to this deployment. Further, vortices were seen to transport toward the threshold of runway 22, verifying an assumption based on analysis of the winds that wake transport is not a solution in this case. The quantity and type of data collected were insufficient to formulate a clear relationship between atmospheric turbulence and vortex decay. However, it may be possible to develop such a model by exploiting the data gathered during previous lidar deployments.
Summary
The majority of research into the wake vortex hazard has concentrated on the in-trail encounter scenario for arrivals. At LaGuardia Airport, wake vortex spacings are applied to arrivals on runway 22 following a heavy departure on the intersecting runway 31, resulting in delay and increased workload for controllers. Previous analysis...
READ MORE
The MIT Lincoln Laboratory RT-04F diarization systems: applications to broadcast audio and telephone conversations
November 8, 2004
Conference Paper
Published in:
NIST Rich Transcription Workshop, 8-11 November 2004.
Topic:
R&D area:
Summary
Audio diarization is the process of annotating an input audio channel with information that attributes (possibly overlapping) temporal regions of signal energy to their specific sources. These sources can include particular speakers, music, background noise sources, and other signal source/channel characteristics. Diarization has utility in making automatic transcripts more readable and in searching and indexing audio archives. In this paper we describe the systems developed by MITLL and used in DARPA EARS Rich Transcription Fall 2004 (RT-04F) speaker diarization evaluation. The primary system is based on a new proxy speaker model approach and the secondary system follows a more standard BIC based clustering approach. We present experiments analyzing performance of the systems and present a cross-cluster recombination approach that significantly improves performance. In addition, we also present results applying our system to a telephone speech, summed channel speaker detection task.
Summary
Audio diarization is the process of annotating an input audio channel with information that attributes (possibly overlapping) temporal regions of signal energy to their specific sources. These sources can include particular speakers, music, background noise sources, and other signal source/channel characteristics. Diarization has utility in making automatic transcripts more readable...
READ MORE
Two new experimental protocols for measuring speech transcript readability for timed question-answering tasks
November 8, 2004
Conference Paper
Published in:
Proc. DARPA EARS Rich Translation Workshop, 8-11 November 2004.
Topic:
R&D area:
Summary
This paper reports results from two recent psycholinguistic experiments that measure the readability of four types of speech transcripts for the DARPA EARS Program. The two key questions in these experiments are (1) how much speech transcript cleanup aids readability and (2) how much the type of cleanup matters. We employ two variants of the four-part figure of merit to measure readability defined at the RT02 workshop and described in our Eurospeech 2003 paper [4] namely: accuracy of answers to comprehension questions, reaction-time for passage reading, reaction-time for question answering and a subjective rating of passage difficulty. The first protocol employs a question-answering task under time pressure. The second employs a self-paced line-by-line paradigm. Both protocols yield similar results: all three types of clean-up in the experiment improve readability 5-10%, but the self-paced reading protocol needs far fewer subjects for statistical significance.
Summary
This paper reports results from two recent psycholinguistic experiments that measure the readability of four types of speech transcripts for the DARPA EARS Program. The two key questions in these experiments are (1) how much speech transcript cleanup aids readability and (2) how much the type of cleanup matters. We...
READ MORE
MIMO radar theory and experimental results
November 7, 2004
Conference Paper
Published in:
38th Asilomar Conf. on Signals, Systems and Computers, Vol. 2, 7-10 November 2004, pp. 300-304.
Topic:
Summary
The continuing progress of Moore's law has enabled the development of radar systems that simultaneously transmit and receive multiple coded waveforms from multiple phase centers and to process them in ways that have been unavailable in the past. The signals available for processing from these Multiple-Input Multiple-Output (MIMO) radar systems appear as spatial samples corresponding to the convolution of the transmit and receive aperture phase centers. The samples provide the ability to excite and measure the channel that consists of the transmit/receive propagation paths, the target and incidental scattering or clutter. These signals may be processed and combined to form an adaptive coherent transmit beam, or to search an extended area with high resolution in a single dwell. Adaptively combining the received data provides the effect of adaptively controlling the transmit beamshape and the spatial extent provides improved track-while-scan accuracy. This paper describes the theory behind the improved surveillance radar performance and illustrates this with measurements from experimental MIMO radars.
Summary
The continuing progress of Moore's law has enabled the development of radar systems that simultaneously transmit and receive multiple coded waveforms from multiple phase centers and to process them in ways that have been unavailable in the past. The signals available for processing from these Multiple-Input Multiple-Output (MIMO) radar systems...
READ MORE