Publications

Refine Results

(Filters Applied) Clear All

Two experiments comparing reading with listening for human processing of conversational telephone speech

September 4, 2005
  |
Conference Paper
Author:
Douglas A. Jones
…
Published in:
6th Annual Conf. of the Int. Speech Communication Association, INTERSPEECH 2005, 4-8 September 2005.
Topic:
machine translation
R&D area:
R&D group:

Summary

We report on results of two experiments designed to compare subjects' ability to extract information from audio recordings of conversational telephone speech (CTS) with their ability to extract information from text transcripts of these conversations, with and without the ability to hear the audio recordings. Although progress in machine processing of CTS speech is well documented, human processing of these materials has not been as well studied. These experiments compare subject's processing time and comprehension of widely-available CTS data in audio and written formats -- one experiment involves careful reading and one involves visual scanning for information. We observed a very modest improvement using transcripts compared with the audio-only condition for the careful reading task (speed-up by a factor of 1.2) and a much more dramatic improvement using transcripts in the visual scanning task (speed-up by a factor of 2.9). The implications of the experiments are twofold: (1) we expect to see similar gains in human productivity for comparable applications outside the laboratory environment and (2) the gains can vary widely, depending on the specific tasks involved.
READ LESS

Summary

We report on results of two experiments designed to compare subjects' ability to extract information from audio recordings of conversational telephone speech (CTS) with their ability to extract information from text transcripts of these conversations, with and without the ability to hear the audio recordings. Although progress in machine processing...

READ MORE
Two experiments comparing reading with listening for human processing of conversational telephone speech

Dynamic buffer overflow detection

August 10, 2005
  |
Conference Paper
Author:
Michael A. Zhivich
…
Published in:
Workshop on Defining the State of the Art in Security Software Tools, 10-11 August 2005.
Topic:
software
R&D area:
R&D group:

Summary

The capabilities of seven dynamic buffer overflow detection tools (Chaperon, Valgrind, CCured, CRED, Insure++, ProPolice and TinyCC) are evaluated in this paper. These tools employ different approaches to runtime buffer overflow detection and range from commercial products to open-source gcc-enhancements. A comprehensive testsuite was developed consisting of specifically-designed test cases and model programs containing real-world vulnerabilities. Insure++, CCured and CRED provide the highest buffer overflow detection rates, but only CRED provides an open-source, extensible and scalable solution to detecting buffer overflows. Other tools did not detect off-by-one errors, did not scale to large programs, or performed poorly on complex programs.
READ LESS

Summary

The capabilities of seven dynamic buffer overflow detection tools (Chaperon, Valgrind, CCured, CRED, Insure++, ProPolice and TinyCC) are evaluated in this paper. These tools employ different approaches to runtime buffer overflow detection and range from commercial products to open-source gcc-enhancements. A comprehensive testsuite was developed consisting of specifically-designed test cases...

READ MORE
Dynamic buffer overflow detection

A wide area network simulation of single-round group membership algorithms

July 27, 2005
  |
Conference Paper
Author:
Roger I. Khazan
…
Published in:
NCA 2005: 4th IEEE Int. Symp. on Network Computing and Applications, 27-29 July 2005, pp. 159-168.
Topic:
cryptography
R&D area:
R&D group:

Summary

A recent theoretical result proposed Sigma, a novel GM protocol that forms views using a single-round of message exchange. Prior GM protocols have required more rounds in the worst-case. In this paper, we investigate how well Sigma performs in practice. We simulate Sigma using WAN connectivity traces and compare its performance to two leading GM protocols, Moshe and Ensemble. Our simulations show, consistently with theoretical results, that Sigma always terminates within one round of message exchange, faster than Moshe and Ensemble. Moreover, Sigma has less message overhead and produces virtually the same quality of views. We also observe that view-oriented GM in dynamic WAN-like environments is practical only in applications where GM need not respond to every disconnect immediately when detected. These applications are able, and prefer, to delay GM response and ignore transient disconnects, avoiding frequent futile view changes and associated overhead. We reference some applications in this category.
READ LESS

Summary

A recent theoretical result proposed Sigma, a novel GM protocol that forms views using a single-round of message exchange. Prior GM protocols have required more rounds in the worst-case. In this paper, we investigate how well Sigma performs in practice. We simulate Sigma using WAN connectivity traces and compare its...

READ MORE
A wide area network simulation of single-round group membership algorithms

Polymorphous computing architecture (PCA) kernel-level benchmarks [revision 1]

June 13, 2005
  |
Project Report
Author:
James M. Lebak
…
Published in:
MIT Lincoln Laboratory Report PCA-KERNEL-1,REV.1
Topic:
computer architecture
R&D area:
R&D group:

Summary

This document describes a series of kernel benchmarks for the PCA program. Each kernel benchmark is an operation of importance to DoD sensor applications making use of a PCA architecture. Many of these operations are a part of the composite example applications described elsewhere. The kernel-level benchmarks have been chosen to stress both computation and communication aspects of the architecture. "Computation" aspects include floating-point and integer performance, as well as the memory hierarchy, while the "communication" aspects include the network, the memory hierarchy, and the I/O capabilities. The particular benchmarks chosen are based on the frequency of their use in current and future applications. They are drawn from the areas of signal processing, communication, and information and knowledge processing. The specification of the benchmarks in this document is meant to be high-level and largely independent of the implementation.
READ LESS

Summary

This document describes a series of kernel benchmarks for the PCA program. Each kernel benchmark is an operation of importance to DoD sensor applications making use of a PCA architecture. Many of these operations are a part of the composite example applications described elsewhere. The kernel-level benchmarks have been chosen...

READ MORE
Polymorphous computing architecture (PCA) kernel-level benchmarks [revision 1]

Dynamic buffer overflow detection

June 10, 2005
  |
Conference Paper
Author:
Michael A. Zhivich
…
Published in:
Workshop on the Evaluation of Software Defect Detection Tools, 10 June 2005.
Topic:
cyber security
R&D area:
R&D group:

Summary

The capabilities of seven dynamic buffer overflow detection tools (Chaperon, Valgrind, CCured, CRED, Insure++, ProPolice and TinyCC) are evaluated in this paper. These tools employ different approaches to runtime buffer overflow detection and range from commercial products to open source gcc-enhancements. A comprehensive test suite was developed consisting of specifically-designed test cases and model programs containing real-world vulnerabilities. Insure++, CCured and CRED provide the highest buffer overflow detection rates, but only CRED provides an open-source, extensible and scalable solution to detecting buffer overflows. Other tools did not detect off-by-one errors, did not scale to large programs, or performed poorly on complex programs.
READ LESS

Summary

The capabilities of seven dynamic buffer overflow detection tools (Chaperon, Valgrind, CCured, CRED, Insure++, ProPolice and TinyCC) are evaluated in this paper. These tools employ different approaches to runtime buffer overflow detection and range from commercial products to open source gcc-enhancements. A comprehensive test suite was developed consisting of specifically-designed...

READ MORE
Dynamic buffer overflow detection

Application of a development time productivity metric to parallel software development

May 15, 2005
  |
Conference Paper
Author:
Andrew P. Funk
…
Published in:
SE-HPCS '05, 2nd Int. Worskhop on Software Engineering for High Performance Computing System Applications, 15 May 2005, pp. 8-12.
Topic:
high performance computing
R&D area:
R&D group:

Summary

Evaluation of High Performance Computing (HPC) systems should take into account software development time productivity in addition to hardware performance, cost, and other factors. We propose a new metric for HPC software development time productivity, defined as the ratio of relative runtime performance to relative programmer effort. This formula has been used to analyze several HPC benchmark codes and classroom programming assignments. The results of this analysis show consistent trends for various programming models. This method enables a high-level evaluation of development time productivity for a given code implementation, which is essential to the task of estimating cost associated with HPC software development.
READ LESS

Summary

Evaluation of High Performance Computing (HPC) systems should take into account software development time productivity in addition to hardware performance, cost, and other factors. We propose a new metric for HPC software development time productivity, defined as the ratio of relative runtime performance to relative programmer effort. This formula has...

READ MORE
Application of a development time productivity metric to parallel software development

Measuring translation quality by testing English speakers with a new Defense Language Proficiency Test for Arabic

May 2, 2005
  |
Conference Paper
Author:
Douglas A. Jones
…
Published in:
Int. Conf. on Intelligence Analysis, 2-5 May 2005.
Topic:
machine translation
R&D area:
R&D group:

Summary

We present results from an experiment in which educated English-native speakers answered questions from a machine translated version of a standardized Arabic language test. We compare the machine translation (MT) results with professional reference translations as a baseline for the purpose of determining the level of Arabic reading comprehension that current machine translation technology enables an English speaker to achieve. Furthermore, we explore the relationship between the current, broadly accepted automatic measures of performance for machine translation and the Defense Language Proficiency Test, a broadly accepted measure of effectiveness for evaluating foreign language proficiency. In doing so, we intend to help translate MT system performance into terms that are meaningful for satisfying Government foreign language processing requirements. The results of this experiment suggest that machine translation may enable Interagency Language Roundtable Level 2 performance, but is not yet adequate to achieve ILR Level 3. Our results are based on 69 human subjects reading 68 documents and answering 173 questions, giving a total of 4,692 timed document trials and 7,950 question trials. We propose Level 3 as a reasonable nearterm target for machine translation research and development.
READ LESS

Summary

We present results from an experiment in which educated English-native speakers answered questions from a machine translated version of a standardized Arabic language test. We compare the machine translation (MT) results with professional reference translations as a baseline for the purpose of determining the level of Arabic reading comprehension that...

READ MORE
Measuring translation quality by testing English speakers with a new Defense Language Proficiency Test for Arabic

Using leader-based communication to improve the scalability of single-round group membership algorithms

April 4, 2005
  |
Conference Paper
Author:
Roger I. Khazan
…
Published in:
IPDPS 2005: 19th Int. Parallel and Distributed Processing Symp., 4-8 April 2005, pp. 280-287.
Topic:
algorithms
R&D area:
R&D group:

Summary

Sigma, the first single-round group membership (GM) algorithm, was recently introduced and demonstrated to operate consistently with theoretical expectations in a simulated WAN environment. Sigma achieved similar quality of membership configurations as existing algorithms but required fewer message exchange rounds. We now consider Sigma in terms of scalability. Sigma involves all-to-all (A2A) type of communication among members. A2A protocols have been shown to perform worse than leader-based (LB) protocols in certain networks, due to greater message overhead and higher likelihood of message loss. Thus, although LB protocols often involve additional communication steps, they can be more efficient in practice, particularly in fault-prone networks with large numbers of participating nodes. In this paper, we present Leader-Based Sigma, which transforms the original all-to-all version into a more scalable centralized communication scheme, and discuss the rounds vs. messages tradeoff involved in optimizing GM algorithms for deployment in large-scale, fault-prone dynamic network environments.
READ LESS

Summary

Sigma, the first single-round group membership (GM) algorithm, was recently introduced and demonstrated to operate consistently with theoretical expectations in a simulated WAN environment. Sigma achieved similar quality of membership configurations as existing algorithms but required fewer message exchange rounds. We now consider Sigma in terms of scalability. Sigma involves...

READ MORE
Using leader-based communication to improve the scalability of single-round group membership algorithms

An annotated review of past papers on attack graphs

March 31, 2005
  |
Project Report
Author:
Richard P. Lippmann
…
Published in:
MIT Lincoln Laboratory Report IA-1
Topic:
attack graphs
R&D area:
R&D group:

Summary

This report reviews past research papers that describe how to construct attack graphs, how to use them to improve security of computer networks, and how to use them to analyze alerts from intrusion detection systems. Two commercial systems are described [I, 2], and a summary table compares important characteristics of past research studies. For each study, information is provided on the number of attacker goals, how graphs are constructed, sizes of networks analyzed, how well the approach scales to larger networks, and the general approach. Although research has made significant progress in the past few years, no system has analyzed networks with more than 20 hosts, and computation for most approaches scales poorly and would be impractical for networks with more than even a few hundred hosts. Current approaches also are limited because many require extensive and difficult-to-obtain details on attacks, many assume that host-to-host reachability information between all hosts is already available, and many produce an attack graph but do not automatically generate recommendations from that graph. Researchers have suggested promising approaches to alleviate some of these limitations, including grouping hosts to improve scaling, using worst-case default values for unknown attack details, and symbolically analyzing attack graphs to generate recommendations that improve security for critical hosts. Future research should explore these and other approaches to develop attack graph construction and analysis algorithms that can be applied to large enterprise networks.
READ LESS

Summary

This report reviews past research papers that describe how to construct attack graphs, how to use them to improve security of computer networks, and how to use them to analyze alerts from intrusion detection systems. Two commercial systems are described [I, 2], and a summary table compares important characteristics of...

READ MORE
An annotated review of past papers on attack graphs

Speaker adaptive cohort selection for Tnorm in text-independent speaker verification

March 19, 2005
  |
Conference Paper
Author:
Douglas E. Sturim
…
Published in:
Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, ICASSP, Vol. 1, 19-23 March 2005, pp. I-741 - I-744.
Topic:
speaker recognition
R&D area:
R&D group:

Summary

In this paper we discuss an extension to the widely used score normalization technique of test normalization (Tnorm) for text-independent speaker verification. A new method of speaker Adaptive-Tnorm that offers advantages over the standard Tnorm by adjusting the speaker set to the target model is presented. Examples of this improvement using the 2004 NIST SRE data are also presented.
READ LESS

Summary

In this paper we discuss an extension to the widely used score normalization technique of test normalization (Tnorm) for text-independent speaker verification. A new method of speaker Adaptive-Tnorm that offers advantages over the standard Tnorm by adjusting the speaker set to the target model is presented. Examples of this improvement...

READ MORE
Speaker adaptive cohort selection for Tnorm in text-independent speaker verification
571-580 of 803