Publications
Liquid crystal uncooled thermal imager development
September 9, 2014
Conference Paper
Published in:
2014 Military Sensing Symposia, (MSS 2014), Detectors and Materials, 9 September 2014.
Topic:
R&D area:
Summary
An uncooled thermal imager is being developed based on a liquid crystal transducer. The liquid crystal transducer changes a long-wavelength infrared scene into a visible image as opposed to an electric signal in microbolometers. This approach has the potential for making a more flexible thermal sensor. One objective is to develop imager technology scalable to large formats (tens of megapixels) while maintaining or improving the noise equivalent temperature difference (NETD) compared to microbolometers. Our work is demonstrating that the liquid crystals have the required performance (sensitivity, dynamic range, speed, etc.) to make state-of-the-art uncooled imagers. A process has been developed and arrays have been fabricated using the liquid crystals. A breadboard camera system has been assembled to test the imagers. Results of the measurements are discussed.
Summary
An uncooled thermal imager is being developed based on a liquid crystal transducer. The liquid crystal transducer changes a long-wavelength infrared scene into a visible image as opposed to an electric signal in microbolometers. This approach has the potential for making a more flexible thermal sensor. One objective is to...
READ MORE
Computing on masked data: a high performance method for improving big data veracity
September 9, 2014
Conference Paper
Published in:
HPEC 2014: IEEE Conf. on High Performance Extreme Computing, 9-11 September 2014.
Topic:
R&D area:
R&D group:
Summary
The growing gap between data and users calls for innovative tools that address the challenges faced by big data volume, velocity and variety. Along with these standard three V's of big data, an emerging fourth "V" is veracity, which addresses the confidentiality, integrity, and availability of the data. Traditional cryptographic techniques that ensure the veracity of data can have overheads that are too large to apply to big data. This work introduces a new technique called Computing on Masked Data (CMD), which improves data veracity by allowing computations to be performed directly on masked data and ensuring that only authorized recipients can unmask the data. Using the sparse linear algebra of associative arrays, CMD can be performed with significantly less overhead than other approaches while still supporting a wide range of linear algebraic operations on the masked data. Databases with strong support of sparse operations, such as SciDB or Apache Accumulo, are ideally suited to this technique. Examples are shown for the application of CMD to a complex DNA matching algorithm and to database operations over social media data.
Summary
The growing gap between data and users calls for innovative tools that address the challenges faced by big data volume, velocity and variety. Along with these standard three V's of big data, an emerging fourth "V" is veracity, which addresses the confidentiality, integrity, and availability of the data. Traditional cryptographic...
READ MORE
A survey of cryptographic approaches to securing big-data analytics in the cloud
September 9, 2014
Conference Paper
Published in:
HPEC 2014: IEEE Conf. on High Performance Extreme Computing, 9-11 September 2014.
Topic:
R&D area:
R&D group:
Summary
The growing demand for cloud computing motivates the need to study the security of data received, stored, processed, and transmitted by a cloud. In this paper, we present a framework for such a study. We introduce a cloud computing model that captures a rich class of big-data use-cases and allows reasoning about relevant threats and security goals. We then survey three cryptographic techniques - homomorphic encryption, verifiable computation, and multi-party computation - that can be used to achieve these goals. We describe the cryptographic techniques in the context of our cloud model and highlight the differences in performance cost associated with each.
Summary
The growing demand for cloud computing motivates the need to study the security of data received, stored, processed, and transmitted by a cloud. In this paper, we present a framework for such a study. We introduce a cloud computing model that captures a rich class of big-data use-cases and allows...
READ MORE
A test-suite generator for database systems
September 9, 2014
Conference Paper
Published in:
HPEC 2014: IEEE Conf. on High Performance Extreme Computing, 9-11 September 2014.
Topic:
R&D area:
R&D group:
Summary
In this paper, we describe the SPAR Test Suite Generator (STSG), a new test-suite generator for SQL style database systems. This tool produced an entire test suite (data, queries, and ground-truth answers) as a unit and in response to a user's specification. Thus, database evaluators could use this tool to craft test suites for particular aspects of a specific database system. The inclusion of ground-truth answers in the produced test suite, furthermore, allowed this tool to support both benchmarking (at various scales) and correctness-checking in a repeatable way. Lastly, the test-suite generator of this document was extensively profiled and optimized, and was designed for test-time agility.
Summary
In this paper, we describe the SPAR Test Suite Generator (STSG), a new test-suite generator for SQL style database systems. This tool produced an entire test suite (data, queries, and ground-truth answers) as a unit and in response to a user's specification. Thus, database evaluators could use this tool to...
READ MORE
Sparse matrix partitioning for parallel eigenanalysis of large static and dynamic graphs
September 9, 2014
Conference Paper
Published in:
HPEC 2014: IEEE Conf. on High Performance Extreme Computing, 9-11 September 2014.
Topic:
R&D area:
Summary
Numerous applications focus on the analysis of entities and the connections between them, and such data are naturally represented as graphs. In particular, the detection of a small subset of vertices with anomalous coordinated connectivity is of broad interest, for problems such as detecting strange traffic in a computer network or unknown communities in a social network. These problems become more difficult as the background graph grows larger and noisier and the coordination patterns become more subtle. In this paper, we discuss the computational challenges of a statistical framework designed to address this cross-mission challenge. The statistical framework is based on spectral analysis of the graph data, and three partitioning methods are evaluated for computing the principal eigenvector of the graph's residuals matrix. While a standard one-dimensional partitioning technique enables this computation for up to four billion vertices, the communication overhead prevents this method from being used for even larger graphs. Recent two-dimensional partitioning methods are shown to have much more favorable scaling properties. A data-dependent partitioning method, which has the best scaling performance, is also shown to improve computation time even as a graph changes over time, allowing amortization of the upfront cost.
Summary
Numerous applications focus on the analysis of entities and the connections between them, and such data are naturally represented as graphs. In particular, the detection of a small subset of vertices with anomalous coordinated connectivity is of broad interest, for problems such as detecting strange traffic in a computer network...
READ MORE
Big Data dimensional analysis
September 9, 2014
Conference Paper
Published in:
HPEC 2014: IEEE Conf. on High Performance Extreme Computing, 9-11 September 2014.
Topic:
R&D area:
R&D group:
Summary
The ability to collect and analyze large amounts of data is a growing problem within the scientific community. The growing gap between data and users calls for innovative tools that address the challenges faced by big data volume, velocity and variety. One of the main challenges associated with big data variety is automatically understanding the underlying structures and patterns of the data. Such an understanding is required as a pre-requisite to the application of advanced analytics to the data. Further, big data sets often contain anomalies and errors that are difficult to know a priori. Current approaches to understanding data structure are drawn from the traditional database ontology design. These approaches are effective, but often require too much human involvement to be effective for the volume, velocity and variety of data encountered by big data systems. Dimensional Data Analysis (DDA) is a proposed technique that allows big data analysts to quickly understand the overall structure of a big dataset, determine anomalies. DDA exploits structures that exist in a wide class of data to quickly determine the nature of the data and its statistical anomalies. DDA leverages existing schemas that are employed in big data databases today. This paper presents DDA, applies it to a number of data sets, and measures its performance. The overhead of DDA is low and can be applied to existing big data systems without greatly impacting their computing requirements.
Summary
The ability to collect and analyze large amounts of data is a growing problem within the scientific community. The growing gap between data and users calls for innovative tools that address the challenges faced by big data volume, velocity and variety. One of the main challenges associated with big data...
READ MORE
Achieving 100,000,000 database inserts per second using Accumulo and D4M
September 9, 2014
Conference Paper
Published in:
HPEC 2014: IEEE Conf. on High Performance Extreme Computing, 9-11 September 2014.
Topic:
R&D area:
R&D group:
Summary
The Apache Accumulo database is an open source relaxed consistency database that is widely used for government applications. Accumulo is designed to deliver high performance on unstructured data such as graphs of network data. This paper tests the performance of Accumulo using data from the Graph500 benchmark. The Dynamic Distributed Dimensional Data Model (D4M) software is used to implement the benchmark on a 216-node cluster running the MIT SuperCloud software stack. A peak performance of over 100,000,000 database inserts per second was achieved which is 100x larger than the highest previously published value for any other database. The performance scales linearly with the number of ingest clients, number of database servers, and data size. The performance was achieved by adapting several supercomputing techniques to this application: distributed arrays, domain decomposition, adaptive load balancing, and single-program-multiple-data programming.
Summary
The Apache Accumulo database is an open source relaxed consistency database that is widely used for government applications. Accumulo is designed to deliver high performance on unstructured data such as graphs of network data. This paper tests the performance of Accumulo using data from the Graph500 benchmark. The Dynamic Distributed...
READ MORE
Genetic sequence matching using D4M big data approaches
September 9, 2014
Conference Paper
Published in:
HPEC 2014: IEEE Conf. on High Performance Extreme Computing, 9-11 September 2014.
Summary
Recent technological advances in Next Generation Sequencing tools have led to increasing speeds of DNA sample collection, preparation, and sequencing. One instrument can produce over 600 Gb of genetic sequence data in a single run. This creates new opportunities to efficiently handle the increasing workload. We propose a new method of fast genetic sequence analysis using the Dynamic Distributed Dimensional Data Model (D4M) - an associative array environment for MATLAB developed at MIT Lincoln Laboratory. Based on mathematical and statistical properties, the method leverages big data techniques and the implementation of an Apache Acculumo database to accelerate computations one-hundred fold over other methods. Comparisons of the D4M method with the current gold-standard for sequence analysis, BLAST, show the two are comparable in the alignments they find. This paper will present an overview of the D4M genetic sequence algorithm and statistical comparisons with BLAST.
Summary
Recent technological advances in Next Generation Sequencing tools have led to increasing speeds of DNA sample collection, preparation, and sequencing. One instrument can produce over 600 Gb of genetic sequence data in a single run. This creates new opportunities to efficiently handle the increasing workload. We propose a new method...
READ MORE
Adaptive optics program at TMT
September 5, 2014
Conference Paper
Published in:
SPIE, Vol. 9148, Adaptive Optics Systems IV, 5 September 2014, 914801.
Summary
The TMT first light Adaptive Optics (AO) facility consists of the Narrow Field Infra-Red AO System (NFIRAOS) and the associated Laser Guide Star Facility (LGSF). NFIRAOS is a 60 x 60 laser guide star (LGS) multi-conjugate AO (MCAO) system, which provides uniform, diffraction-limited performance in the J, H, and K bands over 17-30 arc sec diameter fields with 50 per cent sky coverage at the galactive pole, as required to support the TMT science cases. NFIRAOS includes two deformable mirrors, six laser guide star wavefront sensors, and three low-order, infrared, natural guide star wavefront sensors within each client instument. The first light LGSF system includes six sodium lasers required to generate the NFIRAOS laser guide stars. In this paper, we will provide an update on the progress in designing, modeling, and validating the TMT first light AO systems and their components over the last two years. This will include pre-final design and prototyping for the deformable mirrors, fabrication and tests for the visible detectors, benchmarking and comparison of different algorithms and processing architecture for the Real Time Controller (RTC) and development tests of prototype candidate lasers. Comprehensive and detailed AO modeling is continuing to support the design and development of the first light AO facility. Main modeling topics studied during the last two years include further studies in the area of wavefront error budget, sky coverage, high precision astrometry for the galactic center and other observations, high contrast imaging with NFIRAOS and its first light instruments, Point Spread Function (PSF) reconstruction for LGS MCAO, LGS photon return and sophisticated low order mode temporal filtering.
Summary
The TMT first light Adaptive Optics (AO) facility consists of the Narrow Field Infra-Red AO System (NFIRAOS) and the associated Laser Guide Star Facility (LGSF). NFIRAOS is a 60 x 60 laser guide star (LGS) multi-conjugate AO (MCAO) system, which provides uniform, diffraction-limited performance in the J, H, and K...
READ MORE
Detecting small asteroids with the Space Surveillance Telescope
September 1, 2014
Journal Article
Published in:
Icarus, Vol. 239, 2014, pp. 253-9.
Topic:
R&D area:
Summary
The ability of the Space Surveillance Telescope (SST) to find small (2-15 m diameter) NEAs suitable for the NASA asteroid retrieval mission is investigated. Orbits from a simulated population of targetable small asteroids were propagated and observations with the SST were simulated. Different search patterns and telescope time allocation cases were considered, as well as losses due to FOV gaps and weather. It is concluded that a full-time, dedicated survey at the SST is likely necessary to find a useful population of these NEAs within the mission launch timeframe, especially if an object must be observed on >1 night at SST to qualify as a detection. The simulations were also performed for an identical telescope in the southern hemisphere, which is found to produce results very similar to the SST in New Mexico due to significant (~80%) overlap in the population of objects detected at each site. In addition to considering the SST's ability to detect small NEAs, a parallel study was performed focusing on >100 m diameter objects. This work shows that even with limited telescope time (3 nights per month) a substantial number of these larger objects would be detected.
Summary
The ability of the Space Surveillance Telescope (SST) to find small (2-15 m diameter) NEAs suitable for the NASA asteroid retrieval mission is investigated. Orbits from a simulated population of targetable small asteroids were propagated and observations with the SST were simulated. Different search patterns and telescope time allocation cases...
READ MORE