Publications

Refine Results

(Filters Applied) Clear All

The Offshore Precipitation Capability

September 16, 2016
  |
Project Report
Author:
Mark S. Veillette
…
Published in:
MIT Lincoln Laboratory Report ATC-430
Topic:
artificial intelligence
R&D area:
R&D group:

Summary

In this work, machine learning and image processing methods are used to estimate radar-like precipitation intensity and echo top heights beyond the range of weather radar. The technology, called the Offshore Precipitation Capability (OPC), combines global lightning data with existing radar mosaics, five Geostationary Operational Environmental Satellite (GOES) channels, and several fields from the Rapid Refresh (RAP) 13 km numerical weather prediction model to create precipitation and echo top fields similar to those provided by existing Federal Aviation Administration (FAA) weather systems. Preprocessing and feature extraction methods are described to construct inputs for model training. A variety of machine learning algorithms are investigated to identify which provides the most accuracy. Output from the machine learning model is blended with existing radar mosaics to create weather radar-like analyses that extend into offshore regions. The resulting fields are validated using land radars and satellite precipitation measurements provided by the National Aeronautics and Space Administration (NASA) Global Precipitation Measurement Mission (GPM) core observatory satellite. This capability is initially being developed for the Miami Oceanic airspace with the goal of providing improved situational awareness for offshore air traffic control.
READ LESS

Summary

In this work, machine learning and image processing methods are used to estimate radar-like precipitation intensity and echo top heights beyond the range of weather radar. The technology, called the Offshore Precipitation Capability (OPC), combines global lightning data with existing radar mosaics, five Geostationary Operational Environmental Satellite (GOES) channels, and...

READ MORE
The Offshore Precipitation Capability

High-throughput ingest of data provenance records in Accumulo

September 13, 2016
  |
Conference Paper
Author:
Thomas M. Moyer
…
Published in:
HPEC 2016: IEEE Conf. on High Performance Extreme Computing, 13-15 September 2016.
Topic:
supercomputing
R&D area:
R&D group:

Summary

Whole-system data provenance provides deep insight into the processing of data on a system, including detecting data integrity attacks. The downside to systems that collect whole-system data provenance is the sheer volume of data that is generated under many heavy workloads. In order to make provenance metadata useful, it must be stored somewhere where it can be queried. This problem becomes even more challenging when considering a network of provenance-aware machines all collecting this metadata. In this paper, we investigate the use of D4M and Accumulo to support high-throughput data ingest of whole-system provenance data. We find that we are able to ingest 3,970 graph components per second. Centrally storing the provenance metadata allows us to build systems that can detect and respond to data integrity attacks that are captured by the provenance system.
READ LESS

Summary

Whole-system data provenance provides deep insight into the processing of data on a system, including detecting data integrity attacks. The downside to systems that collect whole-system data provenance is the sheer volume of data that is generated under many heavy workloads. In order to make provenance metadata useful, it must...

READ MORE
High-throughput ingest of data provenance records in Accumulo

HF vector sensor for radio astronomy: ground testing results

September 13, 2016
  |
Conference Paper
Author:
Mary Knapp
…
Published in:
AIAA Space 2016 Conf. and Exposition, 13-16 September 2016.
Topic:
sensors
R&D area:
R&D group:

Summary

The radio sky below ~10 MHz is largely unexplored due to the inability of ground-based telescopes to observe near or below the ionospheric plasma frequency, or cut-off frequency. A space-based interferometric array is required to probe the portion of the electromagnetic (E-M) spectrum below 10 MHz with sufficient angular resolution and sensitivity to be scientifically useful. Multi-spacecraft constellations scale quickly in cost and complexity as the number of spacecraft increases, so minimizing the number of required spacecraft for an interferometric array (while maintaining performance) is critical for feasibility. We present the HF (High Frequency, 3 to 30 MHz) Vector Sensor as a high performance spacecraft instrument in a future space-based interferometric array. The HF Vector Sensor is composed of three orthogonal dipoles and three orthogonal loop antennas with a common phase center. These six elements fully measure the E-M field of incoming radiation. We present the design of two prototype HF Vector Sensors, ground-based data collection at frequencies above the ionospheric cut-off, and imaging results using several different algorithms.
READ LESS

Summary

The radio sky below ~10 MHz is largely unexplored due to the inability of ground-based telescopes to observe near or below the ionospheric plasma frequency, or cut-off frequency. A space-based interferometric array is required to probe the portion of the electromagnetic (E-M) spectrum below 10 MHz with sufficient angular resolution...

READ MORE
HF vector sensor for radio astronomy: ground testing results

High-throughput ingest of data provenance records in Accumulo

September 13, 2016
  |
Conference Paper
Author:
Thomas M. Moyer
…
Published in:
HPEC 2016: IEEE Conf. on High Performance Extreme Computing, 13-15 September 2016.
Topic:
supercomputing
R&D area:
R&D group:

Summary

Whole-system data provenance provides deep insight into the processing of data on a system, including detecting data integrity attacks. The downside to systems that collect whole-system data provenance is the sheer volume of data that is generated under many heavy workloads. In order to make provenance metadata useful, it must be stored somewhere where it can be queried. This problem becomes even more challenging when considering a network of provenance-aware machines all collecting this metadata. In this paper, we investigate the use of D4M and Accumulo to support high-throughput data ingest of whole-system provenance data. We find that we are able to ingest 3,970 graph components per second. Centrally storing the provenance metadata allows us to build systems that can detect and respond to data integrity attacks that are captured by the provenance system.
READ LESS

Summary

Whole-system data provenance provides deep insight into the processing of data on a system, including detecting data integrity attacks. The downside to systems that collect whole-system data provenance is the sheer volume of data that is generated under many heavy workloads. In order to make provenance metadata useful, it must...

READ MORE
High-throughput ingest of data provenance records in Accumulo

I-vector speaker and language recognition system on Android

September 13, 2016
  |
Conference Paper
Author:
Christian Vazquez-Machado
…
Published in:
HPEC 2016: IEEE Conf. on High Performance Extreme Computing, 13-15 September 2016.
Topic:
language recognition
R&D area:
R&D group:

Summary

I-Vector based speaker and language identification provides state of the art performance. However, this comes as a more computationally complex solution, which can often lead to challenges in resource-limited devices, such as phones or tablets. We present the implementation of an I-Vector speaker and language recognition system on the Android platform in the form of a fully functional application that allows speaker enrollment and language/speaker scoring within mobile contexts. We include a detailed account of the challenges to port the system and its dependencies, which were necessary to optimize matrix operations in the I-Vector implementation. The system was benchmarked on a for a Google Nexus 6, showing a speed increase of 61.68% in scoring and 82.63% in enrollment operations with the implemented optimizations. The application was tested in mobile settings on a Nexus 7 tablet with forty participants, showing a rough accuracy of 84%. The optimized platform showed the capacity to perform near real-time recognition within a mobile setting and showcases the viability of I-Vector systems on resource-limited environments.
READ LESS

Summary

I-Vector based speaker and language identification provides state of the art performance. However, this comes as a more computationally complex solution, which can often lead to challenges in resource-limited devices, such as phones or tablets. We present the implementation of an I-Vector speaker and language recognition system on the Android...

READ MORE
I-vector speaker and language recognition system on Android

Sparse-coded net model and applications

September 13, 2016
  |
Conference Paper
Author:
Youngjune L. Gwon
…
Published in:
2016 IEEE Int. Workshop on Machine Learning for Signal Processing, 13-16 September 2016.
Topic:
artificial intelligence
R&D area:
R&D group:

Summary

As an unsupervised learning method, sparse coding can discover high-level representations for an input in a large variety of learning problems. Under semi-supervised settings, sparse coding is used to extract features for a supervised task such as classification. While sparse representations learned from unlabeled data independently of the supervised task perform well, we argue that sparse coding should also be built as a holistic learning unit optimizing on the supervised task objectives more explicitly. In this paper, we propose sparse-coded net, a feedforward model that integrates sparse coding and task-driven output layers, and describe training methods in detail. After pretraining a sparse-coded net via semi-supervised learning, we optimize its task-specific performance in a novel backpropagation algorithm that can traverse nonlinear feature pooling operators to update the dictionary. Thus, sparse-coded net can be applied to supervised dictionary learning. We evaluate sparse-coded net with classification problems in sound, image, and text data. The results confirm a significant improvement over semi-supervised learning as well as superior classification performance against deep stacked autoencoder neural network and GMM-SVM pipelines in small to medium-scale settings.
READ LESS

Summary

As an unsupervised learning method, sparse coding can discover high-level representations for an input in a large variety of learning problems. Under semi-supervised settings, sparse coding is used to extract features for a supervised task such as classification. While sparse representations learned from unlabeled data independently of the supervised task...

READ MORE
Sparse-coded net model and applications

Benchmarking SciDB data import on HPC systems

September 13, 2016
  |
Conference Paper
Author:
Siddharth S. Samsi
…
Published in:
HPEC 2016: IEEE Conf. on High Performance Extreme Computing, 13-15 September 2016.
Topic:
high performance computing
R&D area:
R&D group:

Summary

SciDB is a scalable, computational database management system that uses an array model for data storage. The array data model of SciDB makes it ideally suited for storing and managing large amounts of imaging data. SciDB is designed to support advanced analytics in database, thus reducing the need for extracting data for analysis. It is designed to be massively parallel and can run on commodity hardware in a high performance computing (HPC) environment. In this paper, we present the performance of SciDB using simulated image data. The Dynamic Distributed Dimensional Data Model (D4M) software is used to implement the benchmark on a cluster running the MIT SuperCloud software stack. A peak performance of 2.2M database inserts per second was achieved on a single node of this system. We also show that SciDB and the D4M toolbox provide more efficient ways to access random sub-volumes of massive datasets compared to the traditional approaches of reading volumetric data from individual files. This work describes the D4M and SciDB tools we developed and presents the initial performance results. This performance was achieved by using parallel inserts, a in-database merging of arrays as well as supercomputing techniques, such as distributed arrays and single-program-multiple-data programming.
READ LESS

Summary

SciDB is a scalable, computational database management system that uses an array model for data storage. The array data model of SciDB makes it ideally suited for storing and managing large amounts of imaging data. SciDB is designed to support advanced analytics in database, thus reducing the need for extracting...

READ MORE
Benchmarking SciDB data import on HPC systems

Enhancing HPC security with a user-based firewall

September 13, 2016
  |
Conference Paper
Author:
Andrew J. Prout
…
Published in:
HPEC 2016: IEEE Conf. on High Performance Extreme Computing, 13-15 September 2016.
Topic:
high performance computing
R&D area:
R&D group:

Summary

High Performance Computing (HPC) systems traditionally allow their users unrestricted use of their internal network. While this network is normally controlled enough to guarantee privacy without the need for encryption, it does not provide a method to authenticate peer connections. Protocols built upon this internal network, such as those used in MPI, Lustre, Hadoop, or Accumulo, must provide their own authentication at the application layer. Many methods have been employed to perform this authentication, such as operating system privileged ports, Kerberos, munge, TLS, and PKI certificates. However, support for all of these methods requires the HPC application developer to include support and the user to configure and enable these services. The user-based firewall capability we have prototyped enables a set of rules governing connections across the HPC internal network to be put into place using Linux netfilter. By using an operating system-level capability, the system is not reliant on any developer or user actions to enable security. The rules we have chosen and implemented are crafted to not impact the vast majority of users and be completely invisible to them. Additionally, we have measured the performance impact of this system under various workloads.
READ LESS

Summary

High Performance Computing (HPC) systems traditionally allow their users unrestricted use of their internal network. While this network is normally controlled enough to guarantee privacy without the need for encryption, it does not provide a method to authenticate peer connections. Protocols built upon this internal network, such as those used...

READ MORE
Enhancing HPC security with a user-based firewall

Benchmarking the Graphulo processing framework

September 13, 2016
  |
Conference Paper
Author:
Vijay N. Gadepally
…
Published in:
HPEC 2016: IEEE Conf. on High Performance Extreme Computing, 13-15 September 2016.
Topic:
high performance computing
R&D area:
R&D group:

Summary

Graph algorithms have wide applicability to a variety of domains and are often used on massive datasets. Recent standardization efforts such as the GraphBLAS are designed to specify a set of key computational kernels that hardware and software developers can adhere to. Graphulo is a processing framework that enables GraphBLAS kernels in the Apache Accumulo database. In our previous work, we have demonstrated a core Graphulo operation that performs large scale multiplication operations of database tables called TableMult. In this article, we present results of scaling the Graphulo engine to larger problems and scalablity when using greater number of resources. Specifically, we present the results of two experiments that demonstrate Graphulo scaling performance as linear with the number of available resources. The first experiment demonstrates cluster processing rates through Graphulo's TableMult operator on two large graphs, scaled between 2^17 and 2^19 vertices. The second experiment uses TableMult to extract a random set of rows from a large graph (2^19 nodes) to simulate a cued graph analytic. These benchmarking results are of relevance to Graphulo users who wish to apply Graphulo to their graph problems.
READ LESS

Summary

Graph algorithms have wide applicability to a variety of domains and are often used on massive datasets. Recent standardization efforts such as the GraphBLAS are designed to specify a set of key computational kernels that hardware and software developers can adhere to. Graphulo is a processing framework that enables GraphBLAS...

READ MORE
Benchmarking the Graphulo processing framework

In-storage embedded accelerator for sparse pattern processing

September 13, 2016
  |
Conference Paper
Author:
Sang-Woo Jun
…
Published in:
HPEC 2016: IEEE Conf. on High Performance Extreme Computing, 13-15 September 2016.
Topic:
high performance computing
R&D area:
R&D group:

Summary

We present a novel architecture for sparse pattern processing, using flash storage with embedded accelerators. Sparse pattern processing on large data sets is the essence of applications such as document search, natural language processing, bioinformatics, subgraph matching, machine learning, and graph processing. One slice of our prototype accelerator is capable of handling up to 1TB of data, and experiments show that it can outperform C/C++ software solutions on a 16-core system at a fraction of the power and cost; an optimized version of the accelerator can match the performance of a 48-core server.
READ LESS

Summary

We present a novel architecture for sparse pattern processing, using flash storage with embedded accelerators. Sparse pattern processing on large data sets is the essence of applications such as document search, natural language processing, bioinformatics, subgraph matching, machine learning, and graph processing. One slice of our prototype accelerator is capable...

READ MORE
In-storage embedded accelerator for sparse pattern processing
431-440 of 2301