Publications
Large scale parallelization using file-based communications
September 24, 2019
Conference Paper
Published in:
2019 IEEE High Performance Computing Conf., HPEC, 24-26 September 2019.
Topic:
R&D area:
R&D group:
Summary
In this paper, we present a novel and new file-based communication architecture using the local filesystem for large scale parallelization. This new approach eliminates the issues with filesystem overload and resource contention when using the central filesystem for large parallel jobs. The new approach incurs additional overhead due to inter-node message file transfers when both the sending and receiving processes are not on the same node. However, even with this additional overhead cost, its benefits are far greater for the overall cluster operation in addition to the performance enhancement in message communications for large scale parallel jobs. For example, when running a 2048-process parallel job, it achieved about 34 times better performance with MPI_Bcast() when using the local filesystem. Furthermore, since the security for transferring message files is handled entirely by using the secure copy protocol (scp) and the file system permissions, no additional security measures or ports are required other than those that are typically required on an HPC system.
Summary
In this paper, we present a novel and new file-based communication architecture using the local filesystem for large scale parallelization. This new approach eliminates the issues with filesystem overload and resource contention when using the central filesystem for large parallel jobs. The new approach incurs additional overhead due to inter-node...
READ MORE
Sparse Deep Neural Network graph challenge
September 24, 2019
Conference Paper
Published in:
IEEE High Performance Extreme Computing Conf., HPEC, 24-26 September 2019.
Topic:
R&D area:
R&D group:
Summary
The MIT/IEEE/Amazon GraphChallenge.org encourages community approaches to developing new solutions for analyzing graphs and sparse data. Sparse AI analytics present unique scalability difficulties. The proposed Sparse Deep Neural Network (DNN) Challenge draws upon prior challenges from machine learning, high performance computing, and visual analytics to create a challenge that is reflective of emerging sparse AI systems. The Sparse DNN Challenge is based on a mathematically well-defined DNN inference computation and can be implemented in any programming environment. Sparse DNN inference is amenable to both vertex-centric implementations and array-based implementations (e.g., using the GraphBLAS.org standard). The computations are simple enough that performance predictions can be made based on simple computing hardware models. The input data sets are derived from the MNIST handwritten letters. The surrounding I/O and verification provide the context for each sparse DNN inference that allows rigorous definition of both the input and the output. Furthermore, since the proposed sparse DNN challenge is scalable in both problem size and hardware, it can be used to measure and quantitatively compare a wide range of present day and future systems. Reference implementations have been implemented and their serial and parallel performance have been measured. Specifications, data, and software are publicly available at GraphChallenge.org.
Summary
The MIT/IEEE/Amazon GraphChallenge.org encourages community approaches to developing new solutions for analyzing graphs and sparse data. Sparse AI analytics present unique scalability difficulties. The proposed Sparse Deep Neural Network (DNN) Challenge draws upon prior challenges from machine learning, high performance computing, and visual analytics to create a challenge that is...
READ MORE
Introducing DyMonDS-as-a-Service (DyMaaS) for Internet of Things
September 24, 2019
Conference Paper
Published in:
2019 IEEE High Performance Computing Conf., HPEC, 24-26 September 2019.
Summary
With recent trends in computation and communication architecture, it is becoming possible to simulate complex networked dynamical systems by employing high-fidelity models. The inherent spatial and temporal complexity of these systems, however, still acts as a roadblock. It is thus desirable to have adaptive platform design facilitating zooming-in and out of the models to emulate time-evolution of processes at a desired spatial and temporal granularity. In this paper, we propose new computing and networking abstractions, that can embrace physical dynamics and computations in a unified manner, by taking advantage of the inherent structure. We further design multi-rate numerical methods that can be implemented by computing architectures to facilitate adaptive zooming-in and out of the models spanning multiple spatial and temporal layers. These methods are all embedded in a platform called Dynamic Monitoring and Decision Systems (DyMonDS). We introduce a new service model of cloud computing called DyMonDS-as-a-Service (DyMaas), for use by operators at various spatial granularities to efficiently emulate the interconnection of IoT devices. The usage of this platform is described in the context of an electric microgrid system emulation.
Summary
With recent trends in computation and communication architecture, it is becoming possible to simulate complex networked dynamical systems by employing high-fidelity models. The inherent spatial and temporal complexity of these systems, however, still acts as a roadblock. It is thus desirable to have adaptive platform design facilitating zooming-in and out...
READ MORE
Security Design of Mission-Critical Embedded Systems
September 24, 2019
Abstract
Published in:
HPEC 2019: IEEE Conf. on High Performance Extreme Computing, 22-24 September 2019.
Topic:
R&D area:
R&D group:
Summary
This tutorial explains a systematic approach of co-designing functionality and security into mission-criticalembedded systems. The tutorial starts by reviewing common issues in embedded applications to define mission objectives,threat models, and security/resilience goals. We then introduce an overview of security technologies toachieve goals of confidentiality, integrity, and availability given design criteria and a realistic threatmodel. The technologies range from practical cryptography and key management, protection of data atrest, data in transit, and data in use, and tamper resistance.A major portion of the tutorial is dedicated to exploring the mission critical embedded system solutionspace. We discuss the search for security vulnerabilities (red teaming) and the search for solutions (blueteaming). Besides the lecture, attendees, under instructor guidance, will perform realistic andmeaningful hands-on exercises of defining mission and security objectives, assessing principal issues,applying technologies, and understanding their interactions. The instructor will provide an exampleapplication (distributed sensing, communicating, and computing) to be used in these exercises.Attendees could also bring their own applications for the exercises.Attendees are encouraged to work collaboratively throughout the development process, thus creatingopportunities to learn from each other. During the exercise, attendees will consider the use of varioussecurity/resilience features, articulate and justify the use of resources, and assess the system’ssuitability for mission assurance. Attendees can expect to gain valuable insight and experience in thesubject after completing the lecture and exercises.The instructor, who is an expert and practitioner in the field, will offer insight, advice, and concreteexamples and discussions. The tutorial draws from the instructor’s decades of experience in secure,resilient systems and technology.
Summary
This tutorial explains a systematic approach of co-designing functionality and security into mission-criticalembedded systems. The tutorial starts by reviewing common issues in embedded applications to define mission objectives,threat models, and security/resilience goals. We then introduce an overview of security technologies toachieve goals of confidentiality, integrity, and availability given design criteria...
READ MORE
Streaming 1.9 billion hyperspace network updates per second with D4M
September 24, 2019
Conference Paper
Published in:
2019 IEEE High Performance Computing Conf., HPEC, 24-26 September 2019.
Topic:
R&D area:
R&D group:
Summary
The Dynamic Distributed Dimensional Data Model (D4M) library implements associative arrays in a variety of languages (Python, Julia, and Matlab/Octave) and provides a lightweight in-memory database implementation of hypersparse arrays that are ideal for analyzing many types of network data. D4M relies on associative arrays which combine properties of spreadsheets, databases, matrices, graphs, and networks, while providing rigorous mathematical guarantees, such as linearity. Streaming updates of D4M associative arrays put enormous pressure on the memory hierarchy. This work describes the design and performance optimization of an implementation of hierarchical associative arrays that reduces memory pressure and dramatically increases the update rate into an associative array. The parameters of hierarchical associative arrays rely on controlling the number of entries in each level in the hierarchy before an update is cascaded. The parameters are easily tunable to achieve optimal performance for a variety of applications. Hierarchical arrays achieve over 40,000 updates per second in a single instance. Scaling to 34,000 instances of hierarchical D4M associative arrays on 1,100 server nodes on the MIT SuperCloud achieved a sustained update rate of 1,900,000,000 updates per second. This capability allows the MIT SuperCloud to analyze extremely large streaming network data sets.
Summary
The Dynamic Distributed Dimensional Data Model (D4M) library implements associative arrays in a variety of languages (Python, Julia, and Matlab/Octave) and provides a lightweight in-memory database implementation of hypersparse arrays that are ideal for analyzing many types of network data. D4M relies on associative arrays which combine properties of spreadsheets...
READ MORE
Survey and benchmarking of machine learning accelerators
September 24, 2019
Conference Paper
Published in:
IEEE High Performance Extreme Computing Conf., HPEC, 24-26 September 2019.
Topic:
R&D area:
R&D group:
Summary
Advances in multicore processors and accelerators have opened the flood gates to greater exploration and application of machine learning techniques to a variety of applications. These advances, along with breakdowns of several trends including Moore's Law, have prompted an explosion of processors and accelerators that promise even greater computational and machine learning capabilities. These processors and accelerators are coming in many forms, from CPUs and GPUs to ASICs, FPGAs, and dataflow accelerators. This paper surveys the current state of these processors and accelerators that have been publicly announced with performance and power consumption numbers. The performance and power values are plotted on a scatter graph and a number of dimensions and observations from the trends on this plot are discussed and analyzed. For instance, there are interesting trends in the plot regarding power consumption, numerical precision, and inference versus training. We then select and benchmark two commercially-available low size, weight, and power (SWaP) accelerators as these processors are the most interesting for embedded and mobile machine learning inference applications that are most applicable to the DoD and other SWaP constrained users. We determine how they actually perform with real-world images and neural network models, compare those results to the reported performance and power consumption values and evaluate them against an Intel CPU that is used in some embedded applications.
Summary
Advances in multicore processors and accelerators have opened the flood gates to greater exploration and application of machine learning techniques to a variety of applications. These advances, along with breakdowns of several trends including Moore's Law, have prompted an explosion of processors and accelerators that promise even greater computational and...
READ MORE
Toward technically feasible and economically efficient integration of distributed energy resources
September 24, 2019
Conference Paper
Published in:
57th Annual Allerton Conf. on Communication, Control, and Computing, 24-27 September 2019.
Summary
This paper formulates the efficient and feasible participation of distributed energy resources (DERs) in complex electricity services as a centralized nonlinear optimization problem first. This problem is then re-stated using the novel energy/power transformed state space. It is shown that the DER dynamics in closed-loop can be made linear in this new state space. The decision making by the DERs then becomes a distributed model predictive control problem and it forms the basis for deriving physically implementable convex market bids. A multi-layered interactive optimization for clearing the distributed bids by higher layer decision makers, such as market aggregators, is posed and shown to lead to near-optimal system-level performance at the slower market clearing rates. A proof-of-concept example is illustrated involving close to one hundred heterogeneous controllable DERs with real consumption data of a distribution feeder in Texas, contributing to automatic generation control (AGC).
Summary
This paper formulates the efficient and feasible participation of distributed energy resources (DERs) in complex electricity services as a centralized nonlinear optimization problem first. This problem is then re-stated using the novel energy/power transformed state space. It is shown that the DER dynamics in closed-loop can be made linear in...
READ MORE
Corpora design and score calibration for text dependent pronunciation proficiency recognition
September 20, 2019
Conference Paper
Published in:
8th ISCA Workshop on Speech and Language Technology in Education, SLaTe 2019, 20-21 September 2019.
Topic:
R&D area:
Summary
This work investigates methods for improving a pronunciation proficiency recognition system, both in terms of phonetic level posterior probability calibration, and in ordinal utterance level classification, for Modern Standard Arabic (MSA), Spanish and Russian. To support this work, utterance level labels were obtained by crowd-sourcing the annotation of language learners' recordings. Phonetic posterior probability estimates extracted using automatic speech recognition systems trained in each language were estimated using a beta calibration approach [1] and language proficiency level was estimated using an ordinal regression [2]. Fusion with language recognition (LR) scores from an i-vector system [3] trained on 23 languages is also explored. Initial results were promising for all three languages and it was demonstrated that the calibrated posteriors were effective for predicting pronunciation proficiency. Significant relative gains of 16% mean absolute error for the ordinal regression and 17% normalized cross entropy for the binary beta regression were achieved on MSA through fusion with LR scores.
Summary
This work investigates methods for improving a pronunciation proficiency recognition system, both in terms of phonetic level posterior probability calibration, and in ordinal utterance level classification, for Modern Standard Arabic (MSA), Spanish and Russian. To support this work, utterance level labels were obtained by crowd-sourcing the annotation of language learners'...
READ MORE
Using K-means in SVR-based text difficulty estimation
September 20, 2019
Conference Paper
Published in:
8th ISCA Workshop on Speech and Language Technology in Education, SLaTE, 20-21 September 2019.
Topic:
R&D area:
Summary
A challenge for second language learners, educators, and test creators is the identification of authentic materials at the right level of difficulty. In this work, we present an approach to automatically measure text difficulty, integrated into Auto-ILR, a web-based system that helps find text material at the right level for learners in 18 languages. The Auto-ILR subscription service scans web feeds, extracts article content, evaluates the difficulty, and notifies users of documents that match their skill level. Difficulty is measured on the standard ILR scale with language-specific support vector machine regression (SVR) models built from vectors incorporating length features, term frequencies, relative entropy, and K-means clustering.
Summary
A challenge for second language learners, educators, and test creators is the identification of authentic materials at the right level of difficulty. In this work, we present an approach to automatically measure text difficulty, integrated into Auto-ILR, a web-based system that helps find text material at the right level for...
READ MORE
State-of-the-art speaker recognition for telephone and video speech: the JHU-MIT submission for NIST SRE18
September 15, 2019
Conference Paper
Published in:
Interspeech, 15-19 September 2019.
Topic:
R&D area:
Summary
We present a condensed description of the joint effort of JHUCLSP, JHU-HLTCOE, MIT-LL., MIT CSAIL and LSE-EPITA for NIST SRE18. All the developed systems consisted of xvector/i-vector embeddings with some flavor of PLDA backend. Very deep x-vector architectures–Extended and Factorized TDNN, and ResNets– clearly outperformed shallower xvectors and i-vectors. The systems were tailored to the video (VAST) or to the telephone (CMN2) condition. The VAST data was challenging, yielding 4 times worse performance than other video based datasets like Speakers in the Wild. We were able to calibrate the VAST data with very few development trials by using careful adaptation and score normalization methods. The VAST primary fusion yielded EER=10.18% and Cprimary= 0.431. By improving calibration in post-eval, we reached Cprimary=0.369. In CMN2, we used unsupervised SPLDA adaptation based on agglomerative clustering and score normalization to correct the domain shift between English and Tunisian Arabic models. The CMN2 primary fusion yielded EER=4.5% and Cprimary=0.313. Extended TDNN x-vector was the best single system obtaining EER=11.1% and Cprimary=0.452 in VAST; and 4.95% and 0.354 in CMN2.
Summary
We present a condensed description of the joint effort of JHUCLSP, JHU-HLTCOE, MIT-LL., MIT CSAIL and LSE-EPITA for NIST SRE18. All the developed systems consisted of xvector/i-vector embeddings with some flavor of PLDA backend. Very deep x-vector architectures–Extended and Factorized TDNN, and ResNets– clearly outperformed shallower xvectors and i-vectors. The...
READ MORE