Publications
Self-supervised contrastive pre-training for time series via time-frequency consistency
July 11, 2022
Journal Article
Published in:
arXiv, June 16, 2022.
Summary
Pre-training on time series poses a unique challenge due to the potential mismatch between pre-training and target domains, such as shifts in temporal dynamics, fast-evolving trends, and long-range and short cyclic effects, which can lead to poor downstream performance. While domain adaptation methods can mitigate these shifts, most methods need examples directly from the target domain, making them suboptimal for pre-training. To address this challenge, methods need to accommodate target domains with different temporal dynamics and be capable of doing so without seeing any target examples during pre-training. Relative to other modalities, in time series, we expect that time-based and frequency-based representations of the same example are located close together in the time-frequency space. To this end, we posit that time-frequency consistency (TF-C) — embedding a time-based neighborhood of a particular example close to its frequency-based neighborhood and back—is desirable for pre-training. Motivated by TF-C, we define a decomposable pre-training model, where the self-supervised signal is provided by the distance between time and frequency components, each individually trained by contrastive estimation. We evaluate the new method on eight datasets, including electrodiagnostic testing, human activity recognition, mechanical fault detection, and physical status monitoring. Experiments against eight state-of-the-art methods show that TF-C outperforms baselines by 15.4% (F1 score) on average in one-to-one settings (e.g., fine-tuning an EEG-pretrained model on EMG data) and by up to 8.4% (F1 score) in challenging one-to-many settings (e.g., fine-tuning an EEG-pretrained model for either hand-gesture recognition or mechanical fault prediction), reflecting the breadth of scenarios that arise in real-world applications. The source code and datasets are available at https://anonymous.4open.science/r/TFC-pretraining-6B07.
Summary
Pre-training on time series poses a unique challenge due to the potential mismatch between pre-training and target domains, such as shifts in temporal dynamics, fast-evolving trends, and long-range and short cyclic effects, which can lead to poor downstream performance. While domain adaptation methods can mitigate these shifts, most methods need...
READ MORE
Axon tracing and centerline detection using topologically-aware 3D U-nets
July 11, 2022
Journal Article
Published in:
2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2022, pp. 238-242
Summary
As advances in microscopy imaging provide an ever clearer window into the human brain, accurate reconstruction of neural connectivity can yield valuable insight into the relationship between brain structure and function. However, human manual tracing is a slow and laborious task, and requires domain expertise. Automated methods are thus needed to enable rapid and accurate analysis at scale. In this paper, we explored deep neural networks for dense axon tracing and incorporated axon topological information into the loss function with a goal to improve the performance on both voxel-based segmentation and axon centerline detection. We evaluated three approaches using a modified 3D U-Net architecture trained on a mouse brain dataset imaged with light sheet microscopy and achieved a 10% increase in axon tracing accuracy over previous methods. Furthermore, the addition of centerline awareness in the loss function outperformed the baseline approach across all metrics, including a boost in Rand Index by 8%.
Summary
As advances in microscopy imaging provide an ever clearer window into the human brain, accurate reconstruction of neural connectivity can yield valuable insight into the relationship between brain structure and function. However, human manual tracing is a slow and laborious task, and requires domain expertise. Automated methods are thus needed...
READ MORE
Fun as a strategic advantage: applying lessons in engagement from commercial games to military logistics training
July 1, 2022
Journal Article
Published in:
Train., Educ., Learn. Sci., Vol. 59, 2022, p. 180-7.
Topic:
Summary
Digital games offer many elements to augment traditional classroom lectures and reading assignments. They enable players to explore concepts through repeat play in a low-risk environment, and allow players to integrate feedback given during gameplay and evaluate their own performance. Commercial games leverage a number of features to engage players and hold their attention. But do those engagement-improving methods have a place in instructional environments with a captive and motivated audience? Our experience building a logistics supply chain training game for the Marine Corps University suggests that yes; applying lessons in engagement from commercial games can both help improve player experience with the learning environment, and potentially improve learning outcomes.
Summary
Digital games offer many elements to augment traditional classroom lectures and reading assignments. They enable players to explore concepts through repeat play in a low-risk environment, and allow players to integrate feedback given during gameplay and evaluate their own performance. Commercial games leverage a number of features to engage players...
READ MORE
Impact of haptic cues and an active ankle exoskeleton on gait characteristics
July 1, 2022
Journal Article
Published in:
Hum. Factors, Vol. 0, No. 0, July 2022, p. 1-12.
Summary
Objective This study examined the interaction of gait-synchronized vibrotactile cues with an active ankle exoskeleton that provides plantarflexion assistance. Background An exoskeleton that augments gait may support collaboration through feedback to the user about the state of the exoskeleton or characteristics of the task. Methods Participants (N = 16) were provided combinations of torque assistance and vibrotactile cues at pre-specified time points in late swing and early stance while walking on a self-paced treadmill. Participants were either given explicit instructions (N = 8) or were allowed to freely interpret (N=8) how to coordinate with cues. Results For the free interpretation group, the data support an 8% increase in stride length and 14% increase in speed with exoskeleton torque across cue timing, as well as a 5% increase in stride length and 7% increase in speed with only vibrotactile cues. When given explicit instructions, participants modulated speed according to cue timing-increasing speed by 17% at cues in late swing and decreasing speed 11% at cues in early stance compared to no cue when exoskeleton torque was off. When torque was on, participants with explicit instructions had reduced changes in speed. Conclusion These findings support that the presence of torque mitigates how cues were used and highlights the importance of explicit instructions for haptic cuing. Interpreting cues while walking with an exoskeleton may increase cognitive load, influencing overall human-exoskeleton performance for novice users. Application Interactions between haptic feedback and exoskeleton use during gait can inform future feedback designs to support coordination between users and exoskeletons.
Summary
Objective This study examined the interaction of gait-synchronized vibrotactile cues with an active ankle exoskeleton that provides plantarflexion assistance. Background An exoskeleton that augments gait may support collaboration through feedback to the user about the state of the exoskeleton or characteristics of the task. Methods Participants (N = 16) were...
READ MORE
Impact of WSR-88D intra-volume low-level scans on sever weather warning performance
July 1, 2022
Journal Article
Published in:
Weather Forecast., Vol. 37, No. 7, July 2022, p. 1169-98.
Summary
The statistical relationship between supplemental adaptive intra-volume low-level scan (SAILS) usage on the Weather Surveillance Radar-1988 Doppler and National Weather Service severe storm warning performance during 2014–20 is analyzed. Results show statistically significant improvement in severe thunderstorm (SVR), flash flood (FF), and tornado (TOR) warning performance associated with SAILS-on versus SAILS-off. Within the three possible SAILS modes of one (SAILSx1), two (SAILSx2), and three (SAILSx3) additional base scans per volume, for SVR, SAILSx2 and SAILSx3 are associated with better warning performance compared to SAILSx1; for FF and TOR, SAILSx3 is associated with better warning performance relative to SAILSx1 and SAILSx2. Two severe storm cases (one that spawned a tornado, one that did not) are presented where SAILS usage helped forecasters make the correct TOR warning decision, lending real-life credence to the statistical results. Furthermore, a statistical analysis of automated volume scan evaluation and termination effects, parsed by SAILS usage and mode, yield a statistically significant association between volume scan update rate and SVR warning lead time.
Summary
The statistical relationship between supplemental adaptive intra-volume low-level scan (SAILS) usage on the Weather Surveillance Radar-1988 Doppler and National Weather Service severe storm warning performance during 2014–20 is analyzed. Results show statistically significant improvement in severe thunderstorm (SVR), flash flood (FF), and tornado (TOR) warning performance associated with SAILS-on versus...
READ MORE
Advances in cross-lingual and cross-source audio-visual speaker recognition: The JHU-MIT system for NIST SRE21
June 28, 2022
Conference Paper
Published in:
Speaker and Language Recognition Workshop, Odyssey 2022, pp. 213-220.
Topic:
R&D area:
Summary
We present a condensed description of the joint effort of JHUCLSP/HLTCOE, MIT-LL and AGH for NIST SRE21. NIST SRE21 consisted of speaker detection over multilingual conversational telephone speech (CTS) and audio from video (AfV). Besides the regular audio track, the evaluation also contains visual (face recognition) and multi-modal tracks. This evaluation exposes new challenges, including cross-source–i.e., CTS vs. AfV– and cross-language trials. Each speaker can speak two or three languages among English, Mandarin and Cantonese. For the audio track, we evaluated embeddings based on Res2Net and ECAPA-TDNN, where the former performed the best. We used PLDA based back-ends trained on previous SRE and VoxCeleb and adapted to a subset of Mandarin/Cantonese speakers. Some novel contributions of this submission are: the use of neural bandwidth extension (BWE) to reduce the mismatch between the AFV and CTS conditions; and invariant representation learning (IRL) to make the embeddings from a given speaker invariant to language. Res2Net with neural BWE was the best monolithic system. We used a pre-trained RetinaFace face detector and ArcFace embeddings for the visual track, following our NIST SRE19 work. We also included a new system using a deep pyramid single shot face detector and face embeddings trained on Crystal loss and probabilistic triplet loss, which performed the best. The number of face embeddings in the test video was reduced by agglomerative clustering or weighting the embedding based on the face detection confidence. Cosine scoring was used to compare embeddings. For the multi-modal track, we just added the calibrated likelihood ratios of the audio and visual conditions, assuming independence between modalities. The multi-modal fusion improved Cprimary by 72% w.r.t. audio.
Summary
We present a condensed description of the joint effort of JHUCLSP/HLTCOE, MIT-LL and AGH for NIST SRE21. NIST SRE21 consisted of speaker detection over multilingual conversational telephone speech (CTS) and audio from video (AfV). Besides the regular audio track, the evaluation also contains visual (face recognition) and multi-modal tracks. This...
READ MORE
Advances in speaker recognition for multilingual conversational telephone speech: the JHU-MIT system for NIST SRE20 CTS challenge
June 27, 2022
Conference Paper
Published in:
Speaker and Language Recognition Workshop, Odyssey 2022, pp. 338-345.
R&D area:
Summary
We present a condensed description of the joint effort of JHUCLSP/HLTCOE and MIT-LL for NIST SRE20. NIST SRE20 CTS consisted of multilingual conversational telephone speech. The set of languages included in the evaluation was not provided, encouraging the participants to develop systems robust to any language. We evaluated x-vector architectures based on ResNet, squeeze-excitation ResNets, Transformers and EfficientNets. Though squeeze-excitation ResNets and EfficientNets provide superior performance in in-domain tasks like VoxCeleb, regular ResNet34 was more robust in the challenge scenario. On the contrary, squeeze-excitation networks over-fitted to the training data, mostly in English. We also proposed a novel PLDA mixture and k-NN PLDA back-ends to handle the multilingual trials. The former clusters the x-vector space expecting that each cluster will correspond to a language family. The latter trains a PLDA model adapted to each enrollment speaker using the nearest speakers–i.e., those with similar language/channel. The k-NN back-end improved Act. Cprimary (Cp) by 68% in SRE16-19 and 22% in SRE20 Progress w.r.t. a single adapted PLDA back-end. Our best single system achieved Act. Cp=0.110 in SRE20 progress. Meanwhile, our best fusion obtained Act. Cp=0.110 in the progress–8% better than single– and Cp=0.087 in the eval set.
Summary
We present a condensed description of the joint effort of JHUCLSP/HLTCOE and MIT-LL for NIST SRE20. NIST SRE20 CTS consisted of multilingual conversational telephone speech. The set of languages included in the evaluation was not provided, encouraging the participants to develop systems robust to any language. We evaluated x-vector architectures...
READ MORE
AAM-Gym: Artificial intelligence testbed for advanced air mobility
June 9, 2022
Journal Article
Published in:
arXiv, 9 June 2022.
Topic:
R&D area:
R&D group:
Summary
We introduce AAM-Gym, a research and development testbed for Advanced Air Mobility (AAM). AAM has the potential to revolutionize travel by reducing ground traffic and emissions by leveraging new types of aircraft such as electric vertical take-off and landing (eVTOL) aircraft and new advanced artificial intelligence (AI) algorithms. Validation of AI algorithms require representative AAM scenarios, as well as a fast time simulation testbed to evaluate their performance. Until now, there has been no such testbed available for AAM to enable a common research platform for individuals in government, industry, or academia. MIT Lincoln Laboratory has developed AAM-Gym to address this gap by providing an ecosystem to develop, train, and validate new and established AI algorithms across a wide variety of AAM use-cases. In this paper, we use AAM-Gym to study the performance of two reinforcement learning algorithms on an AAM use-case, separation assurance in AAM corridors. The performance of the two algorithms is demonstrated based on a series of metrics provided by AAM-Gym, showing the testbed’s utility to AAM research.
Summary
We introduce AAM-Gym, a research and development testbed for Advanced Air Mobility (AAM). AAM has the potential to revolutionize travel by reducing ground traffic and emissions by leveraging new types of aircraft such as electric vertical take-off and landing (eVTOL) aircraft and new advanced artificial intelligence (AI) algorithms. Validation of...
READ MORE
Toward improving EN adoption: Bridging the gap between stated intention and actual use
June 3, 2022
Project Report
Published in:
MIT Lincoln Laboratory Report ACTA-7
Summary
As the COVID-19 pandemic swept the globe in the spring of 2020, technologists looked to enlist technology to assist public health authorities (PHAs) and help stem the tide of infections. As part of this technology push, experts in health care, cryptography, and other related fields developed the Private Automated Contact Tracing (PACT) protocol and related projects to assist the public health objective of slowing the spread of SARS-CoV-2 through digital contact tracing. The joint Google and Apple deployed protocol (Google-Apple Exposure Notifications, also known as GAEN or EN), which became the de facto standard in the U.S., employs the same features as detailed by PACT. The protocol leverages smartphone Bluetooth communications to alert users of potential contact with those carrying the COVID-19 virus in a way that preserves the privacy of both the known-infected individual, and the users receiving the alert. Contact tracing and subsequent personal precautions are more effective at reducing disease spread when more of the population participates, but there are known difficulties with the adoption of novel technology. In order to help the U.S. Centers for Disease Control and Prevention (CDC) and U.S. state-level public health teams address these difficulties, a team of staff from MIT's Lincoln Laboratory (MIT LL) and Computer Science and Artificial Intelligence Laboratory (MIT CSAIL) focused on studying user perception and information needs.
Summary
As the COVID-19 pandemic swept the globe in the spring of 2020, technologists looked to enlist technology to assist public health authorities (PHAs) and help stem the tide of infections. As part of this technology push, experts in health care, cryptography, and other related fields developed the Private Automated Contact...
READ MORE
The thundering herd: Amplifying kernel interference to attack response times
May 4, 2022
Conference Paper
Published in:
2022 IEEE 28th Real-Time and Embedded Technology and Applications Symp., RTAS, 4-6 May 2022.
Topic:
R&D area:
R&D group:
Summary
Embedded and real-time systems are increasingly attached to networks. This enables broader coordination beyond the physical system, but also opens the system to attacks. The increasingly complex workloads of these systems include software of varying assurance levels, including that which might be susceptible to compromise by remote attackers. To limit the impact of compromise, u-kernels focus on maintaining strong memory protection domains between different bodies of software, including system services. They enable limited coordination between processes through Inter-Process Communication (IPC). Real-time systems also require strong temporal guarantees for tasks, and thus need temporal isolation to limit the impact of malicious software. This is challenging as multiple client threads that use IPC to request service from a shared server will impact each other's response times. To constrain the temporal interference between threads, modern u-kernels often build priority and budget awareness into the system. Unfortunately, this paper demonstrates that this is more challenging than previously thought. Adding priority awareness to IPC processing can lead to significant interference due to the kernel's prioritization logic. Adding budget awareness similarly creates opportunities for interference due to the budget tracking and management operations. In both situations, a Thundering Herd of malicious threads can significantly delay the activation of mission-critical tasks. The Thundering Herd effects are evaluated on seL4 and results demonstrate that high-priority threads can be delayed by over 100,000 cycles per malicious thread. This paper reveals a challenging dilemma: the temporal protections u-kernels add can, themselves, provide means of threatening temporal isolation. Finally, to defend the system, we identify and empirically evaluate possible mitigations, and propose an admission-control test based upon an interference-aware analysis.
Summary
Embedded and real-time systems are increasingly attached to networks. This enables broader coordination beyond the physical system, but also opens the system to attacks. The increasingly complex workloads of these systems include software of varying assurance levels, including that which might be susceptible to compromise by remote attackers. To limit...
READ MORE