Publications
Transfer learning for automated COVID-19 B-line classification in lung ultrasound
July 15, 2022
Conference Paper
Published in:
44th Annual Int. Conf. of IEEE Engineering in Medicine & Biology Society (EMBC), DOI: 10.1109/EMBC48229.2022.9871894.
Topic:
R&D area:
R&D group:
Summary
Lung ultrasound (LUS) as a diagnostic tool is gaining support for its role in the diagnosis and management of COVID-19 and a number of other lung pathologies. B-lines are a predominant feature in COVID-19, however LUS requires a skilled clinician to interpret findings. To facilitate the interpretation, our main objective was to develop automated methods to classify B-lines as pathologic vs. normal. We developed transfer learning models based on ResNet networks to classify B-lines as pathologic (at least 3 B-lines per lung field) vs. normal using COVID-19 LUS data. Assessment of B-line severity on a 0-4 multi-class scale was also explored. For binary B-line classification, at the frame-level, all ResNet models pretrained with ImageNet yielded higher performance than the baseline nonpretrained ResNet-18. Pretrained ResNet-18 has the best Equal Error Rate (EER) of 9.1% vs the baseline of 11.9%. At the clip-level, all pretrained network models resulted in better Cohen's kappa agreement (linear-weighted) and clip score accuracy, with the pretrained ResNet-18 having the best Cohen's kappa of 0.815 [95% CI: 0.804-0.826], and ResNet-101 the best clip scoring accuracy of 93.6%. Similar results were shown for multi-class scoring, where pretrained network models outperformed the baseline model. A class activation map is also presented to guide clinicians in interpreting LUS findings. Future work aims to further improve the multi-class assessment for severity of B-lines with a more diverse LUS dataset.
Summary
Lung ultrasound (LUS) as a diagnostic tool is gaining support for its role in the diagnosis and management of COVID-19 and a number of other lung pathologies. B-lines are a predominant feature in COVID-19, however LUS requires a skilled clinician to interpret findings. To facilitate the interpretation, our main objective...
READ MORE
Axon tracing and centerline detection using topologically-aware 3D U-nets
July 11, 2022
Journal Article
Published in:
2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2022, pp. 238-242
Summary
As advances in microscopy imaging provide an ever clearer window into the human brain, accurate reconstruction of neural connectivity can yield valuable insight into the relationship between brain structure and function. However, human manual tracing is a slow and laborious task, and requires domain expertise. Automated methods are thus needed to enable rapid and accurate analysis at scale. In this paper, we explored deep neural networks for dense axon tracing and incorporated axon topological information into the loss function with a goal to improve the performance on both voxel-based segmentation and axon centerline detection. We evaluated three approaches using a modified 3D U-Net architecture trained on a mouse brain dataset imaged with light sheet microscopy and achieved a 10% increase in axon tracing accuracy over previous methods. Furthermore, the addition of centerline awareness in the loss function outperformed the baseline approach across all metrics, including a boost in Rand Index by 8%.
Summary
As advances in microscopy imaging provide an ever clearer window into the human brain, accurate reconstruction of neural connectivity can yield valuable insight into the relationship between brain structure and function. However, human manual tracing is a slow and laborious task, and requires domain expertise. Automated methods are thus needed...
READ MORE
Self-supervised contrastive pre-training for time series via time-frequency consistency
July 11, 2022
Journal Article
Published in:
arXiv, June 16, 2022.
Summary
Pre-training on time series poses a unique challenge due to the potential mismatch between pre-training and target domains, such as shifts in temporal dynamics, fast-evolving trends, and long-range and short cyclic effects, which can lead to poor downstream performance. While domain adaptation methods can mitigate these shifts, most methods need examples directly from the target domain, making them suboptimal for pre-training. To address this challenge, methods need to accommodate target domains with different temporal dynamics and be capable of doing so without seeing any target examples during pre-training. Relative to other modalities, in time series, we expect that time-based and frequency-based representations of the same example are located close together in the time-frequency space. To this end, we posit that time-frequency consistency (TF-C) — embedding a time-based neighborhood of a particular example close to its frequency-based neighborhood and back—is desirable for pre-training. Motivated by TF-C, we define a decomposable pre-training model, where the self-supervised signal is provided by the distance between time and frequency components, each individually trained by contrastive estimation. We evaluate the new method on eight datasets, including electrodiagnostic testing, human activity recognition, mechanical fault detection, and physical status monitoring. Experiments against eight state-of-the-art methods show that TF-C outperforms baselines by 15.4% (F1 score) on average in one-to-one settings (e.g., fine-tuning an EEG-pretrained model on EMG data) and by up to 8.4% (F1 score) in challenging one-to-many settings (e.g., fine-tuning an EEG-pretrained model for either hand-gesture recognition or mechanical fault prediction), reflecting the breadth of scenarios that arise in real-world applications. The source code and datasets are available at https://anonymous.4open.science/r/TFC-pretraining-6B07.
Summary
Pre-training on time series poses a unique challenge due to the potential mismatch between pre-training and target domains, such as shifts in temporal dynamics, fast-evolving trends, and long-range and short cyclic effects, which can lead to poor downstream performance. While domain adaptation methods can mitigate these shifts, most methods need...
READ MORE
Fun as a strategic advantage: applying lessons in engagement from commercial games to military logistics training
July 1, 2022
Journal Article
Published in:
Train., Educ., Learn. Sci., Vol. 59, 2022, p. 180-7.
Topic:
Summary
Digital games offer many elements to augment traditional classroom lectures and reading assignments. They enable players to explore concepts through repeat play in a low-risk environment, and allow players to integrate feedback given during gameplay and evaluate their own performance. Commercial games leverage a number of features to engage players and hold their attention. But do those engagement-improving methods have a place in instructional environments with a captive and motivated audience? Our experience building a logistics supply chain training game for the Marine Corps University suggests that yes; applying lessons in engagement from commercial games can both help improve player experience with the learning environment, and potentially improve learning outcomes.
Summary
Digital games offer many elements to augment traditional classroom lectures and reading assignments. They enable players to explore concepts through repeat play in a low-risk environment, and allow players to integrate feedback given during gameplay and evaluate their own performance. Commercial games leverage a number of features to engage players...
READ MORE
Impact of WSR-88D intra-volume low-level scans on sever weather warning performance
July 1, 2022
Journal Article
Published in:
Weather Forecast., Vol. 37, No. 7, July 2022, p. 1169-98.
Summary
The statistical relationship between supplemental adaptive intra-volume low-level scan (SAILS) usage on the Weather Surveillance Radar-1988 Doppler and National Weather Service severe storm warning performance during 2014–20 is analyzed. Results show statistically significant improvement in severe thunderstorm (SVR), flash flood (FF), and tornado (TOR) warning performance associated with SAILS-on versus SAILS-off. Within the three possible SAILS modes of one (SAILSx1), two (SAILSx2), and three (SAILSx3) additional base scans per volume, for SVR, SAILSx2 and SAILSx3 are associated with better warning performance compared to SAILSx1; for FF and TOR, SAILSx3 is associated with better warning performance relative to SAILSx1 and SAILSx2. Two severe storm cases (one that spawned a tornado, one that did not) are presented where SAILS usage helped forecasters make the correct TOR warning decision, lending real-life credence to the statistical results. Furthermore, a statistical analysis of automated volume scan evaluation and termination effects, parsed by SAILS usage and mode, yield a statistically significant association between volume scan update rate and SVR warning lead time.
Summary
The statistical relationship between supplemental adaptive intra-volume low-level scan (SAILS) usage on the Weather Surveillance Radar-1988 Doppler and National Weather Service severe storm warning performance during 2014–20 is analyzed. Results show statistically significant improvement in severe thunderstorm (SVR), flash flood (FF), and tornado (TOR) warning performance associated with SAILS-on versus...
READ MORE
Impact of haptic cues and an active ankle exoskeleton on gait characteristics
July 1, 2022
Journal Article
Published in:
Hum. Factors, Vol. 0, No. 0, July 2022, p. 1-12.
Summary
Objective This study examined the interaction of gait-synchronized vibrotactile cues with an active ankle exoskeleton that provides plantarflexion assistance. Background An exoskeleton that augments gait may support collaboration through feedback to the user about the state of the exoskeleton or characteristics of the task. Methods Participants (N = 16) were provided combinations of torque assistance and vibrotactile cues at pre-specified time points in late swing and early stance while walking on a self-paced treadmill. Participants were either given explicit instructions (N = 8) or were allowed to freely interpret (N=8) how to coordinate with cues. Results For the free interpretation group, the data support an 8% increase in stride length and 14% increase in speed with exoskeleton torque across cue timing, as well as a 5% increase in stride length and 7% increase in speed with only vibrotactile cues. When given explicit instructions, participants modulated speed according to cue timing-increasing speed by 17% at cues in late swing and decreasing speed 11% at cues in early stance compared to no cue when exoskeleton torque was off. When torque was on, participants with explicit instructions had reduced changes in speed. Conclusion These findings support that the presence of torque mitigates how cues were used and highlights the importance of explicit instructions for haptic cuing. Interpreting cues while walking with an exoskeleton may increase cognitive load, influencing overall human-exoskeleton performance for novice users. Application Interactions between haptic feedback and exoskeleton use during gait can inform future feedback designs to support coordination between users and exoskeletons.
Summary
Objective This study examined the interaction of gait-synchronized vibrotactile cues with an active ankle exoskeleton that provides plantarflexion assistance. Background An exoskeleton that augments gait may support collaboration through feedback to the user about the state of the exoskeleton or characteristics of the task. Methods Participants (N = 16) were...
READ MORE
Advances in cross-lingual and cross-source audio-visual speaker recognition: The JHU-MIT system for NIST SRE21
June 28, 2022
Conference Paper
Published in:
Speaker and Language Recognition Workshop, Odyssey 2022, pp. 213-220.
Topic:
R&D area:
Summary
We present a condensed description of the joint effort of JHUCLSP/HLTCOE, MIT-LL and AGH for NIST SRE21. NIST SRE21 consisted of speaker detection over multilingual conversational telephone speech (CTS) and audio from video (AfV). Besides the regular audio track, the evaluation also contains visual (face recognition) and multi-modal tracks. This evaluation exposes new challenges, including cross-source–i.e., CTS vs. AfV– and cross-language trials. Each speaker can speak two or three languages among English, Mandarin and Cantonese. For the audio track, we evaluated embeddings based on Res2Net and ECAPA-TDNN, where the former performed the best. We used PLDA based back-ends trained on previous SRE and VoxCeleb and adapted to a subset of Mandarin/Cantonese speakers. Some novel contributions of this submission are: the use of neural bandwidth extension (BWE) to reduce the mismatch between the AFV and CTS conditions; and invariant representation learning (IRL) to make the embeddings from a given speaker invariant to language. Res2Net with neural BWE was the best monolithic system. We used a pre-trained RetinaFace face detector and ArcFace embeddings for the visual track, following our NIST SRE19 work. We also included a new system using a deep pyramid single shot face detector and face embeddings trained on Crystal loss and probabilistic triplet loss, which performed the best. The number of face embeddings in the test video was reduced by agglomerative clustering or weighting the embedding based on the face detection confidence. Cosine scoring was used to compare embeddings. For the multi-modal track, we just added the calibrated likelihood ratios of the audio and visual conditions, assuming independence between modalities. The multi-modal fusion improved Cprimary by 72% w.r.t. audio.
Summary
We present a condensed description of the joint effort of JHUCLSP/HLTCOE, MIT-LL and AGH for NIST SRE21. NIST SRE21 consisted of speaker detection over multilingual conversational telephone speech (CTS) and audio from video (AfV). Besides the regular audio track, the evaluation also contains visual (face recognition) and multi-modal tracks. This...
READ MORE
Advances in speaker recognition for multilingual conversational telephone speech: the JHU-MIT system for NIST SRE20 CTS challenge
June 27, 2022
Conference Paper
Published in:
Speaker and Language Recognition Workshop, Odyssey 2022, pp. 338-345.
R&D area:
Summary
We present a condensed description of the joint effort of JHUCLSP/HLTCOE and MIT-LL for NIST SRE20. NIST SRE20 CTS consisted of multilingual conversational telephone speech. The set of languages included in the evaluation was not provided, encouraging the participants to develop systems robust to any language. We evaluated x-vector architectures based on ResNet, squeeze-excitation ResNets, Transformers and EfficientNets. Though squeeze-excitation ResNets and EfficientNets provide superior performance in in-domain tasks like VoxCeleb, regular ResNet34 was more robust in the challenge scenario. On the contrary, squeeze-excitation networks over-fitted to the training data, mostly in English. We also proposed a novel PLDA mixture and k-NN PLDA back-ends to handle the multilingual trials. The former clusters the x-vector space expecting that each cluster will correspond to a language family. The latter trains a PLDA model adapted to each enrollment speaker using the nearest speakers–i.e., those with similar language/channel. The k-NN back-end improved Act. Cprimary (Cp) by 68% in SRE16-19 and 22% in SRE20 Progress w.r.t. a single adapted PLDA back-end. Our best single system achieved Act. Cp=0.110 in SRE20 progress. Meanwhile, our best fusion obtained Act. Cp=0.110 in the progress–8% better than single– and Cp=0.087 in the eval set.
Summary
We present a condensed description of the joint effort of JHUCLSP/HLTCOE and MIT-LL for NIST SRE20. NIST SRE20 CTS consisted of multilingual conversational telephone speech. The set of languages included in the evaluation was not provided, encouraging the participants to develop systems robust to any language. We evaluated x-vector architectures...
READ MORE
AAM-Gym: Artificial intelligence testbed for advanced air mobility
June 9, 2022
Journal Article
Published in:
arXiv, 9 June 2022.
Topic:
R&D area:
R&D group:
Summary
We introduce AAM-Gym, a research and development testbed for Advanced Air Mobility (AAM). AAM has the potential to revolutionize travel by reducing ground traffic and emissions by leveraging new types of aircraft such as electric vertical take-off and landing (eVTOL) aircraft and new advanced artificial intelligence (AI) algorithms. Validation of AI algorithms require representative AAM scenarios, as well as a fast time simulation testbed to evaluate their performance. Until now, there has been no such testbed available for AAM to enable a common research platform for individuals in government, industry, or academia. MIT Lincoln Laboratory has developed AAM-Gym to address this gap by providing an ecosystem to develop, train, and validate new and established AI algorithms across a wide variety of AAM use-cases. In this paper, we use AAM-Gym to study the performance of two reinforcement learning algorithms on an AAM use-case, separation assurance in AAM corridors. The performance of the two algorithms is demonstrated based on a series of metrics provided by AAM-Gym, showing the testbed’s utility to AAM research.
Summary
We introduce AAM-Gym, a research and development testbed for Advanced Air Mobility (AAM). AAM has the potential to revolutionize travel by reducing ground traffic and emissions by leveraging new types of aircraft such as electric vertical take-off and landing (eVTOL) aircraft and new advanced artificial intelligence (AI) algorithms. Validation of...
READ MORE
Toward improving EN adoption: Bridging the gap between stated intention and actual use
June 3, 2022
Project Report
Published in:
MIT Lincoln Laboratory Report ACTA-7
Summary
As the COVID-19 pandemic swept the globe in the spring of 2020, technologists looked to enlist technology to assist public health authorities (PHAs) and help stem the tide of infections. As part of this technology push, experts in health care, cryptography, and other related fields developed the Private Automated Contact Tracing (PACT) protocol and related projects to assist the public health objective of slowing the spread of SARS-CoV-2 through digital contact tracing. The joint Google and Apple deployed protocol (Google-Apple Exposure Notifications, also known as GAEN or EN), which became the de facto standard in the U.S., employs the same features as detailed by PACT. The protocol leverages smartphone Bluetooth communications to alert users of potential contact with those carrying the COVID-19 virus in a way that preserves the privacy of both the known-infected individual, and the users receiving the alert. Contact tracing and subsequent personal precautions are more effective at reducing disease spread when more of the population participates, but there are known difficulties with the adoption of novel technology. In order to help the U.S. Centers for Disease Control and Prevention (CDC) and U.S. state-level public health teams address these difficulties, a team of staff from MIT's Lincoln Laboratory (MIT LL) and Computer Science and Artificial Intelligence Laboratory (MIT CSAIL) focused on studying user perception and information needs.
Summary
As the COVID-19 pandemic swept the globe in the spring of 2020, technologists looked to enlist technology to assist public health authorities (PHAs) and help stem the tide of infections. As part of this technology push, experts in health care, cryptography, and other related fields developed the Private Automated Contact...
READ MORE