Publications
Modeling real-world affective and communicative nonverbal vocalizations from minimally speaking individuals
October 1, 2022
Conference Paper
Published in:
IEEE Trans. on Affect. Comput., Vol. 13, No. 4, October 2022, pp. 2238-53.
Topic:
R&D area:
R&D group:
Summary
Nonverbal vocalizations from non- and minimally speaking individuals (mv*) convey important communicative and affective information. While nonverbal vocalizations that occur amidst typical speech and infant vocalizations have been studied extensively in the literature, there is limited prior work on vocalizations by mv* individuals. Our work is among the first studies of the communicative and affective information expressed in nonverbal vocalizations by mv* children and adults. We collected labeled vocalizations in real-world settings with eight mv* communicators, with communicative and affective labels provided in-the-moment by a close family member. Using evaluation strategies suitable for messy, real-world data, we show that nonverbal vocalizations can be classified by function (with 4- and 5-way classifications) with F1 scores above chance for all participants. We analyze labeling and data collection practices for each participating family, and discuss the classification results in the context of our novel real-world data collection protocol. The presented work includes results from the largest classification experiments with nonverbal vocalizations from mv* communicators to date.
Summary
Nonverbal vocalizations from non- and minimally speaking individuals (mv*) convey important communicative and affective information. While nonverbal vocalizations that occur amidst typical speech and infant vocalizations have been studied extensively in the literature, there is limited prior work on vocalizations by mv* individuals. Our work is among the first studies...
READ MORE
Contrast-enhanced ultrasound to detect active bleeding
October 1, 2022
Abstract
Published in:
J. Acoust. Soc. Am. 152, A280 (2022)
Topic:
R&D area:
R&D group:
Summary
Non-compressible internal hemorrhage (NCIH) is the most common cause of death in acute non-penetrating trauma. NCIH management requires accurate hematoma localization and evaluation for ongoing bleeding for risk stratification. The current standard point-of-care diagnostic tool, the focused assessment with sonography for trauma (FAST), detects free fluid in body cavities with conventional B-mode imaging. The FAST does not assess whether bleeding is ongoing, at which location(s), and to what extent. Here, we propose contrast-enhanced ultrasound (CEUS) techniques to better identify, localize, and quantify hemorrhage. We designed and fabricated a custom hemorrhage-mimicking phantom, comprising a perforated vessel and cavity to simulate active bleeding. Lumason contrast agents (UCAs) were introduced at clinically relevant concentrations (3.5×108 bubbles/ml). Conventional and contrast pulse sequence images were captured, and post-processed with bubble localization techniques (SVD clutter filter and bubble localization). The results showed contrast pulse sequences enabled a 2.2-fold increase in the number of microbubbles detected compared with conventional CEUS imaging, over a range of flow rates, concentrations, and localization processing parameters. Additionally, particle velocimetry enabled mapping of dynamic flow within the simulated bleeding site. Our findings indicate that CEUS combined with advanced image processing may enhance visualization of hemodynamics and improve non-invasive, real-time detection of active bleeding.
Summary
Non-compressible internal hemorrhage (NCIH) is the most common cause of death in acute non-penetrating trauma. NCIH management requires accurate hematoma localization and evaluation for ongoing bleeding for risk stratification. The current standard point-of-care diagnostic tool, the focused assessment with sonography for trauma (FAST), detects free fluid in body cavities with...
READ MORE
Multimodal physiological monitoring during virtual reality piloting tasks
August 25, 2022
Journal Article
Published in:
Multimodal Physiological Monitoring During Virtual Reality Piloting Tasks (version 1.0.0). PhysioNet. https://doi.org/10.13026/azwa-ge48.
Summary
This dataset includes multimodal physiologic, flight performance, and user interaction data streams, collected as participants performed virtual flight tasks of varying difficulty. In virtual reality, individuals flew an "Instrument Landing System" (ILS) protocol, in which they had to land an aircraft mostly relying on the cockpit instrument readings. Participants were presented with four levels of difficulty, which were generated by varying wind speed, turbulence, and visibility. Each of the participants performed 12 runs, split into 3 blocks of four consecutive runs, one run at each difficulty, in a single experimental session. The sequence of difficulty levels was presented in a counterbalanced manner across blocks. Flight performance was quantified as a function of horizontal and vertical deviation from an ideal path towards the runway as well as deviation from the prescribed ideal speed of 115 knots. Multimodal physiological signals were aggregated and synchronized using Lab Streaming Layer. Descriptions of data quality are provided to assess each data stream. The starter code provides examples of loading and plotting the time synchronized data streams, extracting sample features from the eye tracking data, and building models to predict pilot performance from the physiology data streams.
Summary
This dataset includes multimodal physiologic, flight performance, and user interaction data streams, collected as participants performed virtual flight tasks of varying difficulty. In virtual reality, individuals flew an "Instrument Landing System" (ILS) protocol, in which they had to land an aircraft mostly relying on the cockpit instrument readings. Participants were...
READ MORE
The tale of discovering a side channel in secure message transmission systems
August 13, 2022
Conference Paper
Published in:
The Conf. for Failed Approaches and Insightful Losses in Cryptology, CFAIL, 13 August 2022.
Topic:
R&D area:
R&D group:
Summary
Secure message transmission (SMT) systems provide information theoretic security for point-to-point message transmission in networks that are partially controlled by an adversary. This is the story of a research project that aimed to implement a flavour of SMT protocols that uses "path hopping" with the goal of quantifying the real-life efficiency of the system, and while failing to achieve this initial goal, let to the discovery a side-channel that affects the security of a wide range of SMT implementations.
Summary
Secure message transmission (SMT) systems provide information theoretic security for point-to-point message transmission in networks that are partially controlled by an adversary. This is the story of a research project that aimed to implement a flavour of SMT protocols that uses "path hopping" with the goal of quantifying the real-life...
READ MORE
Development and validation of the public-facing SimAEN web application
August 1, 2022
Project Report
Published in:
MIT Lincoln Laboratory Report ACTA-6
Topic:
R&D area:
Summary
During a pandemic such as COVID-19, non-pharmaceutical interventions (NPIs) can help protect public health; however, it is not always clear which actions will have the greatest positive impact, or what the trade-offs are between different options. Exposure Notification (EN) was introduced as a prevention measure during the COVID-19 pandemic to supplement traditional contact tracing activities. To predict the estimated impacts of EN, a model for "simulation of automated exposure notification" (SimAEN) was developed by researchers at MIT Lincoln Laboratory (MIT LL) with CDC funding [2]. The model was published through an accessible web interface, available for use by the general public at https://SimAEN.philab.cdc.gov/.
Summary
During a pandemic such as COVID-19, non-pharmaceutical interventions (NPIs) can help protect public health; however, it is not always clear which actions will have the greatest positive impact, or what the trade-offs are between different options. Exposure Notification (EN) was introduced as a prevention measure during the COVID-19 pandemic to...
READ MORE
Feature importance analysis for compensatory reserve to predict hemorrhagic shock
July 15, 2022
Conference Paper
Published in:
44th Annual Int. Conf. of IEEE Engineering in Medicine & Biology Society (EMBC), DOI: 10.1109/EMBC48229.2022.9871661.
Topic:
R&D area:
R&D group:
Summary
Hemorrhage is the leading cause of preventable death from trauma. Traditionally, vital signs have been used to detect blood loss and possible hemorrhagic shock. However, vital signs are not sensitive for early detection because of physiological mechanisms that compensate for blood loss. As an alternative, machine learning algorithms that operate on an arterial blood pressure (ABP) waveform acquired via photoplethysmography have been shown to provide an effective early indicator. However, these machine learning approaches lack physiological interpretability. In this paper, we evaluate the importance of nine ABP-derived features that provide physiological insight, using a database of 40 human subjects from a lower-body negative pressure model of progressive central hypovolemia. One feature was found to be considerably more important than any other. That feature, the half-rise to dicrotic notch (HRDN), measures an approximate time delay between the ABP ejected and reflected wave components. This delay is an indication of compensatory mechanisms such as reduced arterial compliance and vasoconstriction. For a scale of 0% to 100%, with 100% representing normovolemia and 0% representing decompensation, linear regression of the HRDN feature results in root-mean-squared error of 16.9%, R2 of 0.72, and an area under the receiver operating curve for detecting decompensation of 0.88. These results are comparable to previously reported results from the more complex black box machine learning models. Clinical Relevance- A single physiologically interpretable feature measured from an arterial blood pressure waveform is shown to be effective in monitoring for blood loss and impending hemorrhagic shock based on data from a human lower-body negative pressure model of progressive central hypolemia.
Summary
Hemorrhage is the leading cause of preventable death from trauma. Traditionally, vital signs have been used to detect blood loss and possible hemorrhagic shock. However, vital signs are not sensitive for early detection because of physiological mechanisms that compensate for blood loss. As an alternative, machine learning algorithms that operate...
READ MORE
Transfer learning for automated COVID-19 B-line classification in lung ultrasound
July 15, 2022
Conference Paper
Published in:
44th Annual Int. Conf. of IEEE Engineering in Medicine & Biology Society (EMBC), DOI: 10.1109/EMBC48229.2022.9871894.
Topic:
R&D area:
R&D group:
Summary
Lung ultrasound (LUS) as a diagnostic tool is gaining support for its role in the diagnosis and management of COVID-19 and a number of other lung pathologies. B-lines are a predominant feature in COVID-19, however LUS requires a skilled clinician to interpret findings. To facilitate the interpretation, our main objective was to develop automated methods to classify B-lines as pathologic vs. normal. We developed transfer learning models based on ResNet networks to classify B-lines as pathologic (at least 3 B-lines per lung field) vs. normal using COVID-19 LUS data. Assessment of B-line severity on a 0-4 multi-class scale was also explored. For binary B-line classification, at the frame-level, all ResNet models pretrained with ImageNet yielded higher performance than the baseline nonpretrained ResNet-18. Pretrained ResNet-18 has the best Equal Error Rate (EER) of 9.1% vs the baseline of 11.9%. At the clip-level, all pretrained network models resulted in better Cohen's kappa agreement (linear-weighted) and clip score accuracy, with the pretrained ResNet-18 having the best Cohen's kappa of 0.815 [95% CI: 0.804-0.826], and ResNet-101 the best clip scoring accuracy of 93.6%. Similar results were shown for multi-class scoring, where pretrained network models outperformed the baseline model. A class activation map is also presented to guide clinicians in interpreting LUS findings. Future work aims to further improve the multi-class assessment for severity of B-lines with a more diverse LUS dataset.
Summary
Lung ultrasound (LUS) as a diagnostic tool is gaining support for its role in the diagnosis and management of COVID-19 and a number of other lung pathologies. B-lines are a predominant feature in COVID-19, however LUS requires a skilled clinician to interpret findings. To facilitate the interpretation, our main objective...
READ MORE
Self-supervised contrastive pre-training for time series via time-frequency consistency
July 11, 2022
Journal Article
Published in:
arXiv, June 16, 2022.
Topic:
R&D area:
Summary
Pre-training on time series poses a unique challenge due to the potential mismatch between pre-training and target domains, such as shifts in temporal dynamics, fast-evolving trends, and long-range and short cyclic effects, which can lead to poor downstream performance. While domain adaptation methods can mitigate these shifts, most methods need examples directly from the target domain, making them suboptimal for pre-training. To address this challenge, methods need to accommodate target domains with different temporal dynamics and be capable of doing so without seeing any target examples during pre-training. Relative to other modalities, in time series, we expect that time-based and frequency-based representations of the same example are located close together in the time-frequency space. To this end, we posit that time-frequency consistency (TF-C) — embedding a time-based neighborhood of a particular example close to its frequency-based neighborhood and back—is desirable for pre-training. Motivated by TF-C, we define a decomposable pre-training model, where the self-supervised signal is provided by the distance between time and frequency components, each individually trained by contrastive estimation. We evaluate the new method on eight datasets, including electrodiagnostic testing, human activity recognition, mechanical fault detection, and physical status monitoring. Experiments against eight state-of-the-art methods show that TF-C outperforms baselines by 15.4% (F1 score) on average in one-to-one settings (e.g., fine-tuning an EEG-pretrained model on EMG data) and by up to 8.4% (F1 score) in challenging one-to-many settings (e.g., fine-tuning an EEG-pretrained model for either hand-gesture recognition or mechanical fault prediction), reflecting the breadth of scenarios that arise in real-world applications. The source code and datasets are available at https://anonymous.4open.science/r/TFC-pretraining-6B07.
Summary
Pre-training on time series poses a unique challenge due to the potential mismatch between pre-training and target domains, such as shifts in temporal dynamics, fast-evolving trends, and long-range and short cyclic effects, which can lead to poor downstream performance. While domain adaptation methods can mitigate these shifts, most methods need...
READ MORE
Axon tracing and centerline detection using topologically-aware 3D U-nets
July 11, 2022
Journal Article
Published in:
2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2022, pp. 238-242
Summary
As advances in microscopy imaging provide an ever clearer window into the human brain, accurate reconstruction of neural connectivity can yield valuable insight into the relationship between brain structure and function. However, human manual tracing is a slow and laborious task, and requires domain expertise. Automated methods are thus needed to enable rapid and accurate analysis at scale. In this paper, we explored deep neural networks for dense axon tracing and incorporated axon topological information into the loss function with a goal to improve the performance on both voxel-based segmentation and axon centerline detection. We evaluated three approaches using a modified 3D U-Net architecture trained on a mouse brain dataset imaged with light sheet microscopy and achieved a 10% increase in axon tracing accuracy over previous methods. Furthermore, the addition of centerline awareness in the loss function outperformed the baseline approach across all metrics, including a boost in Rand Index by 8%.
Summary
As advances in microscopy imaging provide an ever clearer window into the human brain, accurate reconstruction of neural connectivity can yield valuable insight into the relationship between brain structure and function. However, human manual tracing is a slow and laborious task, and requires domain expertise. Automated methods are thus needed...
READ MORE
Impact of haptic cues and an active ankle exoskeleton on gait characteristics
July 1, 2022
Journal Article
Published in:
Hum. Factors, Vol. 0, No. 0, July 2022, p. 1-12.
Summary
Objective This study examined the interaction of gait-synchronized vibrotactile cues with an active ankle exoskeleton that provides plantarflexion assistance. Background An exoskeleton that augments gait may support collaboration through feedback to the user about the state of the exoskeleton or characteristics of the task. Methods Participants (N = 16) were provided combinations of torque assistance and vibrotactile cues at pre-specified time points in late swing and early stance while walking on a self-paced treadmill. Participants were either given explicit instructions (N = 8) or were allowed to freely interpret (N=8) how to coordinate with cues. Results For the free interpretation group, the data support an 8% increase in stride length and 14% increase in speed with exoskeleton torque across cue timing, as well as a 5% increase in stride length and 7% increase in speed with only vibrotactile cues. When given explicit instructions, participants modulated speed according to cue timing-increasing speed by 17% at cues in late swing and decreasing speed 11% at cues in early stance compared to no cue when exoskeleton torque was off. When torque was on, participants with explicit instructions had reduced changes in speed. Conclusion These findings support that the presence of torque mitigates how cues were used and highlights the importance of explicit instructions for haptic cuing. Interpreting cues while walking with an exoskeleton may increase cognitive load, influencing overall human-exoskeleton performance for novice users. Application Interactions between haptic feedback and exoskeleton use during gait can inform future feedback designs to support coordination between users and exoskeletons.
Summary
Objective This study examined the interaction of gait-synchronized vibrotactile cues with an active ankle exoskeleton that provides plantarflexion assistance. Background An exoskeleton that augments gait may support collaboration through feedback to the user about the state of the exoskeleton or characteristics of the task. Methods Participants (N = 16) were...
READ MORE