Publications
Variability of speech timing features across repeated recordings: a comparison of open-source extraction techniques
September 1, 2024
Conference Paper
Published in:
Proc. Interspeech 2024, 2015-2019, doi: 10.21437/Interspeech.2024-1074
Topic:
R&D area:
R&D group:
Summary
Variations in speech timing features have been reliably linked to symptoms of various health conditions, demonstrating clinical potential. However, replication challenges hinder their
translation; extracted speech features are susceptible to methodological variations in the recording and processing pipeline. Investigating this, we compared exemplar timing features extracted via three different techniques from recordings of healthy speech. Our results show that features extracted via an intensity-based method differ from those produced by forced alignment. Different extraction methods also led to differing estimates of within-speaker feature variability over time in an analysis of recordings repeated systematically over three sessions in one day (n=26) and in one week (n=28). Our findings highlight the importance of feature extraction in study design and interpretation, and the need for consistent, accurate extraction techniques for clinical research.
translation; extracted speech features are susceptible to methodological variations in the recording and processing pipeline. Investigating this, we compared exemplar timing features extracted via three different techniques from recordings of healthy speech. Our results show that features extracted via an intensity-based method differ from those produced by forced alignment. Different extraction methods also led to differing estimates of within-speaker feature variability over time in an analysis of recordings repeated systematically over three sessions in one day (n=26) and in one week (n=28). Our findings highlight the importance of feature extraction in study design and interpretation, and the need for consistent, accurate extraction techniques for clinical research.
Summary
Variations in speech timing features have been reliably linked to symptoms of various health conditions, demonstrating clinical potential. However, replication challenges hinder their
translation; extracted speech features are susceptible to methodological variations in the recording and processing pipeline. Investigating this, we compared exemplar timing features extracted via three different techniques...
READ MORE
Building digital twins for cardiovascular health: From principles to clinical impact
August 1, 2024
Journal Article
Published in:
J. Am. Heart Assoc., online ahead of print, 1 August 2024.
Topic:
R&D area:
R&D group:
Summary
The past several decades have seen rapid advances in diagnosis and treatment of cardiovascular diseases and stroke, enabled by technological breakthroughs in imaging, genomics, and physiological monitoring, coupled with therapeutic interventions. We now face the challenge of how to (1) rapidly process large, complex multimodal and multiscale medical measurements; (2) map all available data streams to the trajectories of disease states over the patient's lifetime; and (3) apply this information for optimal clinical interventions and outcomes. Here we review new advances that may address these challenges using digital twin technology to fulfill the promise of personalized cardiovascular medical practice. Rooted in engineering mechanics and manufacturing, the digital twin is a virtual representation engineered to model and simulate its physical counterpart. Recent breakthroughs in scientific computation, artificial intelligence, and sensor technology have enabled rapid bidirectional interactions between the virtual-physical counterparts with measurements of the physical twin that inform and improve its virtual twin, which in turn provide updated virtual projections of disease trajectories and anticipated clinical outcomes. Verification, validation, and uncertainty quantification builds confidence and trust by clinicians and patients in the digital twin and establishes boundaries for the use of simulations in cardiovascular medicine. Mechanistic physiological models form the fundamental building blocks of the personalized digital twin that continuously forecast optimal management of cardiovascular health using individualized data streams. We present exemplars from the existing body of literature pertaining to mechanistic model development for cardiovascular dynamics and summarize existing technical challenges and opportunities pertaining to the foundation of a digital twin.
Summary
The past several decades have seen rapid advances in diagnosis and treatment of cardiovascular diseases and stroke, enabled by technological breakthroughs in imaging, genomics, and physiological monitoring, coupled with therapeutic interventions. We now face the challenge of how to (1) rapidly process large, complex multimodal and multiscale medical measurements; (2)...
READ MORE
Microbubble contrast agents improve detection of active hemorrhage
June 18, 2024
Journal Article
Published in:
IEEE Open Journal of Engineering in Medicine and Biology, doi: 10.1109/OJEMB.2024.3414974
Topic:
R&D area:
R&D group:
Summary
Assessment of trauma-induced hemorrhage with ultrasound is particularly challenging outside of the clinic, where its detection is crucial. The current clinical standard for hematoma detection – the focused assessment with sonography of trauma (FAST) exam – does not aim to detect ongoing blood loss, and thus is unable to detect injuries of increasing severity. To enhance detection of active bleeding, we propose the use of ultrasound contrast agents (UCAs), together with a novel flow phantom and contrast-sensitive processing techniques, to facilitate efficient, practical characterization of internal bleeding. Within a the custom phantom, UCAs and processing techniques enabled a significant enhancement of the hemorrhage visualization (mean increase in generalized contrast-to-noise ratio of 17 %) compared to the contrast-free case over a range of flow rates up to 40 ml/min. Moreover, we have shown that the use of UCAs improves the probability of detection: the area under the receiver operating characteristic curve for a flow rate of 40 ml/min was 0.99, compared to 0.72 without contrast. We also demonstrate how additional processing of the spatial and temporal information further localizes the bleeding site. UCAs also enhanced Doppler signals over the non-contrast case. These results show that specialized nonlinear processing (NLP) pipelines together with UCAs may offer an efficient means to improve substantially the detection of slower hemorrhages and increase survival rates for trauma-induced injury in pre-hospital settings.
Summary
Assessment of trauma-induced hemorrhage with ultrasound is particularly challenging outside of the clinic, where its detection is crucial. The current clinical standard for hematoma detection – the focused assessment with sonography of trauma (FAST) exam – does not aim to detect ongoing blood loss, and thus is unable to detect...
READ MORE
An exploratory characterization of speech- and fine-motor coordination in verbal children with Autism spectrum disorder
May 22, 2024
Journal Article
Published in:
Comput. Speech Lang., Vol. 89, 2025, 101665.
Topic:
R&D area:
R&D group:
Summary
Autism spectrum disorder (ASD) is a neurodevelopmental disorder often associated with difficulties in speech production and fine-motor tasks. Thus, there is a need to develop objective measures to assess and understand speech production and other fine-motor challenges in individuals with ASD. In addition, recent research suggests that difficulties with speech production and fine-motor tasks may contribute to language difficulties in ASD. In this paper, we explore the utility of an off-body recording platform, from which we administer a speech- and fine-motor protocol to verbal children with ASD and neurotypical controls. We utilize a correlation-based analysis technique to develop proxy measures of motor coordination from signals derived from recordings of speech- and fine-motor behaviors. Eigenvalues of the resulting correlation matrix are inputs to Gaussian Mixture Models to discriminate between highly-verbal children with ASD and neurotypical controls. These eigenvalues also characterize the complexity (underlying dimensionality) of representative signals of speech- and fine-motor movement dynamics, and form the feature basis to estimate scores on an expressive vocabulary measure. Based on a pilot dataset (15 ASD and 15 controls), features derived from an oral story reading task are used in discriminating between the two groups with AUCs > 0.80, and highlight lower complexity of coordination in children with ASD. Features derived from handwriting and maze tracing tasks led to AUCs of 0.86 and 0.91, however features derived from ocular tasks did not aid in discrimination between the ASD and neurotypical groups. In addition, features derived from free speech and sustained vowel tasks are strongly correlated with expressive vocabulary scores. These results indicate the promise of a correlation-based analysis in elucidating motor differences between individuals with ASD and neurotypical controls.
Summary
Autism spectrum disorder (ASD) is a neurodevelopmental disorder often associated with difficulties in speech production and fine-motor tasks. Thus, there is a need to develop objective measures to assess and understand speech production and other fine-motor challenges in individuals with ASD. In addition, recent research suggests that difficulties with speech...
READ MORE
A neurophysiological-auditory "listen receipt" for communication enhancement
April 14, 2024
Conference Paper
Published in:
49th IEEE Intl. Conf. on Acoustics, Speech, and Signal Processing, ICASSP, 14-19 April 2024.
Topic:
R&D area:
R&D group:
Summary
Information overload, and specifically auditory overload, is common in critical situations and detrimental to communication. Currently, there is no auditory equivalent of an email read receipt to know if a person has heard a message, other than waiting for a reply. This work hypothesizes that it may be possible to decode whether a person has indeed heard a message, or in other words, create an an auditory "listen receipt," through use of non-invasive physiological or neural monitoring. We extracted a variety of features derived from Electrodermal activity (EDA), Electroencephalography (EEG), and the correlations between the acoustic envelope of the radio message and EEG to use in the decoder. We were able to classify the cases in which the subject responded correctly to the question in the message, versus the cases where they missed or heard the message incorrectly, with an accuracy of 79% and a receiver operating characteristic (ROC) area under the curve (AUC) of 0.83. This work suggests that the concept of a "listen receipt" may be possible, and future wearable machine-brain interface technologies may be able to automatically determine if an important radio message has been missed for both human-to-human and human-to-machine communication.
Summary
Information overload, and specifically auditory overload, is common in critical situations and detrimental to communication. Currently, there is no auditory equivalent of an email read receipt to know if a person has heard a message, other than waiting for a reply. This work hypothesizes that it may be possible to...
READ MORE
Quantifying speech production coordination from non- and minimally-speaking individuals
April 13, 2024
Journal Article
Published in:
J. Autism Dev. Disord., 13 April 2024.
Topic:
R&D area:
R&D group:
Summary
Purpose: Non-verbal utterances are an important tool of communication for individuals who are non- or minimally-speaking. While these utterances are typically understood by caregivers, they can be challenging to interpret by their larger community. To date, there has been little work done to detect and characterize the vocalizations produced by non- or minimally-speaking individuals. This paper aims to characterize five categories of utterances across a set of 7 non- or minimally-speaking individuals. Methods: The characterization is accomplished using a correlation structure methodology, acting as a proxy measurement for motor coordination, to localize similarities and differences to specific speech production systems. Results: We specifically find that frustrated and dysregulated utterances show similar correlation structure outputs, especially when compared to self-talk, request, and delighted utterances. We additionally witness higher complexity of coordination between articulatory and respiratory subsystems and lower complexity of coordination between laryngeal and respiratory subsystems in frustration and dysregulation as compared to self-talk, request, and delight. Finally, we observe lower complexity of coordination across all three speech subsystems in the request utterances as compared to self-talk and delight. Conclusion: The insights from this work aid in understanding of the modifications made by non- or minimally-speaking individuals to accomplish specific goals in non-verbal communication.
Summary
Purpose: Non-verbal utterances are an important tool of communication for individuals who are non- or minimally-speaking. While these utterances are typically understood by caregivers, they can be challenging to interpret by their larger community. To date, there has been little work done to detect and characterize the vocalizations produced by...
READ MORE
A vocal model to predict readiness under sleep deprivation
October 9, 2023
Conference Paper
Published in:
Proc. 2023 IEEE 19th Intl. Conf. on Body Sensor Networks, BSN, 9-11 October 2023.
Topic:
R&D area:
R&D group:
Summary
A variety of factors can affect cognitive readiness and influence human performance in tasks that are mission critical. Sleep deprivation is one of the most prevalent factors that degrade performance. One risk-mitigation approach is to use vocal biomarkers to detect cognitive fatigue and resulting performance decrements. In this study, a group of 20 subjects were deprived of sleep for a period of 24 hours. Every two hours, they performed a battery of both speech tasks and cognitive performance tasks, including the psychomotor vigilance test (PVT). Performance on the PVT declined dramatically during nighttime hours between 2 AM and 8 AM. We demonstrate that a model using vocal biomarkers from read speech and free speech can be successfully trained to detect performance decrements on the PVT. We also demonstrate that the vocal model successfully generalizes to other outcomes at a similar level as PVT, detecting sleep deprivation (AUC=0.79) and cognitive performance declines on a battery of cognitive tasks (AUC=0.79). In comparison, using PVT as the basis for detecting sleep deprivation and performance declines resulted in AUC=0.75 and AUC=0.80, respectively.
Summary
A variety of factors can affect cognitive readiness and influence human performance in tasks that are mission critical. Sleep deprivation is one of the most prevalent factors that degrade performance. One risk-mitigation approach is to use vocal biomarkers to detect cognitive fatigue and resulting performance decrements. In this study, a...
READ MORE
Optimizing MobileNet algorithms for real-time vessel detection on smartphones
October 9, 2023
Conference Paper
Published in:
Proc. 2023 IEEE 19th Intl. Conf. on Body Sensor Networks, BSN, 9-11 October 2023.
Topic:
R&D area:
R&D group:
Summary
Internal bleeding due to non-compressible torso hemorrhage is the leading cause of prehospital fatalities in civilian and military trauma. A limited number of trauma surgeons are expected to be available in disaster scenarios and future large-scale combat operations. As a result, non-specialists will need to perform life-saving interventions to address internal bleeding. A first step in mitigation is ultrasound-guided central vascular access, which involves identifying a deep blood vessel in the imagery, such as the femoral vein, femoral artery, or internal jugular vein, and then placing a needle and catheter into the vessel for follow-on resuscitation. In this paper, we demonstrate machine learning algorithms for both femoral and neck vessel detection with high accuracy and real-time speed on smartphones. The algorithms are integrated with commercial ultrasound and optimized for use on low size, weight, and power devices. Coupled with custom robotics, this technology can enable rapid vascular access by non-specialist operators using a handheld platform.
Summary
Internal bleeding due to non-compressible torso hemorrhage is the leading cause of prehospital fatalities in civilian and military trauma. A limited number of trauma surgeons are expected to be available in disaster scenarios and future large-scale combat operations. As a result, non-specialists will need to perform life-saving interventions to address...
READ MORE
Daily activity profiles and activity fluctuations correlate with BMI
October 9, 2023
Conference Paper
Published in:
2023 IEEE 19th Intl. Conf. on Body Sensor Networks, BSN, 9-11 October 2023.
Topic:
R&D area:
R&D group:
Summary
The rising levels of obesity have been declared a global epidemic by the World Health Organization, with obesity rates surpassing 50% in many countries. Between the late 1970s and the early 2000s in the U.S., the prevalence of obesity doubled while the prevalence of severe obesity more than tripled. One of the factors underlying the obesity epidemic is secular changes in activity patterns due to an increasingly sedentary lifestyle. A better understanding is needed of how daily activity patterns relate to obesity. In this study we use wrist-worn accelerometry from the National Health And Nutrition Examination Survey (NHANES) data set to develop a number of features that characterize daily activity profiles, as well as fluctuations in those profiles over time, and determine how those features correlate with body mass index (BMI). Using a data set of 2,882 subjects split evenly between a training and test fold, we constructed regression models that estimate BMI based on activity profiles and fluctuations. We found a correlation of r=0.47 between estimated and true BMI, resulting in detection of overweight, obese, and severely obese subjects with area under the ROC curve (AUC) of 0.69, 0.73 and 0.85. These results indicate how patterns of activity levels across daily sleep/wake cycles are associated with higher risk for obesity.
Summary
The rising levels of obesity have been declared a global epidemic by the World Health Organization, with obesity rates surpassing 50% in many countries. Between the late 1970s and the early 2000s in the U.S., the prevalence of obesity doubled while the prevalence of severe obesity more than tripled. One...
READ MORE
Towards robust paralinguistic assessment for real-world mobile health (mHealth) monitoring: an initial study of reverberation effects on speech
August 20, 2023
Conference Paper
Published in:
Proc. Annual Conf. Intl. Speech Communication Assoc., INTERSPEECH 2023, 20-24 August 2023, pp. 2373-77.
Topic:
R&D area:
R&D group:
Summary
Speech is promising as an objective, convenient tool to monitor health remotely over time using mobile devices. Numerous paralinguistic features have been demonstrated to contain salient information related to an individual's health. However, mobile device specification and acoustic environments vary widely, risking the reliability of the extracted features. In an initial step towards quantifying these effects, we report the variability of 13 exemplar paralinguistic features commonly reported in the speech-health literature and extracted from the speech of 42 healthy volunteers recorded consecutively in rooms with low and high reverberation with one budget and two higher-end smartphones, and a condenser microphone. Our results show reverberation has a clear effect on several features, in particular voice quality markers. They point to new research directions investigating how best to record and process in-the-wild speech for reliable longitudinal health state assessment.
Summary
Speech is promising as an objective, convenient tool to monitor health remotely over time using mobile devices. Numerous paralinguistic features have been demonstrated to contain salient information related to an individual's health. However, mobile device specification and acoustic environments vary widely, risking the reliability of the extracted features. In an...
READ MORE