Publications
A vocal model to predict readiness under sleep deprivation
October 9, 2023
Conference Paper
Published in:
Proc. 2023 IEEE 19th Intl. Conf. on Body Sensor Networks, BSN, 9-11 October 2023.
Topic:
R&D area:
R&D group:
Summary
A variety of factors can affect cognitive readiness and influence human performance in tasks that are mission critical. Sleep deprivation is one of the most prevalent factors that degrade performance. One risk-mitigation approach is to use vocal biomarkers to detect cognitive fatigue and resulting performance decrements. In this study, a group of 20 subjects were deprived of sleep for a period of 24 hours. Every two hours, they performed a battery of both speech tasks and cognitive performance tasks, including the psychomotor vigilance test (PVT). Performance on the PVT declined dramatically during nighttime hours between 2 AM and 8 AM. We demonstrate that a model using vocal biomarkers from read speech and free speech can be successfully trained to detect performance decrements on the PVT. We also demonstrate that the vocal model successfully generalizes to other outcomes at a similar level as PVT, detecting sleep deprivation (AUC=0.79) and cognitive performance declines on a battery of cognitive tasks (AUC=0.79). In comparison, using PVT as the basis for detecting sleep deprivation and performance declines resulted in AUC=0.75 and AUC=0.80, respectively.
Summary
A variety of factors can affect cognitive readiness and influence human performance in tasks that are mission critical. Sleep deprivation is one of the most prevalent factors that degrade performance. One risk-mitigation approach is to use vocal biomarkers to detect cognitive fatigue and resulting performance decrements. In this study, a...
READ MORE
Daily activity profiles and activity fluctuations correlate with BMI
October 9, 2023
Conference Paper
Published in:
2023 IEEE 19th Intl. Conf. on Body Sensor Networks, BSN, 9-11 October 2023.
Topic:
R&D area:
R&D group:
Summary
The rising levels of obesity have been declared a global epidemic by the World Health Organization, with obesity rates surpassing 50% in many countries. Between the late 1970s and the early 2000s in the U.S., the prevalence of obesity doubled while the prevalence of severe obesity more than tripled. One of the factors underlying the obesity epidemic is secular changes in activity patterns due to an increasingly sedentary lifestyle. A better understanding is needed of how daily activity patterns relate to obesity. In this study we use wrist-worn accelerometry from the National Health And Nutrition Examination Survey (NHANES) data set to develop a number of features that characterize daily activity profiles, as well as fluctuations in those profiles over time, and determine how those features correlate with body mass index (BMI). Using a data set of 2,882 subjects split evenly between a training and test fold, we constructed regression models that estimate BMI based on activity profiles and fluctuations. We found a correlation of r=0.47 between estimated and true BMI, resulting in detection of overweight, obese, and severely obese subjects with area under the ROC curve (AUC) of 0.69, 0.73 and 0.85. These results indicate how patterns of activity levels across daily sleep/wake cycles are associated with higher risk for obesity.
Summary
The rising levels of obesity have been declared a global epidemic by the World Health Organization, with obesity rates surpassing 50% in many countries. Between the late 1970s and the early 2000s in the U.S., the prevalence of obesity doubled while the prevalence of severe obesity more than tripled. One...
READ MORE
Gait instability and estimated core temperature predict exertional heat stroke
January 12, 2022
Journal Article
Published in:
Br J Sports Med., online ahead of print, 12 January 2022.
Topic:
R&D area:
R&D group:
Summary
Objective Exertional heat stroke (EHS), characterised by a high core body temperature (Tcr) and central nervous system (CNS) dysfunction, is a concern for athletes, workers and military personnel who must train and perform in hot environments. The objective of this study was to determine whether algorithms that estimate Tcr from heart rate and gait instability from a trunk-worn sensor system can forward predict EHS onset. Methods Heart rate and three-axis accelerometry data were collected from chest-worn sensors from 1806 US military personnel participating in timed 4/5-mile runs, and loaded marches of 7 and 12 miles; in total, 3422 high EHS-risk training datasets were available for analysis. Six soldiers were diagnosed with heat stroke and all had rectal temperatures of >41°C when first measured and were exhibiting CNS dysfunction. Estimated core temperature (ECTemp) was computed from sequential measures of heart rate. Gait instability was computed from three-axis accelerometry using features of pattern dispersion and autocorrelation. Results The six soldiers who experienced heat stroke were among the hottest compared with the other soldiers in the respective training events with ECTemps ranging from 39.2°C to 40.8°C. Combining ECTemp and gait instability measures successfully identified all six EHS casualties at least 3.5 min in advance of collapse while falsely identifying 6.1% (209 total false positives) examples where exertional heat illness symptoms were neither observed nor reported. No false-negative cases were noted. Conclusion The combination of two algorithms that estimate Tcr and ataxic gate appears promising for real-time alerting of impending EHS.
Summary
Objective Exertional heat stroke (EHS), characterised by a high core body temperature (Tcr) and central nervous system (CNS) dysfunction, is a concern for athletes, workers and military personnel who must train and perform in hot environments. The objective of this study was to determine whether algorithms that estimate Tcr from...
READ MORE
Detecting Parkinson's disease from wrist-worn accelerometry in the U.K. Biobank
March 14, 2021
Journal Article
Published in:
Sensors, Vol. 21, No. 6, 2021, Art. No. 2047.
Topic:
R&D area:
R&D group:
Summary
Parkinson's disease (PD) is a chronic movement disorder that produces a variety of characteristic movement abnormalities. The ubiquity of wrist-worn accelerometry suggests a possible sensor modality for early detection of PD symptoms and subsequent tracking of PD symptom severity. As an initial proof of concept for this technological approach, we analyzed the U.K. Biobank data set, consisting of one week of wrist-worn accelerometry from a population with a PD primary diagnosis and an age-matched healthy control population. Measures of movement dispersion were extracted from automatically segmented gait data, and measures of movement dimensionality were extracted from automatically segmented low-movement data. Using machine learning classifiers applied to one week of data, PD was detected with an area under the curve (AUC) of 0.69 on gait data, AUC = 0.84 on low-movement data, and AUC = 0.85 on a fusion of both activities. It was also found that classification accuracy steadily improved across the one-week data collection, suggesting that higher accuracy could be achievable from a longer data collection. These results suggest the viability of using a low-cost and easy-to-use activity sensor for detecting movement abnormalities due to PD and motivate further research on early PD detection and tracking of PD symptom severity.
Summary
Parkinson's disease (PD) is a chronic movement disorder that produces a variety of characteristic movement abnormalities. The ubiquity of wrist-worn accelerometry suggests a possible sensor modality for early detection of PD symptoms and subsequent tracking of PD symptom severity. As an initial proof of concept for this technological approach, we...
READ MORE
Using oculomotor features to predict changes in optic nerve sheath diameter and ImPACT scores from contact-sport athletes
March 4, 2021
Journal Article
Published in:
Front. Neurol., 4 March 2021, Art. No. 584684.
Topic:
R&D area:
Summary
There is mounting evidence linking the cumulative effects of repetitive head impacts to neuro-degenerative conditions. Robust clinical assessment tools to identify mild traumatic brain injuries are needed to assist with timely diagnosis for return-to-field decisions and appropriately guide rehabilitation. The focus of the present study is to investigate the potential for oculomotor features to complement existing diagnostic tools, such as measurements of Optic Nerve Sheath Diameter (ONSD) and Immediate Post-concussion Assessment and Cognitive Testing (ImPACT). Thirty-one high school American football and soccer athletes were tracked through the course of a sports season. Given the high risk of repetitive head impacts associated with both soccer and football, our hypotheses were that (1) ONSD and ImPACT scores would worsen through the season and (2) oculomotor features would effectively capture both neurophysiological changes reflected by ONSD and neuro-functional status assessed via ImPACT. Oculomotor features were used as input to Linear Mixed-Effects Regression models to predict ONSD and ImPACT scores as outcomes. Prediction accuracy was evaluated to identify explicit relationships between eye movements, ONSD, and ImPACT scores. Significant Pearson correlations were observed between predicted and actual outcomes for ONSD (Raw = 0.70; Normalized = 0.45) and for ImPACT (Raw = 0.86; Normalized = 0.71), demonstrating the capability of oculomotor features to capture neurological changes detected by both ONSD and ImPACT. The most predictive features were found to relate to motor control and visual-motor processing. In future work, oculomotor models, linking neural structures to oculomotor function, can be built to gain extended mechanistic insights into neurophysiological changes observed through seasons of participation in contact sports.
Summary
There is mounting evidence linking the cumulative effects of repetitive head impacts to neuro-degenerative conditions. Robust clinical assessment tools to identify mild traumatic brain injuries are needed to assist with timely diagnosis for return-to-field decisions and appropriately guide rehabilitation. The focus of the present study is to investigate the potential...
READ MORE
Estimating sedentary breathing rate from chest-worn accelerometry from free-living data
July 20, 2020
Conference Paper
Published in:
42nd Annual Intl. Conf. IEEE Engineering in Medicine and Biology Society, EMBC, 20-24 July 2020.
Topic:
R&D area:
R&D group:
Summary
Breathing rate was estimated from chest-worn accelerometry collected from 1,522 servicemembers during training by a wearable physiological monitor. A total of 29,189 hours of training and sleep data were analyzed. The primary purpose of the monitor was to assess thermal-work strain and avoid heat injuries. The monitor design was thus not optimized to estimate breathing rate. Since breathing rate cannot be accurately estimated during periods of high activity, a qualifier was applied to identify sedentary time periods, totaling 8,867 hours. Breathing rate was estimated for a total of 4,179 hours, or 14% of the total collection and 47% of the sedentary total, primarily during periods of sleep. The breathing rate estimation method was compared to an FDA 510(K)-cleared criterion breathing rate sensor (Zephyr, Annapolis MD, USA) in a controlled laboratory experiment, which showed good agreement between the two techniques. Contributions of this paper are to: 1) provide the first analysis of accelerometry-derived breathing rate on free-living data including periods of high activity as well as sleep, along with a qualifier that effectively identifies sedentary periods appropriate for estimating breathing rate; 2) test breathing rate estimation on a data set with a total duration that is more than 60 times longer than that of the largest previously reported study, 3) test breathing rate estimation on data from a physiological monitor that has not been expressly designed for that purpose.
Summary
Breathing rate was estimated from chest-worn accelerometry collected from 1,522 servicemembers during training by a wearable physiological monitor. A total of 29,189 hours of training and sleep data were analyzed. The primary purpose of the monitor was to assess thermal-work strain and avoid heat injuries. The monitor design was thus...
READ MORE
Investigation of the relationship of vocal, eye-tracking, and fMRI ROI time-series measures with preclinical mild traumatic brain injury
October 19, 2019
Abstract
Published in:
Neuroscience 2019
Summary
In this work, we are examining correlations between vocal articulatory features, ocular smooth pursuit measures, and features from the fMRI BOLD response in regions of interest (ROI) time series in a high school athlete population susceptible to repeated head impact within a sports season. Initial results have indicated relationships between vocal features and brain ROIs that may show which components of the neural speech networks effected are effected by preclinical mild traumatic brain injury (mTBI). The data used for this study was collected by Purdue University on 32 high school athletes over the entirety of a sports season (Helfer, et al., 2014), and includes fMRI measurements made pre-season, in-season, and postseason. The athletes are 25 male football players and 7 female soccer players. The Immediate Post-Concussion Assessment and Cognitive Testing suite (ImPACT) was used as a means of assessing cognitive performance (Broglio, Ferrara, Macciocchi, Baumgartner, & Elliott, 2007). The test is made up of six sections, which measure verbal memory, visual memory, visual motor speed, reaction time, impulse control, and a total symptom composite. Using each test, a threshold is set for a change in cognitive performance. The threshold for each test is defined as a decline from baseline that exceeds one standard deviation, where the standard deviation is computed over the change from baseline across all subjects’ test scores. Speech features were extracted from audio recordings of the Grandfather Passage, which provides a standardized and phonetically balanced sample of speech. Oculomotor testing included two experimental conditions. In the smooth pursuit condition, a single target moving circularly, at constant speed. In the saccade condition, a target was jumped between one of three location along the horizontal midline of the screen. In both trial types, subjects visually tracked the targets during the trials, which lasted for one minute. The fMRI features are derived from the bold time-series data from resting state fMRI scans of the subjects. The pre-processing of the resting state fMRI and accompanying structural MRI data (for Atlas registration) was performed with the toolkit CONN (Whitfield-Gabrieli & Nieto-Castanon, 2012). Functional connectivity was generated using cortical and sub-cortical atlas registrations. This investigation will explores correlations between these three modalities and a cognitive performance assessment.
Summary
In this work, we are examining correlations between vocal articulatory features, ocular smooth pursuit measures, and features from the fMRI BOLD response in regions of interest (ROI) time series in a high school athlete population susceptible to repeated head impact within a sports season. Initial results have indicated relationships between...
READ MORE
Detecting depression using vocal, facial and semantic communication cues
October 15, 2016
Conference Paper
Published in:
ACM Multimedia Conf. AVEC Workshop, 15-19 October 2016.
Summary
Major depressive disorder (MDD) is known to result in neurophysiological and neurocognitive changes that affect control of motor, linguistic, and cognitive functions. MDD's impact on these processes is reflected in an individual's communication via coupled mechanisms: vocal articulation, facial gesturing and choice of content to convey in a dialogue. In particular, MDD-induced neurophysiological changes are associated with a decline in dynamics and coordination of speech and facial motor control, while neurocognitive changes influence dialogue semantics. In this paper, biomarkers are derived from all of these modalities, drawing first from previously developed neurophysiologically motivated speech and facial coordination and timing features. In addition, a novel indicator of lower vocal tract constriction in articulation is incorporated that relates to vocal projection. Semantic features are analyzed for subject/avatar dialogue content using a sparse coded lexical embedding space, and for contextual clues related to the subject's present or past depression status. The features and depression classification system were developed for the 6th International Audio/Video Emotion Challenge (AVEC), which provides data consisting of audio, video-based facial action units, and transcribed text of individuals communicating with the human-controlled avatar. A clinical Patient Health Questionnaire (PHQ) score and binary depression decision are provided for each participant. PHQ predictions were obtained by fusing outputs from a Gaussian staircase regressor for each feature set, with results on the development set of mean F1=0.81, RMSE=5.31, and MAE=3.34. These compare favorably to the challenge baseline development results of mean F1=0.73, RMSE=6.62, and MAE=5.52. On test set evaluation, our system obtained a mean F1=0.70, which is similar to the challenge baseline test result. Future work calls for consideration of joint feature analyses across modalities in an effort to detect neurological disorders based on the interplay of motor, linguistic, affective, and cognitive components of communication.
Summary
Major depressive disorder (MDD) is known to result in neurophysiological and neurocognitive changes that affect control of motor, linguistic, and cognitive functions. MDD's impact on these processes is reflected in an individual's communication via coupled mechanisms: vocal articulation, facial gesturing and choice of content to convey in a dialogue. In...
READ MORE
Relation of automatically extracted formant trajectories with intelligibility loss and speaking rate decline in amyotrophic lateral sclerosis
September 8, 2016
Conference Paper
Published in:
INTERSPEECH 2016: 16th Annual Conf. of the Int. Speech Communication Assoc., 8-12 September 2016.
Summary
Effective monitoring of bulbar disease progression in persons with amyotrophic lateral sclerosis (ALS) requires rapid, objective, automatic assessment of speech loss. The purpose of this work was to identify acoustic features that aid in predicting intelligibility loss and speaking rate decline in individuals with ALS. Features were derived from statistics of the first (F1) and second (F2) formant frequency trajectories and their first and second derivatives. Motivated by a possible link between components of formant dynamics and specific articulator movements, these features were also computed for low-pass and high-pass filtered formant trajectories. When compared to clinician-rated intelligibility and speaking rate assessments, F2 features, particularly mean F2 speed and a novel feature, mean F2 acceleration, were most strongly correlated with intelligibility and speaking rate, respectively (Spearman correlations > 0.70, p < 0.0001). These features also yielded the best predictions in regression experiments (r > 0.60, p < 0.0001). Comparable results were achieved using low-pass filtered F2 trajectory features, with higher correlations and lower prediction errors achieved for speaking rate over intelligibility. These findings suggest information can be exploited in specific frequency components of formant trajectories, with implications for automatic monitoring of ALS.
Summary
Effective monitoring of bulbar disease progression in persons with amyotrophic lateral sclerosis (ALS) requires rapid, objective, automatic assessment of speech loss. The purpose of this work was to identify acoustic features that aid in predicting intelligibility loss and speaking rate decline in individuals with ALS. Features were derived from statistics...
READ MORE
A vocal modulation model with application to predicting depression severity
June 14, 2016
Conference Paper
Published in:
13th IEEE Int. Conf. on Wearable and Implantable Body Sensor Networks, BSN 2016, 14-17 June 2016.
Summary
Speech provides a potential simple and noninvasive "on-body" means to identify and monitor neurological diseases. Here we develop a model for a class of vocal biomarkers exploiting modulations in speech, focusing on Major Depressive Disorder (MDD) as an application area. Two model components contribute to the envelope of the speech waveform: amplitude modulation (AM) from respiratory muscles, and AM from interaction between vocal tract resonances (formants) and frequency modulation in vocal fold harmonics. Based on the model framework, we test three methods to extract envelopes capturing these modulations of the third formant for synthesized sustained vowels. Using subsequent modulation features derived from the model, we predict MDD severity scores with a Gaussian Mixture Model. Performing global optimization over classifier parameters and number of principal components, we evaluate performance of the features by examining the root-mean-squared error (RMSE), mean absolute error (MAE), and Spearman correlation between the actual and predicted MDD scores. We achieved RMSE and MAE values 10.32 and 8.46, respectively (Spearman correlation=0.487, p<0.001), relative to a baseline RMSE of 11.86 and MAE of 10.05, obtained by predicting the mean MDD severity score. Ultimately, our model provides a framework for detecting and monitoring vocal modulations that could also be applied to other neurological diseases.
Summary
Speech provides a potential simple and noninvasive "on-body" means to identify and monitor neurological diseases. Here we develop a model for a class of vocal biomarkers exploiting modulations in speech, focusing on Major Depressive Disorder (MDD) as an application area. Two model components contribute to the envelope of the speech...
READ MORE