Publications
Detection of COVID-19 using multimodal data from a wearable device: results from the first TemPredict Study
March 2, 2022
Journal Article
Published in:
Sci. Rep., Vol. 12, No. 1, 2 March 2022.
Topic:
R&D area:
Summary
Early detection of diseases such as COVID-19 could be a critical tool in reducing disease transmission by helping individuals recognize when they should self-isolate, seek testing, and obtain early medical intervention. Consumer wearable devices that continuously measure physiological metrics hold promise as tools for early illness detection. We gathered daily questionnaire data and physiological data using a consumer wearable (Oura Ring) from 63,153 participants, of whom 704 self-reported possible COVID-19 disease. We selected 73 of these 704 participants with reliable confirmation of COVID-19 by PCR testing and high-quality physiological data for algorithm training to identify onset of COVID-19 using machine learning classification. The algorithm identified COVID-19 an average of 2.75 days before participants sought diagnostic testing with a sensitivity of 82% and specificity of 63%. The receiving operating characteristic (ROC) area under the curve (AUC) was 0.819 (95% CI [0.809, 0.830]). Including continuous temperature yielded an AUC 4.9% higher than without this feature. For further validation, we obtained SARS CoV-2 antibody in a subset of participants and identified 10 additional participants who self-reported COVID-19 disease with antibody confirmation. The algorithm had an overall ROC AUC of 0.819 (95% CI [0.809, 0.830]), with a sensitivity of 90% and specificity of 80% in these additional participants. Finally, we observed substantial variation in accuracy based on age and biological sex. Findings highlight the importance of including temperature assessment, using continuous physiological features for alignment, and including diverse populations in algorithm development to optimize accuracy in COVID-19 detection from wearables.
Summary
Early detection of diseases such as COVID-19 could be a critical tool in reducing disease transmission by helping individuals recognize when they should self-isolate, seek testing, and obtain early medical intervention. Consumer wearable devices that continuously measure physiological metrics hold promise as tools for early illness detection. We gathered daily...
READ MORE
Bayesian estimation of PLDA in the presence of noisy training labels, with applications to speaker verification
February 18, 2022
Journal Article
Published in:
IEEE/ACM Trans. Audio, Speech, Language Process., Vol. 30, 2022, pp. 414-28.
Topic:
R&D area:
Summary
This paper presents a Bayesian framework for estimating a Probabilistic Linear Discriminant Analysis (PLDA) model in the presence of noisy labels. True class labels are interpreted as latent random variables, which are transmitted through a noisy channel, and received as observed speaker labels. The labeling process is modeled as a Discrete Memoryless Channel (DMC). PLDA hyperparameters are interpreted as random variables, and their joint posterior distribution is derived using meanfield Variational Bayes, allowing maximum a posteriori (MAP) estimates of the PLDA model parameters to be determined. The proposed solution, referred to as VB-MAP, is presented as a general framework, but is studied in the context of speaker verification, and a variety of use cases are discussed. Specifically, VB-MAP can be used for PLDA estimation with unreliable labels, unsupervised PLDA estimation, and to infer the reliability of a PLDA training set. Experimental results show the proposed approach to provide significant performance improvements on a variety of NIST Speaker Recognition Evaluation (SRE) tasks, both for data sets with simulated mislabels, and for data sets with naturally occurring missing or unreliable labels.
Summary
This paper presents a Bayesian framework for estimating a Probabilistic Linear Discriminant Analysis (PLDA) model in the presence of noisy labels. True class labels are interpreted as latent random variables, which are transmitted through a noisy channel, and received as observed speaker labels. The labeling process is modeled as a...
READ MORE
Artificial intelligence for detecting COVID-19 with the aid of human cough, breathing and speech signals: scoping review
February 14, 2022
Journal Article
Published in:
IEEE Open J. Eng. Med. Biol., 14 February 2022, pp. 235-41.
Topic:
R&D area:
R&D group:
Summary
Background: Official tests for COVID-19 are time consuming, costly, can produce high false negatives, use up vital chemicals and may violate social distancing laws. Therefore, a fast and reliable additional solution using recordings of cough, breathing and speech data forpreliminary screening may help alleviate these issues. Objective: This scoping review explores how Artificial Intelligence (AI) technology aims to detect COVID-19 disease by using cough, breathing and speech recordings, as reported in theliterature. Here, we describe and summarize attributes of the identified AI techniques and datasets used for their implementation. Methods: A scoping review was conducted following the guidelines of PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews). Electronic databases (Google Scholar, Science Direct, and IEEE Xplore) were searched between 1st April 2020 and 15st August 2021. Terms were selected based on thetarget intervention (i.e., AI), the target disease (i.e., COVID-19) and acoustic correlates of thedisease (i.e., speech, breathing and cough). A narrative approach was used to summarize the extracted data. Results: 24 studies and 8 Apps out of the 86 retrieved studies met the inclusion criteria. Halfof the publications and Apps were from the USA. The most prominent AI architecture used was a convolutional neural network, followed by a recurrent neural network. AI models were mainly trained, tested and run-on websites and personal computers, rather than on phone apps. More than half of the included studies reported area-under-the-curve performance of greater than 0.90 on symptomatic and negative datasets while one study achieved 100% sensitivity in predicting asymptomatic COVID-19 for cough-, breathing- or speech-based acoustic features. Conclusions: The included studies show that AI has the potential to help detect COVID-19 using cough, breathing and speech samples. However, the proposed methods with some time and appropriate clinical testing would prove to be an effective method in detecting various diseases related to respiratory and neurophysiological changes in human body.
Summary
Background: Official tests for COVID-19 are time consuming, costly, can produce high false negatives, use up vital chemicals and may violate social distancing laws. Therefore, a fast and reliable additional solution using recordings of cough, breathing and speech data forpreliminary screening may help alleviate these issues. Objective: This scoping review...
READ MORE
Speech as a biomarker: opportunities, interoperability, and challenges
February 11, 2022
Journal Article
Published in:
Perspectives of the ASHA Special Interest Groups, Vo. 7, February 2022, pp. 276-83.
Topic:
R&D area:
R&D group:
Summary
Purpose: Over the past decade, the signal processing and machine learning literature has demonstrated notable advancements in automated speech processing with the use of artificial intelligence for medical assessment and monitoring (e.g., depression, dementia, and Parkinson's disease, among others). Meanwhile, the clinical speech literature has identified several interpretable, theoretically motivated measures that are sensitive to abnormalities in the cognitive, linguistic, affective, motoric, and anatomical domains. Both fields have, thus, independently demonstrated the potential for speech to serve as an informative biomarker for detecting different psychiatric and physiological conditions. However, despite these parallel advancements, automated speech biomarkers have not been integrated into routine clinical practice to date. Conclusions: In this article, we present opportunities and challenges for adoption of speech as a biomarker in clinical practice and research. Toward clinical acceptance and adoption of speech-based digital biomarkers, we argue for the importance of several factors such as robustness, specificity, diversity, and physiological interpretability of speech analytics in clinical applications.
Summary
Purpose: Over the past decade, the signal processing and machine learning literature has demonstrated notable advancements in automated speech processing with the use of artificial intelligence for medical assessment and monitoring (e.g., depression, dementia, and Parkinson's disease, among others). Meanwhile, the clinical speech literature has identified several interpretable, theoretically motivated...
READ MORE
Energy resilience: exercises for Marine Corps installations
February 1, 2022
Journal Article
Published in:
Marine Corps Gazette, Vol. 106, No. 2, February 2022, p. 20-24.
Summary
Microgrids are areas that are self-sufficient for power that can controllably disconnect from the incoming utility feed and control generation assets in conjunction with changing load requirements. They are increasingly being touted as a way to improve installations energy resilience because they allow installations to decouple from the larger electric grid if it fails and continue to provide power in the face of growing natural and man-made threats to Marine Corps installations. However, before commanders can put resources toward upgrading infrastructure, they need to identify and understand their vulnerabilities. A key way to do this is by holding exercises designed to simulate grid failures and outages either in a tabletop manner or in realtime. These exercises also help personnel train for disruptions, understand their impact on operations, and identify unknown interdependencies that can be just as important as investing in resilient technology and the physical electric grid. In order for the equipment to work, personnel have to know how to employ it and commands need to understand how outages will affect their installations. These types of exercises are as important as the physical infrastructure or ensuring the energy resilience of Marine Corps installations and the missions that depend on them in the future.
Summary
Microgrids are areas that are self-sufficient for power that can controllably disconnect from the incoming utility feed and control generation assets in conjunction with changing load requirements. They are increasingly being touted as a way to improve installations energy resilience because they allow installations to decouple from the larger electric...
READ MORE
EEG alpha and pupil diameter reflect endogenous auditory attention switching and listening effort
January 19, 2022
Journal Article
Published in:
Eur. J. Neurosci., 2022, pp. 1-16.
Topic:
R&D area:
R&D group:
Summary
Everyday environments often contain distracting competing talkers and background noise, requiring listeners to focus their attention on one acoustic source and reject others. During this auditory attention task, listeners may naturally interrupt their sustained attention and switch attended sources. The effort required to perform this attention switch has not been well studied in the context of competing continuous speech. In this work, we developed two variants of endogenous attention switching and a sustained attention control. We characterized these three experimental conditions under the context of decoding auditory attention, while simultaneously evaluating listening effort and neural markers of spatial-audio cues. A least-squares, electroencephalography (EEG) based, attention decoding algorithm was implemented across all conditions. It achieved an accuracy of 69.4% and 64.0% when computed over non-overlapping 10 and 5-second correlation windows, respectively. Both decoders illustrated smooth transitions in the attended talker prediction through switches at approximately half of the analysis window size (e.g. the mean lag taken across the two switch conditions was 2.2 seconds when the 5-second correlation window was used). Expended listening effort, as measured by simultaneous EEG and pupillometry, was also a strong indicator of whether the listeners sustained attention or performed an endogenous attention switch (peak pupil diameter measure (p = 0.034) and minimum parietal alpha power measure (p = 0.016)). We additionally found evidence of talker spatial cues in the form of centrotemporal alpha power lateralization (p = 0.0428). These results suggest that listener effort and spatial cues may be promising features to pursue in a decoding context, in addition to speech-based features.
Summary
Everyday environments often contain distracting competing talkers and background noise, requiring listeners to focus their attention on one acoustic source and reject others. During this auditory attention task, listeners may naturally interrupt their sustained attention and switch attended sources. The effort required to perform this attention switch has not been...
READ MORE
Tools and practices for responsible AI engineering
January 14, 2022
Journal Article
Published in:
arXiv, 14 January 2022.
Summary
Responsible Artificial Intelligence (AI)—the practice of developing, evaluating, and maintaining accurate AI systems that also exhibit essential properties such as robustness and explainability—represents a multifaceted challenge that often stretches standard machine learning tooling, frameworks, and testing methods beyond their limits. In this paper, we present two new software libraries—hydra-zen and the rAI-toolbox—that address critical needs for responsible AI engineering. hydra-zen dramatically simplifies the process of making complex AI applications configurable, and their behaviors reproducible. The rAI-toolbox is designed to enable methods for evaluating and enhancing the robustness of AI-models in a way that is scalable and that composes naturally with other popular ML frameworks. We describe the design principles and methodologies that make these tools effective, including the use of property-based testing to bolster the reliability of the tools themselves. Finally, we demonstrate the composability and flexibility of the tools by showing how various use cases from adversarial robustness and explainable AI can be concisely implemented with familiar APIs.
Summary
Responsible Artificial Intelligence (AI)—the practice of developing, evaluating, and maintaining accurate AI systems that also exhibit essential properties such as robustness and explainability—represents a multifaceted challenge that often stretches standard machine learning tooling, frameworks, and testing methods beyond their limits. In this paper, we present two new software libraries—hydra-zen and...
READ MORE
Gait instability and estimated core temperature predict exertional heat stroke
January 12, 2022
Journal Article
Published in:
Br J Sports Med., online ahead of print, 12 January 2022.
Topic:
R&D area:
R&D group:
Summary
Objective Exertional heat stroke (EHS), characterised by a high core body temperature (Tcr) and central nervous system (CNS) dysfunction, is a concern for athletes, workers and military personnel who must train and perform in hot environments. The objective of this study was to determine whether algorithms that estimate Tcr from heart rate and gait instability from a trunk-worn sensor system can forward predict EHS onset. Methods Heart rate and three-axis accelerometry data were collected from chest-worn sensors from 1806 US military personnel participating in timed 4/5-mile runs, and loaded marches of 7 and 12 miles; in total, 3422 high EHS-risk training datasets were available for analysis. Six soldiers were diagnosed with heat stroke and all had rectal temperatures of >41°C when first measured and were exhibiting CNS dysfunction. Estimated core temperature (ECTemp) was computed from sequential measures of heart rate. Gait instability was computed from three-axis accelerometry using features of pattern dispersion and autocorrelation. Results The six soldiers who experienced heat stroke were among the hottest compared with the other soldiers in the respective training events with ECTemps ranging from 39.2°C to 40.8°C. Combining ECTemp and gait instability measures successfully identified all six EHS casualties at least 3.5 min in advance of collapse while falsely identifying 6.1% (209 total false positives) examples where exertional heat illness symptoms were neither observed nor reported. No false-negative cases were noted. Conclusion The combination of two algorithms that estimate Tcr and ataxic gate appears promising for real-time alerting of impending EHS.
Summary
Objective Exertional heat stroke (EHS), characterised by a high core body temperature (Tcr) and central nervous system (CNS) dysfunction, is a concern for athletes, workers and military personnel who must train and perform in hot environments. The objective of this study was to determine whether algorithms that estimate Tcr from...
READ MORE
Alternative cue and response modalities maintain the Simon effect but impact task performance
January 7, 2022
Journal Article
Published in:
Appl. Ergon., Vol. 100, 2022, 103648.
Topic:
R&D area:
R&D group:
Summary
Inhibitory control, the ability to inhibit impulsive responses and irrelevant stimuli, enables high level functioning and activities of daily living. The Simon task probes inhibition using interfering stimuli, i.e., cues spatially presented on the opposite side of the indicated response; incongruent response times (RT) are slower than congruent RTs. Operational applicability of the Simon task beyond finger/hand manipulations and visual/auditory cues is unclear, but important to consider as new technologies provide tactile cues and require motor responses from the lower extremity (e.g., exoskeletons). In this study, twenty participants completed the Simon task under four conditions, each combination of two cue (visual/tactile) and response (upper/lower-extremity) modalities. RT were significantly longer for incongruent than congruent cues across cue-response pairs. However, alternative cue-response pairs yielded slower RT and decreased accuracy for tactile cues and lower extremity responses. Results support operational usage of the Simon task to probe inhibition using tactile cues and lower-extremity responses relevant for new technologies like exoskeletons and immersive environments.
Summary
Inhibitory control, the ability to inhibit impulsive responses and irrelevant stimuli, enables high level functioning and activities of daily living. The Simon task probes inhibition using interfering stimuli, i.e., cues spatially presented on the opposite side of the indicated response; incongruent response times (RT) are slower than congruent RTs. Operational...
READ MORE
Individuals differ in muscle activation patterns during early adaptation to a powered ankle exoskeleton
January 1, 2022
Journal Article
Published in:
Applied Ergonomics Volume 98, January 2022, 103593
Topic:
R&D area:
R&D group:
Summary
Exoskeletons have the potential to assist users and augment physical ability. To achieve these goals across users, individual variation in muscle activation patterns when using an exoskeleton need to be evaluated. This study examined individual muscle activation patterns during walking with a powered ankle exoskeleton. 60% of the participants were observed to reduce medial gastrocnemius activation with exoskeleton powered and increase with the exoskeleton unpowered during stance. 80% of the participants showed a significant increase in tibialis anterior activation upon power addition, with inconsistent changes upon power removal during swing. 60% of the participants that were able to adapt to the system, did not de-adapt after 5 min. Muscle activity patterns differ between individuals in response to the exoskeleton power state, and affected the antagonist muscle behavior during this early adaptation. It is important to understand these different individual behaviors to inform the design of exoskeleton controllers and training protocols.
Summary
Exoskeletons have the potential to assist users and augment physical ability. To achieve these goals across users, individual variation in muscle activation patterns when using an exoskeleton need to be evaluated. This study examined individual muscle activation patterns during walking with a powered ankle exoskeleton. 60% of the participants were...
READ MORE