Publications

Private Automated Contact Tracing (PACT) was a collaborative team and effort formed during the beginning of the Coronavirus Disease 2019 (COVID-19) pandemic. PACT's mission was to enhance contact tracing in pandemic response by designing exposure-detection functions in personal digital communication devices that have maximal public health utility while preserving privacy. This report explains and discusses the use of automated exposure notification during the COVID-19 pandemic and to provide some recommendations for those who may try to design and deploy similar technologies in future pandemics.

The COVID-19 pandemic placed unprecedented demands on the global public health systems for disease surveillance and contact tracing. Engineers and scientists recognized that it might be possible to augment the efforts of public health teams, if a system for automated digital contact tracing could be quickly devised and deployed to the population of smartphones. The Private Automated Contact Tracing (PACT) protocol was one of several digital contact tracing proposals offered worldwide. PACT’s mission—to preserve individuals’ privacy and anonymity while enabling them to quickly alert even nearby strangers of a likely risky exposure—was adopted by Google and Apple and realized in the Exposure Notifications (EN) service and API for mobile application development. The Exposure Notifications system, like many digital proximity tools, is based on Bluetooth signal strength estimation, and keeps much of the necessary information and computation on the smartphones themselves. It implemented a decentralized approach to contact tracing: the public health authority, and other governmental authorities, cannot access the records of an individual’s encounters with others; nor is physical location used or shared by the service. Although the service is available on most modern iOS and Android devices, it is not enabled by default; the individual must opt in to use a particular region’s implementation of the service, either by installing the regional app or by enrolling through a menu of regions in the operating system settings. Likewise, individuals must affirm their consent before the service can share anonymized infection status with the regional public health authority, and alert recent close contacts. The widespread availability of Exposure Notifications through Apple and Google’s platforms has made it a de facto world standard. Determining its accuracy and effectiveness as a public health tool has been a subject of intense interest. In July 2020, CDC’s Innovative Technologies Team designated MIT LL and the PACT team as trusted technical advisors on the deployment of private automated contact tracing systems as part of its overall public health response to COVID-19. The Innovative Technologies Team sought to answer the following key question regarding automated contact tracing: Does automated contact tracing have sufficient public health value that it is worthwhile to integrate it at scale into existing and evolving manual contact tracing systems? Rapidly rising caseloads necessitated parallel-path assessment activities of most mature systems at the time. When access to the Google and Apple Exposure Notifications system became available, MIT LL focused the assessment efforts on the systems being built and deployed. There were two immediate and significant challenges to observing and quantifying the performance of the system as a whole: first, the privacy preserving design decisions of PACT and the system implementers denied access to system-level performance metrics, and second, obtaining accurate “ground truth” data about risky encounters in the population, against which to measure the detector performance, would require an unacceptable level of effort and intrusion. Therefore, MIT LL designed a set of parallel research activities to decompose the problem into components that could be assessed quantifiably (Bluetooth sensor performance, algorithm performance, user preferences and behaviors), components that could be assessed qualitatively (potential cybersecurity risks, potential for malicious use), and components that could be modeled based on current and emergent knowledge (population-level effects). The MIT LL research team conducted early assessments of the privacy and security aspects of new EN app implementations and closely reviewed the available system code exercised by the apps, before conducting a series of phone-to-phone data collections both in the laboratory and in simulated real-world conditions. The data from these experiments fed into models and visualization tools created to predict and understand the risk score output of candidate “weights and thresholds” configurations for EN, i.e., to predict the performance of the system as-built against ground truth data for distance and duration of “exposure”. The data and performance predictions from this effort helped to inform the global and local community of practice in making configuration decisions, and can help to predict the performance of future versions of similar tools, or alternative implementations of the current system. We conducted a human factors and usability review of early app user interfaces and messaging from public health, and designed a follow-on large-scale survey to investigate questions about user trust and system adoption decisions. The results of the human factors, user trust, and adoption studies were used by U.S. public health jurisdictions to make adjustments to public-facing communications, and were shared with Apple and Google to improve the user interface. Information gathered from public health experts enabled us to better understand conventional contact tracing workflows and data streams, and we incorporated that information into an agent-based model of “hybrid” contact tracing plus Exposure Notifications. We then combined it with emerging reports on vaccination, mask effectiveness, social interaction, variant transmissibility, and our own data on the sensitivity and specificity of the Bluetooth “dose” estimator, to predict system-level effects under various conditions. Finally, we helped to establish a network of Exposure Notifications “practitioners” in public health, who surfaced desirable system-level key performance indicators (implemented during 2021 and 2022, in the Exposure Notifications Private Analytics system, or ENPA). At the conclusion of the program, many of the initial conditions of the pandemic had changed. The Exposure Notifications service was available to most of the world, but had only been deployed by 28 U.S. states and territories, and had not been adopted by much of the population in those regions. High case rates during the Omicron surge (December 2021 – January 2022) and newly available ENPA data offered the first hints at calculating “real” state-level performance metrics, but those data belong to the states and many are cautious about publishing. Although Google and Apple have stated that Exposure Notifications was designed for COVID-19, and will not be maintained in its current form after the pandemic ends, the public health and engineering communities show clear interest in using the “lessons learned” from Exposure Notifications and other similar solutions to preserve the capabilities developed and prepare better systems for future public health emergencies. The intent of this report is to document the work that has been completed, as well as to inform where the work could be updated or adapted to meet future needs.

This dataset includes multimodal physiologic, flight performance, and user interaction data streams, collected as participants performed virtual flight tasks of varying difficulty. In virtual reality, individuals flew an "Instrument Landing System" (ILS) protocol, in which they had to land an aircraft mostly relying on the cockpit instrument readings. Participants were presented with four levels of difficulty, which were generated by varying wind speed, turbulence, and visibility. Each of the participants performed 12 runs, split into 3 blocks of four consecutive runs, one run at each difficulty, in a single experimental session. The sequence of difficulty levels was presented in a counterbalanced manner across blocks. Flight performance was quantified as a function of horizontal and vertical deviation from an ideal path towards the runway as well as deviation from the prescribed ideal speed of 115 knots. Multimodal physiological signals were aggregated and synchronized using Lab Streaming Layer. Descriptions of data quality are provided to assess each data stream. The starter code provides examples of loading and plotting the time synchronized data streams, extracting sample features from the eye tracking data, and building models to predict pilot performance from the physiology data streams.

During a pandemic such as COVID-19, non-pharmaceutical interventions (NPIs) can help protect public health; however, it is not always clear which actions will have the greatest positive impact, or what the trade-offs are between different options. Exposure Notification (EN) was introduced as a prevention measure during the COVID-19 pandemic to supplement traditional contact tracing activities. To predict the estimated impacts of EN, a model for "simulation of automated exposure notification" (SimAEN) was developed by researchers at MIT Lincoln Laboratory (MIT LL) with CDC funding [2]. The model was published through an accessible web interface, available for use by the general public at https://SimAEN.philab.cdc.gov/.

As the COVID-19 pandemic swept the globe in the spring of 2020, technologists looked to enlist technology to assist public health authorities (PHAs) and help stem the tide of infections. As part of this technology push, experts in health care, cryptography, and other related fields developed the Private Automated Contact Tracing (PACT) protocol and related projects to assist the public health objective of slowing the spread of SARS-CoV-2 through digital contact tracing. The joint Google and Apple deployed protocol (Google-Apple Exposure Notifications, also known as GAEN or EN), which became the de facto standard in the U.S., employs the same features as detailed by PACT. The protocol leverages smartphone Bluetooth communications to alert users of potential contact with those carrying the COVID-19 virus in a way that preserves the privacy of both the known-infected individual, and the users receiving the alert. Contact tracing and subsequent personal precautions are more effective at reducing disease spread when more of the population participates, but there are known difficulties with the adoption of novel technology. In order to help the U.S. Centers for Disease Control and Prevention (CDC) and U.S. state-level public health teams address these difficulties, a team of staff from MIT's Lincoln Laboratory (MIT LL) and Computer Science and Artificial Intelligence Laboratory (MIT CSAIL) focused on studying user perception and information needs.

BLEMUR, or Bluetooth Low Energy Model of User Risk, is a model of the probability of alert at a given duration and distance of an index case for a specific configuration of settings for an Exposure Notification (EN) system.The Google-Apple EN framework operates in the duration and Bluetooth Low Energy (BLE) signal attenuation domains. However, many public health definitions of "exposure" to a disease are based upon the distance between an index case and another person. To bridge the conceptual gap for public health authorities (PHAs) from the familiar distance-and-duration space to the signal attenuation-and-duration space, BLEMUR uses BLE signal attenuation as a proxy for distance between people, albeit an imprecise one. This paper will discuss the EN settings that can be manipulated, the BLE data collected, how data support a model of the relationship between measured attenuation and distance between phones, and how BLEMUR calculates the probability of alert for a distance and duration based on the settings and data.

Automated Exposure Notication (AEN) was implemented in 2020 to supplement traditional contact tracing for COVID-19 by estimating "too close for too long" proximities of people using the service. AEN uses Bluetooth messages to privately label and recall proximity events, so that persons who were likely exposed to SARS-CoV-2 can take the appropriate steps recommended by their health care authority. This paper describes an agent-based model that estimates the effects of AEN deployment on COVID-19 caseloads and public health workloads in the context of other critical public health measures available during the COVID-19 pandemic. We selected simulation variables pertinent to AEN deployment options, varied them in accord with the system dynamics available in 2020-2021, and calculated the outcomes of key metrics across repeated runs of the stochastic multi-week simulation. SimAEN's parameters were set to ranges of observed values in consultation with public health professionals and the rapidly accumulating literature on COVID-19 transmission; the model was validated against available population-level disease metrics. Estimates from SimAEN can help public health officials determine what AEN deployment decisions (e.g., configuration, workflow integration, and targeted adoption levels) can be most effective in their jurisdiction, in combination with other COVID-19 interventions (e.g., mask use, vaccination, quarantine and isolation periods).

Early detection of diseases such as COVID-19 could be a critical tool in reducing disease transmission by helping individuals recognize when they should self-isolate, seek testing, and obtain early medical intervention. Consumer wearable devices that continuously measure physiological metrics hold promise as tools for early illness detection. We gathered daily questionnaire data and physiological data using a consumer wearable (Oura Ring) from 63,153 participants, of whom 704 self-reported possible COVID-19 disease. We selected 73 of these 704 participants with reliable confirmation of COVID-19 by PCR testing and high-quality physiological data for algorithm training to identify onset of COVID-19 using machine learning classification. The algorithm identified COVID-19 an average of 2.75 days before participants sought diagnostic testing with a sensitivity of 82% and specificity of 63%. The receiving operating characteristic (ROC) area under the curve (AUC) was 0.819 (95% CI [0.809, 0.830]). Including continuous temperature yielded an AUC 4.9% higher than without this feature. For further validation, we obtained SARS CoV-2 antibody in a subset of participants and identified 10 additional participants who self-reported COVID-19 disease with antibody confirmation. The algorithm had an overall ROC AUC of 0.819 (95% CI [0.809, 0.830]), with a sensitivity of 90% and specificity of 80% in these additional participants. Finally, we observed substantial variation in accuracy based on age and biological sex. Findings highlight the importance of including temperature assessment, using continuous physiological features for alignment, and including diverse populations in algorithm development to optimize accuracy in COVID-19 detection from wearables.

There is mounting evidence linking the cumulative effects of repetitive head impacts to neuro-degenerative conditions. Robust clinical assessment tools to identify mild traumatic brain injuries are needed to assist with timely diagnosis for return-to-field decisions and appropriately guide rehabilitation. The focus of the present study is to investigate the potential for oculomotor features to complement existing diagnostic tools, such as measurements of Optic Nerve Sheath Diameter (ONSD) and Immediate Post-concussion Assessment and Cognitive Testing (ImPACT). Thirty-one high school American football and soccer athletes were tracked through the course of a sports season. Given the high risk of repetitive head impacts associated with both soccer and football, our hypotheses were that (1) ONSD and ImPACT scores would worsen through the season and (2) oculomotor features would effectively capture both neurophysiological changes reflected by ONSD and neuro-functional status assessed via ImPACT. Oculomotor features were used as input to Linear Mixed-Effects Regression models to predict ONSD and ImPACT scores as outcomes. Prediction accuracy was evaluated to identify explicit relationships between eye movements, ONSD, and ImPACT scores. Significant Pearson correlations were observed between predicted and actual outcomes for ONSD (Raw = 0.70; Normalized = 0.45) and for ImPACT (Raw = 0.86; Normalized = 0.71), demonstrating the capability of oculomotor features to capture neurological changes detected by both ONSD and ImPACT. The most predictive features were found to relate to motor control and visual-motor processing. In future work, oculomotor models, linking neural structures to oculomotor function, can be built to gain extended mechanistic insights into neurophysiological changes observed through seasons of participation in contact sports.

Diagnosis of COVID-19 at point of care is vital to the containment of the global pandemic. Point of care ultrasound (POCUS) provides rapid imagery of lungs to detect COVID-19 in patients in a repeatable and cost effective way. Previous work has used public datasets of POCUS videos to train an AI model for diagnosis that obtains high sensitivity. Due to the high stakes application we propose the use of robust and explainable techniques. We demonstrate experimentally that robust models have more stable predictions and offer improved interpretability. A framework of contrastive explanations based on adversarial perturbations is used to explain model predictions that aligns with human visual perception.