Publications

Refine Results

(Filters Applied) Clear All

Classifier performance estimation with unbalanced, partially labeled data

Published in:
Proc. Machine Learning Research, Vol. 88, 2018, pp. 4-16.

Summary

Class imbalance and lack of ground truth are two significant problems in modern machine learning research. These problems are especially pressing in operational contexts where the total number of data points is extremely large and the cost of obtaining labels is very high. In the face of these issues, accurate estimation of the performance of a detection or classification system is crucial to inform decisions based on the observations. This paper presents a framework for estimating performance of a binary classifier in such a context. We focus on the scenario where each set of measurements has been reduced to a score, and the operator only investigates data when the score exceeds a threshold. The operator is blind to the number of missed detections, so performance estimation targets two quantities: recall and the derivative of precision with respect to recall. Measuring with respect to error in these two metrics, simulations in this context demonstrate that labeling outliers not only outperforms random labeling, but often matches performance of an adaptive method that attempts to choose the optimal data for labeling. Application to real anomaly detection data confirms the utility of the approach, and suggests direction for future work.
READ LESS

Summary

Class imbalance and lack of ground truth are two significant problems in modern machine learning research. These problems are especially pressing in operational contexts where the total number of data points is extremely large and the cost of obtaining labels is very high. In the face of these issues, accurate...

READ MORE

Improving security at the system-call boundary in a type-safe operating system

Published in:
Thesis (M.E.)--Massachusetts Institute of Technology, 2018.

Summary

Historically, most approaches to operating sytems security aim to either protect the kernel (e.g., the MMU) or protect user applications (e.g., W exclusive or X). However, little study has been done into protecting the boundary between these layers. We describe a vulnerability in Tock, a type-safe operating system, at the system-call boundary. We then introduce a technique for providing memory safety at the boundary between userland and the kernel in Tock. We demonstrate that this technique works to prevent against the aforementioned vulnerability and a class of similar vulnerabilities, and we propose how it might be used to protect against simliar vulnerabilities in other operating systems.
READ LESS

Summary

Historically, most approaches to operating sytems security aim to either protect the kernel (e.g., the MMU) or protect user applications (e.g., W exclusive or X). However, little study has been done into protecting the boundary between these layers. We describe a vulnerability in Tock, a type-safe operating system, at the...

READ MORE

Key Challenges and Prospects for Optical Standoff Trace Detection of Explosives

Published in:
Trends in Analytical Chemistry, vol. 100

Summary

Sophisticated improvised explosive devices (IEDs) challenge the capabilities of current sensors, particularly in areas away from static checkpoints. This security gap could be filled by standoff chemical sensors that detect IEDs based on external trace explosive residues. Unfortunately, previous efforts have not led to widely deployed capabilities. Crucially, the physical morphology of trace explosive residues and chemical “clutter” present unique challenges to the operational performance of standoff sensors. In this review, an overview of standoff trace explosive detection systems is provided in the context of these unique challenges. Tradespace analysis is performed for two popular standoff detection methods: longwave infrared hyperspectral imaging and deep-UV Raman spectroscopy. The tradespace analysis method described in this review incorporates realistic trace explosive residues and background clutter into the technology development process. The review predicts system performance and areas where additional research is needed for these two technologies to optimize performance.
READ LESS

Summary

Sophisticated improvised explosive devices (IEDs) challenge the capabilities of current sensors, particularly in areas away from static checkpoints. This security gap could be filled by standoff chemical sensors that detect IEDs based on external trace explosive residues. Unfortunately, previous efforts have not led to widely deployed capabilities. Crucially, the physical...

READ MORE

Potential impacts of climate warming and increased summer heat stress on the electric grid: a case study for a large power transformer (LPT) in the Northeast United States

Published in:
Climatic Change, 20 November 2017, https://doi.org/10.1007/s10584-017-2114-x
R&D group:

Summary

Large power transformers (LPTs) are critical yet vulnerable components of the power grid. More frequent and intense heat waves or high temperatures can degrade their operational lifetime and increase the risk of premature failure. Without adequate preparedness, a widespread situation could ultimately lead to prolonged grid disruption and incur excessive economic costs. Here, we investigate the potential impact of climate warming and corresponding shifts in summertime "hot days" on a selected LPT located in the Northeast United States. We apply an analogue method, which detects the occurrence of hot days based on the salient, associated large-scale atmospheric conditions, to assess the risk of future change in their occurrence. Compared with the more conventional approach that relies on climate model simulated daily maximum temperature, the analogue method produces model medians of late twentieth century hot day frequency that are more consistent with observation and have stronger inter-model consensus. Under the climate warming scenarios, multi-model medians of both model daily maximum temperature and the analogue method indicate strong decadal increases in hot day frequency by the late twenty-first century, but the analogue method improves model consensus considerably. The decrease of transformer lifetime with temperature increase is further assessed. The improved inter-model consensus of the analogue method is viewed as a promising step toward providing actionable information for a more stable, reliable, and environmentally responsible national grid.
READ LESS

Summary

Large power transformers (LPTs) are critical yet vulnerable components of the power grid. More frequent and intense heat waves or high temperatures can degrade their operational lifetime and increase the risk of premature failure. Without adequate preparedness, a widespread situation could ultimately lead to prolonged grid disruption and incur excessive...

READ MORE

Detecting pathogen exposure during the non-symptomatic incubation period using physiological data

Summary

Early pathogen exposure detection allows better patient care and faster implementation of public health measures (patient isolation, contact tracing). Existing exposure detection most frequently relies on overt clinical symptoms, namely fever, during the infectious prodromal period. We have developed a robust machine learning based method to better detect asymptomatic states during the incubation period using subtle, sub-clinical physiological markers. Starting with highresolution physiological waveform data from non-human primate studies of viral (Ebola, Marburg, Lassa, and Nipah viruses) and bacterial (Y. pestis) exposure, we processed the data to reduce short-term variability and normalize diurnal variations, then provided these to a supervised random forest classification algorithm and post-classifier declaration logic step to reduce false alarms. In most subjects detection is achieved well before the onset of fever; subject cross-validation across exposure studies (varying viruses, exposure routes, animal species, and target dose) lead to 51h mean early detection (at 0.93 area under the receiver-operating characteristic curve [AUCROC]). Evaluating the algorithm against entirely independent datasets for Lassa, Nipah, and Y. pestis exposures un-used in algorithm training and development yields a mean 51h early warning time (at AUCROC=0.95). We discuss which physiological indicators are most informative for early detection and options for extending this capability to limited datasets such as those available from wearable, non-invasive, ECG-based sensors.
READ LESS

Summary

Early pathogen exposure detection allows better patient care and faster implementation of public health measures (patient isolation, contact tracing). Existing exposure detection most frequently relies on overt clinical symptoms, namely fever, during the infectious prodromal period. We have developed a robust machine learning based method to better detect asymptomatic states...

READ MORE

Peregrine: 3-D network localization and navigation

Published in:
IEEE 9th Latin-American Conf. on Communications, LATINCOM, 8-10 November 2017.

Summary

Location-aware devices will create new services and applications in emerging fields such as autonomous driving, smart cities, and the Internet of Things. Many existing localization systems rely on anchors such as satellites at known positions which broadcast radio signals. However, such signals may be blocked by obstacles, corrupted by multipath propagation, or provide insufficient localization accuracy. Therefore, ubiquitous localization remains an extremely challenging problem. This paper introduces Peregrine, a 3-D cooperative network localization and navigation (NLN) system. Peregrine nodes are low-cost business-card-sized devices, consisting of a microprocessor, a commercially available ultra-wideband (UWB) radio module, and a small battery. Recently developed distributed algorithms are used in Peregrine to solve the highly interrelated problems of node inference and node activation in real-time, enabling resource efficiency, scalability, and accuracy for NLN. Node inference – based on the recently introduced sigma point belief propagation (SPBP) algorithm – enables spatiotemporal cooperation in realtime and estimates the nodes' positions accurately from UWB distance measurements. A distributed node activation algorithm controls channel access to improve the efficiency and reduce the localization error of the network. Contributions of each algorithmic component to overall system performance are validated through indoor localization experiments. Our results show that Peregrine achieves decimeter-level 3-D position accuracy in a challenging propagation environment.
READ LESS

Summary

Location-aware devices will create new services and applications in emerging fields such as autonomous driving, smart cities, and the Internet of Things. Many existing localization systems rely on anchors such as satellites at known positions which broadcast radio signals. However, such signals may be blocked by obstacles, corrupted by multipath...

READ MORE

Preliminary UAS Weather Research Roadmap(1.51 MB)

Published in:
Project Report ATC-438, MIT Lincoln Laboratory

Summary

A companion Lincoln Laboratory report (ATC-437, “Preliminary Weather Information Gaps for UAS Operations”) identified initial gaps in the ability of current weather products to meet the needs of UAS operations. Building off of that work, this report summarizes the development of a proposed initial roadmap for research to fill the gaps that were identified.
READ LESS

Summary

A companion Lincoln Laboratory report (ATC-437, “Preliminary Weather Information Gaps for UAS Operations”) identified initial gaps in the ability of current weather products to meet the needs of UAS operations. Building off of that work, this report summarizes the development of a proposed initial roadmap for research to fill the...

READ MORE

Preliminary Weather Information Gap Analysis for UAS Operations(4.88 MB)

Published in:
Project Report ATC-437, MIT Lincoln Laboratory

Summary

Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) are rapidly increasing. For example, 2017 has seen dramatically increased low altitude UAS usage for disaster relief and by first responders. The ability to carry out these operations, however, can be strongly impacted by adverse weather conditions. This report documents a preliminary quick-look identification and assessment of gaps in current weather decision support for UAS operations.
READ LESS

Summary

Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) are rapidly increasing. For example, 2017 has seen dramatically increased low altitude UAS usage for disaster relief and by first responders. The ability to carry out these operations, however, can be strongly impacted by adverse weather conditions. This report...

READ MORE

Preliminary weather information gap analysis for UAS operations, revision 1

Published in:
MIT Lincoln Laboratory Report ATC-437-REV-1

Summary

Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) are rapidly increasing. For example, 2017 has seen dramatically increased low altitude UAS usage for disaster relief and by first responders. The ability to carry out these operations, however, can be strongly impacted by adverse weather conditions. This report documents a preliminary quick-look identification and assessment of gaps in current weather decision support for UAS operations. An initial set of surveys and interviews with UAS operators identified 12 major gaps. These gaps were then prioritized based on the importance of the weather phenomena to UAS operations and the current availability of adequate weather information to UAS operators. Low altitude UAS operations are of particular concern. The lack of observations of ceiling, visibility, and winds near most low altitude UAS operational locations causes the validation of numerical weather forecasts of weather conditions for those locations to be the highest priority. Hazardous weather alerting for convective activity and strong surface winds are a major concern for UAS operations that could be addressed in part by access to existing FAA real time conventional aircraft weather products.
READ LESS

Summary

Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) are rapidly increasing. For example, 2017 has seen dramatically increased low altitude UAS usage for disaster relief and by first responders. The ability to carry out these operations, however, can be strongly impacted by adverse weather conditions. This report...

READ MORE

MOVPE growth of LWIR AlInAs/GaInAs/InP quantum cascade lasers: impact of growth and material quality on laser performance

Summary

The quality of epitaxial layers in quantum cascade lasers (QCLs) has a primary impact on QCL performance, and establishing correlations between epitaxial growth and materials properties is of critical importance for continuing improvements. We present an overview of the growth challenges of these complex QCL structures; describe the metalorganic vapor phase epitaxy growth of AlInAs/GaInAs/InP QCL materials; discuss materials properties that impact QCL performance; and investigate various QCL structure modifications and their effects on QCL performance. We demonstrate uncoated buried-heterostructure 9.3-um QCLs with 1.32-W continuous-wave output power and maximum wall plug efficiency (WPE) of 6.8%. This WPE is more than 50% greater than previously reported WPEs for unstrained QCLs emitting at 8.9 um and only 30% below strained QCLs emitting around 9.2 um.
READ LESS

Summary

The quality of epitaxial layers in quantum cascade lasers (QCLs) has a primary impact on QCL performance, and establishing correlations between epitaxial growth and materials properties is of critical importance for continuing improvements. We present an overview of the growth challenges of these complex QCL structures; describe the metalorganic vapor...

READ MORE