Publications
Benchmarking the processing of aircraft tracks with triples mode and self-scheduling
September 20, 2021
Conference Paper
Published in:
2021 IEEE High Performance Extreme Computing Conf., HPEC, 20-24 September 2021.
Summary
As unmanned aircraft systems (UASs) continue to integrate into the U.S. National Airspace System (NAS), there is a need to quantify the risk of airborne collisions between unmanned and manned aircraft to support regulation and standards development. Developing and certifying collision avoidance systems often rely on the extensive use of Monte Carlo collision risk analysis simulations using probabilistic models of aircraft flight. To train these models, high performance computing resources are required. We've prototyped a high performance computing workflow designed and deployed on the Lincoln Laboratory Supercomputing Center to process billions of observations of aircraft. However, the prototype has various computational and storage bottlenecks that limited rapid or more comprehensive analyses and models. In response, we've developed a novel workflow to take advantage of various job launch and task distribution technologies to improve performance. The workflow was benchmarked using two datasets of observations of aircraft, including a new dataset focused on the environment around aerodromes. Optimizing how the workflow was parallelized drastically reduced the execution time from weeks to days.
Summary
As unmanned aircraft systems (UASs) continue to integrate into the U.S. National Airspace System (NAS), there is a need to quantify the risk of airborne collisions between unmanned and manned aircraft to support regulation and standards development. Developing and certifying collision avoidance systems often rely on the extensive use of...
READ MORE
Benefits of realist ontologies to systems engineering
September 11, 2021
Conference Paper
Published in:
Joint Ontology Workshops 2021 Episode VII: The Bolzano Summer of Knowledge, JOWO 2021, 11-18 September 2021.
Summary
Applied ontologies have been used more and more frequently to enhance systems engineering. In this paper, we argue that adopting principles of ontological realism can increase the benefits that ontologies have already been shown to provide to the systems engineering process. Moreover, adopting Basic Formal Ontology (BFO), an ISO standard for top-level ontologies from which more domain specific ontologies are constructed, can lead to benefits in four distinct areas of systems engineering: (1) interoperability, (2) standardization, (3) testing, and (4) data exploitation. Reaping these benefits in a model-based systems engineering (MBSE) context requires utilizing an ontology's vocabulary when modeling systems and entities within those systems. If the chosen ontology abides by the principles of ontological realism, a semantic standard capable of uniting distinct domains, using BFO as a hub, can be leveraged to promote greater interoperability among systems. As interoperability and standardization increase, so does the ability to collect data during the testing and implementation of systems. These data can then be reasoned over by computational reasoners using the logical axioms within the ontology. This, in turn, generates new data that would have been impossible or too inefficient to generate without the aid of computational reasoners.
Summary
Applied ontologies have been used more and more frequently to enhance systems engineering. In this paper, we argue that adopting principles of ontological realism can increase the benefits that ontologies have already been shown to provide to the systems engineering process. Moreover, adopting Basic Formal Ontology (BFO), an ISO standard...
READ MORE
Demand and capacity modeling for advanced air mobility
August 2, 2021
Conference Paper
Published in:
AIAA Aviation 2021 Conf., 2-6 August 2021.
Topic:
R&D area:
R&D group:
Summary
Advanced Air Mobility encompasses emerging aviation technologies that transport people and cargo between local, regional, or urban locations that are currently underserved by aviation and other transportation modalities. The disruptive nature of these technologies has pushed industry, academia, and governments to devote significant investments to understand their impact on airspace risk, operational procedures, and passengers. A flexible framework was designed to assess the operational viability of these technologies and the sensitivity to a variety of assumptions. This framework is used to simulate air taxi traffic within New York City by replacing a portion of the city's taxi requests with trips taken with electric vertical takeoff and landing vehicles and evaluate the sensitivity of passenger trip time to a variety of system wide assumptions. In particular, the paper focuses on the impact of the passenger capacity, landing site vehicle capacity, and fleet size. The operation density is then compared with the current air traffic to assess operation constraints that will challenge the network UAM operations.
Summary
Advanced Air Mobility encompasses emerging aviation technologies that transport people and cargo between local, regional, or urban locations that are currently underserved by aviation and other transportation modalities. The disruptive nature of these technologies has pushed industry, academia, and governments to devote significant investments to understand their impact on airspace...
READ MORE
Applicability and surrogacy of uncorrelated airspace encounter models at low altitudes
July 1, 2021
Journal Article
Published in:
J. Air Transport., Vol. 29, No. 3, July-September 2021, pp. 137-41.
Summary
National Airspace System (NAS) is a complex and evolving system that enables safe and efficient aviation. Advanced air mobility concepts and new airspace entrants, such as unmanned aircraft, must integrate into the NAS without degrading overall safety or efficiency. For instance, regulations, standards, and systems are required to mitigate the risk of a midair collision between aircraft. Monte Carlo simulations have been a foundational capability for decades to develop, assess, and certify aircraft conflict avoidance systems. These are often validated through human-in-the-loop experiments and flight testing. For example, an update to the Traffic Collision Avoidance System (TCAS) mandated for manned aircraft was validated in part using this approach [1]. For many aviation safety studies, manned aircraft behavior is represented using the MIT Lincoln Laboratory statistical encounter models [2–5]. The original models [2–4] were developed from 2008 to 2013 to support safety simulations for altitudes above 500 feet above ground level (AGL). However, these models were not sufficient to assess the safety of smaller unmanned aerial systems (UAS) operations below 500 feet AGL and fully support the ASTM F38 and RTCA SC-147 standards efforts. In response, newer models [5–7] with altitude floors below 500 feet AGL have been in development since 2018. Many of the models assume that aircraft behavior is uncorrelated and not dependent on air traffic services or nearby aircraft. The models were trained using observations of cooperative aircraft equipped with transponders, but data sources and assumptions vary. The newer models are organized by aircraft types of fixed-wing multi-engine, fixed-wing single engine, and rotorcraft, whereas the original models do not consider aircraft type. Our research objective was to compare the various uncorrelated models of conventional aircraft and identify how the models differ. Particularly if models of rotorcraft were sufficiently different from models of fixed-wing aircraft to require type-specific models. The scope of this work was limited to altitudes below 5000 feet AGL, the expected altitude ceiling for many new airspace entrants. The scope was also informed by the Federal Aviation Administration (FAA) UAS Integration Office and Alliance for System Safety of UAS through Research Excellence (ASSURE). The primary contribution is guidance on which uncorrelated models to leverage when evaluating the performance of a collision avoidance system designed for low-altitude operations, such as prescribed by the ASTM F3442 detect and avoid standard for smaller UAS [8]. We also address which models can be surrogates for non-cooperative aircraft without transponders. All models and software used are publicly available under open source licenses [9].
Summary
National Airspace System (NAS) is a complex and evolving system that enables safe and efficient aviation. Advanced air mobility concepts and new airspace entrants, such as unmanned aircraft, must integrate into the NAS without degrading overall safety or efficiency. For instance, regulations, standards, and systems are required to mitigate the...
READ MORE
Towards the next generation operational meteorological radar
July 1, 2021
Journal Article
Published in:
Bull. Am. Meteorol. Soc., Vol. 102, No. 7, July 2021, pp. E1357-82.
Topic:
R&D area:
R&D group:
Summary
This article summarizes research and risk reduction that will inform acquisition decisions regarding NOAA's future national operational weather radar network. A key alternative being evaluated is polarimetric phased-array radar (PAR). Research indicates PAR can plausibly achieve fast, adaptive volumetric scanning, with associated benefits for severe-weather warning performance. We assess these benefits using storm observations and analyses, observing system simulation experiments, and real radar-data assimilation studies. Changes in the number and/or locations of radars in the future network could improve coverage at low altitude. Analysis of benefits that might be so realized indicates the possibility for additional improvement in severe weather and flash-flood warning performance, with associated reduction in casualties. Simulations are used to evaluate techniques for rapid volumetric scanning and assess data quality characteristics of PAR. Finally, we describe progress in developing methods to compensate for polarimetric variable estimate biases introduced by electronic beam-steering. A research-to-operations (R2O) strategy for the PAR alternative for the WSR-88D replacement network is presented.
Summary
This article summarizes research and risk reduction that will inform acquisition decisions regarding NOAA's future national operational weather radar network. A key alternative being evaluated is polarimetric phased-array radar (PAR). Research indicates PAR can plausibly achieve fast, adaptive volumetric scanning, with associated benefits for severe-weather warning performance. We assess these...
READ MORE
Advanced Air Mobility assessment framework: FY20 Homeland Protection and Air Traffic Control Technical Investment Program
March 24, 2021
Project Report
Published in:
MIT Lincoln Laboratory Report TIP-145
R&D area:
R&D group:
Summary
Advanced Air Mobility encompasses emerging aviation technologies that transport people and cargo between local, regional, or urban locations that are currently underserved by aviation and other transportation modalities. The disruptive nature of these technologies has pushed industry, academia, and governments to devote significant investments to understand their impact on airspace risk, operational procedures, and passengers. A flexible framework was designed to assess the operational viability of these technologies and the sensitivity to a variety of assumptions. This framework is used to simulate an initial AAM implementation scenario in New York City. This scenario was created by replacing a portion of NYC taxi requests with electric vertical takeoff and landing vehicles. The framework was used to assess the sensitivity of this scenario to a variety of system assumption.
Summary
Advanced Air Mobility encompasses emerging aviation technologies that transport people and cargo between local, regional, or urban locations that are currently underserved by aviation and other transportation modalities. The disruptive nature of these technologies has pushed industry, academia, and governments to devote significant investments to understand their impact on airspace...
READ MORE
NASA Airspace Integration Detect and Avoid Phase 2: Safety Risk Management Simulation Plan
December 18, 2020
Project Report
Published in:
MIT Lincoln Laboratory Report
Summary
RTCA has been developing Minimum Operational Performance Standards (MOPS) for Detect and Avoid (DAA) and Command and Control (C2) systems as part of Special Committee – 228 (SC-228). The Phase 1 MOPS were published in 2017 and a Phase 2 effort to revise and extend the Phase 1 MOPS is ongoing. In order for the MOPS to be fully utilized, they must be evaluated by the FAA employing the FAA's Safety Risk Management (SRM) process. In order to support the SRM process, there is a need for simulation data focused on the safety of DAA encounters. This analysis focuses on gathering information to validate the use of SC-228 MOPS compliant DAA and C2 systems to enable routine UAS operations in the National Airspace System (NAS) without a chase aircraft or visual observers. The scope of this effort aligns with the SC-228 Terms of Reference (TOR) that can be generally characterized as UAS flying IFR and receiving ATC separation services. This analysis evaluates the system safety in mixed classes B, C, D, E, and G airspaces, and includes IFR, VFR, Cooperative, and Non-Cooperative aircraft. This analysis plan describes the four analysis tasks (Section 2) and the simulation plan (Section 3) that will be executed to accomplish these tasks. It is expected that this analysis plan will be extended to include the analysis results and become the final deliverable.
Summary
RTCA has been developing Minimum Operational Performance Standards (MOPS) for Detect and Avoid (DAA) and Command and Control (C2) systems as part of Special Committee – 228 (SC-228). The Phase 1 MOPS were published in 2017 and a Phase 2 effort to revise and extend the Phase 1 MOPS is...
READ MORE
Adaptive stress testing: finding likely failure events with reinforcement learning
December 1, 2020
Journal Article
Published in:
J. Artif. Intell. Res., Vol. 69, 2020, pp. 1165-1201.
Summary
Finding the most likely path to a set of failure states is important to the analysis of safety critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due to the complex stochastic environment in which the system operates. As a result, safety validation is not only concerned about whether a failure can occur, but also discovering which failures are most likely to occur. This article presents adaptive stress testing (AST), a framework for finding the most likely path to a failure event in simulation. We consider a general black box setting for partially observable and continuous-valued systems operating in an environment with stochastic disturbances. We formulate the problem as a Markov decision process and use reinforcement learning to optimize it. The approach is simulation-based and does not require internal knowledge of the system, making it suitable for black-box testing of large systems. We present different formulations depending on whether the state is fully observable or partially observable. In the latter case, we present a modified Monte Carlo tree search algorithm that only requires access to the pseudorandom number generator of the simulator to overcome partial observability. We also present an extension of the framework, called differential adaptive stress testing (DAST), that can find failures that occur in one system but not in another. This type of differential analysis is useful in applications such as regression testing, where we are concerned with finding areas of relative weakness compared to a baseline. We demonstrate the effectiveness of the approach on an aircraft collision avoidance application, where a prototype aircraft collision avoidance system is stress tested to find the most likely scenarios of near mid-air collision.
Summary
Finding the most likely path to a set of failure states is important to the analysis of safety critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due...
READ MORE
A quantitatively derived NMAC analog for smaller unmanned aircraft systems based on unmitigated collision risk
November 19, 2020
Journal Article
Published in:
Preprints, 19 November 2020.
Summary
The capability to avoid other air traffic is a fundamental component of the layered conflict management system to ensure safe and efficient operations in the National Airspace System. The evaluation of systems designed to mitigate the risk of midair collisions of manned aircraft are based on large-scale modeling and simulation efforts and a quantitative volume defined as a near midair collision (NMAC). Since midair collisions are difficult to observe in simulation and are inherently rare events, basing evaluations on NMAC enables a more robust statistical analysis. However, an NMAC and its underlying assumptions for assessing close encounters with manned aircraft do not adequately consider the different characteristics of smaller UAS-only encounters. The primary contribution of this paper is to explore quantitative criteria to use when simulating two or more smaller UASs in sufficiently close proximity that a midair collision might reasonably occur and without any mitigations to reduce the likelihood of a midair collision. The criteria assumes a historically motivated upper bound for the collision likelihood and subsequently identify the smallest possible NMAC analogs. We also demonstrate the NMAC analogs can be used to support modeling and simulation activities.
Summary
The capability to avoid other air traffic is a fundamental component of the layered conflict management system to ensure safe and efficient operations in the National Airspace System. The evaluation of systems designed to mitigate the risk of midair collisions of manned aircraft are based on large-scale modeling and simulation...
READ MORE
Frequency of ADS-B equipped manned aircraft observed by the OpenSky Network
November 12, 2020
Conference Paper
Published in:
8th OpenSky Symp. 2020, Online, 12–13 November 2020.
Summary
To support integration of unmanned aerial systems into the airspace, the low altitude airspace needs to be characterized. Identifying the frequency of different aircraft types, such as rotorcraft or fixed wing single engine, given criteria such as altitude, airspace class, or quantity of seats can inform surveillance requirements, flight test campaigns, or simulation safety thresholds for detect and avoid systems. We leveraged observations of Automatic Dependent Surveillance-Broadcast (ADS-B) equipped aircraft by the OpenSky Network for this characterization.
Summary
To support integration of unmanned aerial systems into the airspace, the low altitude airspace needs to be characterized. Identifying the frequency of different aircraft types, such as rotorcraft or fixed wing single engine, given criteria such as altitude, airspace class, or quantity of seats can inform surveillance requirements, flight test...
READ MORE