Publications
Benchmarking the processing of aircraft tracks with triples mode and self-scheduling
September 20, 2021
Conference Paper
Published in:
2021 IEEE High Performance Extreme Computing Conf., HPEC, 20-24 September 2021.
Summary
As unmanned aircraft systems (UASs) continue to integrate into the U.S. National Airspace System (NAS), there is a need to quantify the risk of airborne collisions between unmanned and manned aircraft to support regulation and standards development. Developing and certifying collision avoidance systems often rely on the extensive use of Monte Carlo collision risk analysis simulations using probabilistic models of aircraft flight. To train these models, high performance computing resources are required. We've prototyped a high performance computing workflow designed and deployed on the Lincoln Laboratory Supercomputing Center to process billions of observations of aircraft. However, the prototype has various computational and storage bottlenecks that limited rapid or more comprehensive analyses and models. In response, we've developed a novel workflow to take advantage of various job launch and task distribution technologies to improve performance. The workflow was benchmarked using two datasets of observations of aircraft, including a new dataset focused on the environment around aerodromes. Optimizing how the workflow was parallelized drastically reduced the execution time from weeks to days.
Summary
As unmanned aircraft systems (UASs) continue to integrate into the U.S. National Airspace System (NAS), there is a need to quantify the risk of airborne collisions between unmanned and manned aircraft to support regulation and standards development. Developing and certifying collision avoidance systems often rely on the extensive use of...
READ MORE
Benefits of realist ontologies to systems engineering
September 11, 2021
Conference Paper
Published in:
Joint Ontology Workshops 2021 Episode VII: The Bolzano Summer of Knowledge, JOWO 2021, 11-18 September 2021.
Summary
Applied ontologies have been used more and more frequently to enhance systems engineering. In this paper, we argue that adopting principles of ontological realism can increase the benefits that ontologies have already been shown to provide to the systems engineering process. Moreover, adopting Basic Formal Ontology (BFO), an ISO standard for top-level ontologies from which more domain specific ontologies are constructed, can lead to benefits in four distinct areas of systems engineering: (1) interoperability, (2) standardization, (3) testing, and (4) data exploitation. Reaping these benefits in a model-based systems engineering (MBSE) context requires utilizing an ontology's vocabulary when modeling systems and entities within those systems. If the chosen ontology abides by the principles of ontological realism, a semantic standard capable of uniting distinct domains, using BFO as a hub, can be leveraged to promote greater interoperability among systems. As interoperability and standardization increase, so does the ability to collect data during the testing and implementation of systems. These data can then be reasoned over by computational reasoners using the logical axioms within the ontology. This, in turn, generates new data that would have been impossible or too inefficient to generate without the aid of computational reasoners.
Summary
Applied ontologies have been used more and more frequently to enhance systems engineering. In this paper, we argue that adopting principles of ontological realism can increase the benefits that ontologies have already been shown to provide to the systems engineering process. Moreover, adopting Basic Formal Ontology (BFO), an ISO standard...
READ MORE
Demand and capacity modeling for advanced air mobility
August 2, 2021
Conference Paper
Published in:
AIAA Aviation 2021 Conf., 2-6 August 2021.
Topic:
R&D area:
R&D group:
Summary
Advanced Air Mobility encompasses emerging aviation technologies that transport people and cargo between local, regional, or urban locations that are currently underserved by aviation and other transportation modalities. The disruptive nature of these technologies has pushed industry, academia, and governments to devote significant investments to understand their impact on airspace risk, operational procedures, and passengers. A flexible framework was designed to assess the operational viability of these technologies and the sensitivity to a variety of assumptions. This framework is used to simulate air taxi traffic within New York City by replacing a portion of the city's taxi requests with trips taken with electric vertical takeoff and landing vehicles and evaluate the sensitivity of passenger trip time to a variety of system wide assumptions. In particular, the paper focuses on the impact of the passenger capacity, landing site vehicle capacity, and fleet size. The operation density is then compared with the current air traffic to assess operation constraints that will challenge the network UAM operations.
Summary
Advanced Air Mobility encompasses emerging aviation technologies that transport people and cargo between local, regional, or urban locations that are currently underserved by aviation and other transportation modalities. The disruptive nature of these technologies has pushed industry, academia, and governments to devote significant investments to understand their impact on airspace...
READ MORE
Applicability and surrogacy of uncorrelated airspace encounter models at low altitudes
July 1, 2021
Journal Article
Published in:
J. Air Transport., Vol. 29, No. 3, July-September 2021, pp. 137-41.
R&D area:
R&D group:
Summary
National Airspace System (NAS) is a complex and evolving system that enables safe and efficient aviation. Advanced air mobility concepts and new airspace entrants, such as unmanned aircraft, must integrate into the NAS without degrading overall safety or efficiency. For instance, regulations, standards, and systems are required to mitigate the risk of a midair collision between aircraft. Monte Carlo simulations have been a foundational capability for decades to develop, assess, and certify aircraft conflict avoidance systems. These are often validated through human-in-the-loop experiments and flight testing. For example, an update to the Traffic Collision Avoidance System (TCAS) mandated for manned aircraft was validated in part using this approach [1]. For many aviation safety studies, manned aircraft behavior is represented using the MIT Lincoln Laboratory statistical encounter models [2–5]. The original models [2–4] were developed from 2008 to 2013 to support safety simulations for altitudes above 500 feet above ground level (AGL). However, these models were not sufficient to assess the safety of smaller unmanned aerial systems (UAS) operations below 500 feet AGL and fully support the ASTM F38 and RTCA SC-147 standards efforts. In response, newer models [5–7] with altitude floors below 500 feet AGL have been in development since 2018. Many of the models assume that aircraft behavior is uncorrelated and not dependent on air traffic services or nearby aircraft. The models were trained using observations of cooperative aircraft equipped with transponders, but data sources and assumptions vary. The newer models are organized by aircraft types of fixed-wing multi-engine, fixed-wing single engine, and rotorcraft, whereas the original models do not consider aircraft type. Our research objective was to compare the various uncorrelated models of conventional aircraft and identify how the models differ. Particularly if models of rotorcraft were sufficiently different from models of fixed-wing aircraft to require type-specific models. The scope of this work was limited to altitudes below 5000 feet AGL, the expected altitude ceiling for many new airspace entrants. The scope was also informed by the Federal Aviation Administration (FAA) UAS Integration Office and Alliance for System Safety of UAS through Research Excellence (ASSURE). The primary contribution is guidance on which uncorrelated models to leverage when evaluating the performance of a collision avoidance system designed for low-altitude operations, such as prescribed by the ASTM F3442 detect and avoid standard for smaller UAS [8]. We also address which models can be surrogates for non-cooperative aircraft without transponders. All models and software used are publicly available under open source licenses [9].
Summary
National Airspace System (NAS) is a complex and evolving system that enables safe and efficient aviation. Advanced air mobility concepts and new airspace entrants, such as unmanned aircraft, must integrate into the NAS without degrading overall safety or efficiency. For instance, regulations, standards, and systems are required to mitigate the...
READ MORE
Adaptive stress testing: finding likely failure events with reinforcement learning
December 1, 2020
Journal Article
Published in:
J. Artif. Intell. Res., Vol. 69, 2020, pp. 1165-1201.
R&D area:
R&D group:
Summary
Finding the most likely path to a set of failure states is important to the analysis of safety critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due to the complex stochastic environment in which the system operates. As a result, safety validation is not only concerned about whether a failure can occur, but also discovering which failures are most likely to occur. This article presents adaptive stress testing (AST), a framework for finding the most likely path to a failure event in simulation. We consider a general black box setting for partially observable and continuous-valued systems operating in an environment with stochastic disturbances. We formulate the problem as a Markov decision process and use reinforcement learning to optimize it. The approach is simulation-based and does not require internal knowledge of the system, making it suitable for black-box testing of large systems. We present different formulations depending on whether the state is fully observable or partially observable. In the latter case, we present a modified Monte Carlo tree search algorithm that only requires access to the pseudorandom number generator of the simulator to overcome partial observability. We also present an extension of the framework, called differential adaptive stress testing (DAST), that can find failures that occur in one system but not in another. This type of differential analysis is useful in applications such as regression testing, where we are concerned with finding areas of relative weakness compared to a baseline. We demonstrate the effectiveness of the approach on an aircraft collision avoidance application, where a prototype aircraft collision avoidance system is stress tested to find the most likely scenarios of near mid-air collision.
Summary
Finding the most likely path to a set of failure states is important to the analysis of safety critical systems that operate over a sequence of time steps, such as aircraft collision avoidance systems and autonomous cars. In many applications such as autonomous driving, failures cannot be completely eliminated due...
READ MORE
A quantitatively derived NMAC analog for smaller unmanned aircraft systems based on unmitigated collision risk
November 19, 2020
Journal Article
Published in:
Preprints, 19 November 2020.
R&D area:
R&D group:
Summary
The capability to avoid other air traffic is a fundamental component of the layered conflict management system to ensure safe and efficient operations in the National Airspace System. The evaluation of systems designed to mitigate the risk of midair collisions of manned aircraft are based on large-scale modeling and simulation efforts and a quantitative volume defined as a near midair collision (NMAC). Since midair collisions are difficult to observe in simulation and are inherently rare events, basing evaluations on NMAC enables a more robust statistical analysis. However, an NMAC and its underlying assumptions for assessing close encounters with manned aircraft do not adequately consider the different characteristics of smaller UAS-only encounters. The primary contribution of this paper is to explore quantitative criteria to use when simulating two or more smaller UASs in sufficiently close proximity that a midair collision might reasonably occur and without any mitigations to reduce the likelihood of a midair collision. The criteria assumes a historically motivated upper bound for the collision likelihood and subsequently identify the smallest possible NMAC analogs. We also demonstrate the NMAC analogs can be used to support modeling and simulation activities.
Summary
The capability to avoid other air traffic is a fundamental component of the layered conflict management system to ensure safe and efficient operations in the National Airspace System. The evaluation of systems designed to mitigate the risk of midair collisions of manned aircraft are based on large-scale modeling and simulation...
READ MORE
Frequency of ADS-B equipped manned aircraft observed by the OpenSky Network
November 12, 2020
Conference Paper
Published in:
8th OpenSky Symp. 2020, Online, 12–13 November 2020.
R&D area:
R&D group:
Summary
To support integration of unmanned aerial systems into the airspace, the low altitude airspace needs to be characterized. Identifying the frequency of different aircraft types, such as rotorcraft or fixed wing single engine, given criteria such as altitude, airspace class, or quantity of seats can inform surveillance requirements, flight test campaigns, or simulation safety thresholds for detect and avoid systems. We leveraged observations of Automatic Dependent Surveillance-Broadcast (ADS-B) equipped aircraft by the OpenSky Network for this characterization.
Summary
To support integration of unmanned aerial systems into the airspace, the low altitude airspace needs to be characterized. Identifying the frequency of different aircraft types, such as rotorcraft or fixed wing single engine, given criteria such as altitude, airspace class, or quantity of seats can inform surveillance requirements, flight test...
READ MORE
Multi-Agent Systems Collaborative Teaming (MASCOT) definition process to create specifications for Multi-Agent System (MAS) development
November 2, 2020
Conference Paper
Published in:
25th Intl. Command and Control Research and Technology Symp., ICCRTS 2020, 2-5 November 2020.
Summary
The US Army envisions heterogeneous teams of advanced machines and humans that will collaborate together to achieve a common mission goal. It is essential for commanders to quickly and effectively respond to dynamic mission environments with agile re-tasking and computerized aids for plan definition/redefinition, and to perform some tasks with bounded autonomy. Workload constraints limit an individual's ability to concurrently control many platforms, so some mission segments many need to be autonomous or to be quickly selected via a tactics playbook. Denied environments also dictate the need for machine participants in some mission segments to be autonomous (or semi-autonomous). A Multi-Agent System (MAS) provides a natural paradigm for describing a system of agents that work together in such environments. An agent can be a human or machine, but is generally a machine. Creating MAS systems and requirements has proved to be a formidable task due to mission complexities, the necessity to deal with unforeseen circumstances, and the general difficulty of defining autonomous behaviors. We define a process called Multi-Agent Systems Collaborative Teaming (MASCOT) Definition Process that starts with a Subject Matter Experts (SME), produces a set of agent specifications, and derives system requirements in sufficient detail to define a MAS that can be modeled in a test-bed, used for facilitation of a safety analysis, and produced into an actual system. The MASCOT process also enables concurrent development of an effects based ontology. We demonstrate the MASCOT process on an example case study to show the efficacy of our process.
Summary
The US Army envisions heterogeneous teams of advanced machines and humans that will collaborate together to achieve a common mission goal. It is essential for commanders to quickly and effectively respond to dynamic mission environments with agile re-tasking and computerized aids for plan definition/redefinition, and to perform some tasks with...
READ MORE
Method to characterize potential UAS encounters using open source data
November 1, 2020
Journal Article
Published in:
Aerospace, Vol. 7, No. 11, November 2020, art. no. 158.
Summary
As unmanned aerial systems (UASs) increasingly integrate into the US national airspace system, there is an increasing need to characterize how commercial and recreational UASs may encounter each other. To inform the development and evaluation of safety critical technologies, we demonstrate a methodology to analytically calculate all potential relative geometries between different UAS operations performing inspection missions. This method is based on a previously demonstrated technique that leverages open source geospatial information to generate representative unmanned aircraft trajectories. Using open source data and parallel processing techniques,we performed trillions of calculations to estimate the relative horizontal distance between geospatial points across sixteen locations.
Summary
As unmanned aerial systems (UASs) increasingly integrate into the US national airspace system, there is an increasing need to characterize how commercial and recreational UASs may encounter each other. To inform the development and evaluation of safety critical technologies, we demonstrate a methodology to analytically calculate all potential relative geometries...
READ MORE
TCAS II and ACAS Xa traffic and resolution advisories during interval management paired approach operations
October 11, 2020
Conference Paper
Published in:
2020 AIAA/IEEE 39th Digital Avionics Systems Conf., DASC, 11-15 October 2020.
Summary
Interval Management (IM) is an FAA Next-Gen Automatic Dependent Surveillance – Broadcast (ADS-B) In application designed to decrease the variability in spacing between aircraft, thereby increasing the efficiency of the National Airspace System (NAS). One application within IM is Paired Approach (PA). In a PA operation, the lead aircraft and trail aircraft are both established on final approach to dependent parallel runways with runway centerline spacing less than 2500 feet. The trail aircraft follows speed guidance from the IM Avionics to achieve and maintain a desired spacing behind the lead aircraft. PA operations are expected to require a new separation standard that allows the aircraft to be spaced more closely than current dependent parallel separation standards. The behavior of an airborne collision avoidance system, such as TCAS II or ACAS Xa, must be considered during a new operation such as PA, because the aircraft are so closely spaced. This analysis quantified TAs and RAs using TCAS II Change 7.1 and ACAS Xa software with simulated IM PA operations. The results show no RAs using either TCAS II Change 7.1 or ACAS Xa, negligible TAs using TCAS II Change 7.1, and acceptable numbers of TAs using ACAS Xa software during simulated PA operations.
Summary
Interval Management (IM) is an FAA Next-Gen Automatic Dependent Surveillance – Broadcast (ADS-B) In application designed to decrease the variability in spacing between aircraft, thereby increasing the efficiency of the National Airspace System (NAS). One application within IM is Paired Approach (PA). In a PA operation, the lead aircraft and...
READ MORE