Publications
Tagged As
A framework to improve evaluation of novel decision support tools
July 16, 2020
Conference Paper
Published in:
11th Intl. Conf. on Applied Human Factors and Ergonomics, AHFE, 16-20 July 2020.
Summary
Organizations that introduce new technology into an operational environment seek to improve some aspect of task conduct through technology use. Many organizations rely on user acceptance measures to gauge technology viability, though misinterpretation of user feedback can lead organizations to accept non-beneficial technology or reject potentially beneficial technology. Additionally, teams that misinterpret user feedback can spend time and effort on tasks that do not improve either user acceptance or operational task conduct. This paper presents a framework developed through efforts to transition technology to the U.S. Transportation Command (USTRANSCOM). The framework formalizes aspects of user experience with technology to guide organization and development team research and assessments. The case study is examined through the lens of the framework to illustrate how user-focused methodologies can be employed by development teams to systematically improve development of new technology, user acceptance of new technology, and assessments of technology viability.
Summary
Organizations that introduce new technology into an operational environment seek to improve some aspect of task conduct through technology use. Many organizations rely on user acceptance measures to gauge technology viability, though misinterpretation of user feedback can lead organizations to accept non-beneficial technology or reject potentially beneficial technology. Additionally, teams...
READ MORE
Airport surface traffic management decision support - perspectives based on tower flight data manager prototype
May 20, 2013
Project Report
Published in:
MIT Lincoln Laboratory Report ATC-398
Topic:
R&D area:
R&D group:
Summary
This report describes accomplishments and insights gathererd during the development of decision support tools as part of the Terminal Flight Data Manager (TFDM) program. This work was performed by MIT Lincoln Laboratory and sponsored by the Federal Aviation Administration (FAA). The TFDM program integrated flight data, aircraft surveillance, information on weather and traffic flow constraints, and other data required to optimize airport conguration and arrival/departure management functions. The prototype has been evaluated in both human-in-the-loop simulations, and during operational tests at Dallas/Fort Worth (DFW) International Airport. In parallel, the Laboratory estimated future national operational benefits for TFDM decision support functions, using analysis and performance data gathered from major airports in the US. This analysis indicated that the greatest potential operational benefits would come from decision support tools that facilitate: i) managing runway queues and sequences, ii) tactical management of flight routes and times, impacted by weather and traffic constraints, and iii) managing airport configuration changes. Evaluation of TFDM prototype decision support functions in each of these areas provided valuable insights relative to the maturity of current capabilities and research needed to close performance gaps.
Summary
This report describes accomplishments and insights gathererd during the development of decision support tools as part of the Terminal Flight Data Manager (TFDM) program. This work was performed by MIT Lincoln Laboratory and sponsored by the Federal Aviation Administration (FAA). The TFDM program integrated flight data, aircraft surveillance, information on...
READ MORE
Data-driven evaluation of a flight re-route air traffic management decision-support tool
July 21, 2012
Conference Paper
Published in:
Applied Human Factors and Ergonomics Conf., 21 July 2012.
Topic:
R&D area:
R&D group:
Summary
Air traffic delays in the U.S. are problematic and often attributable to convective (thunderstorms) weather. Air traffic management is complex, dynamic, and influenced by many factors such as projected high volume of departures and uncertain forecast convective weather at airports and in the airspace. To support the complexities of making a re-route decision, which is one solution to mitigate airspace congestion, a display integrating convective weather information with departure demand predictions was prototyped jointly by MIT Lincoln Laboratory and the MITRE Corporation. The tool was deployed to twelve air traffic facilities involved in handling New York area flights for operational evaluation during the summer of 2011. Field observations, data mining and analyses were conducted under both fair and convective weather conditions. The system performance metrics chosen to evaluate the tool's effectiveness in supporting re-route decisions include predicted wheels-off error, predicted wheels-off forecast spread, and hourly departure fix demand forecast spread. The wheels-off prediction errors were near zero for half the flights across all days, but the highest 10% errors exceeded 30 minutes on convective weather days. The wheels-off forecast spread exceeded 30 minutes for 25% of forecasts on convective weather days. The hourly departure demand forecast spread was 9 flights or less for 50% of departures across all days except one. Six out of the seven days having the highest hourly departure demand forecast spreads occurred in the presence of long-lived weather impacts.
Summary
Air traffic delays in the U.S. are problematic and often attributable to convective (thunderstorms) weather. Air traffic management is complex, dynamic, and influenced by many factors such as projected high volume of departures and uncertain forecast convective weather at airports and in the airspace. To support the complexities of making...
READ MORE
Evaluation of the Integrated Departure Route Planning (IDRP) Tool 2011 prototype
May 3, 2012
Project Report
Published in:
MIT Lincoln Laboratory Report ATC-388
Topic:
R&D area:
R&D group:
Summary
The Integrated Departure Route Planning (IDRP) tool combines convective weather impact forecasts from the Route Availability Planning Tool (RAPT) with departure demand forecasts from the MITRE tfmCore system to aid traffic managers in formulating plans to mitigate volume congestion in fair weather and during convective weather impacts. An initial prototype was deployed in the summer of 2010 for a very limited field evaluation. A second, more comprehensive field evaluation of the "Phase 2" IDRP prototype was performed in the summer of 2011. The key focus of IDRP is the planning and implementation of departure reroutes to avoid weather impacts and volume congestion on departure fixes and routes. This evaluation assesses three facets of the IDRP prototype critical to the successful realization of its concept of operations: 1. performance of weather impact forecasts from RAPT and departure demand forecasts from tfmCore, 2. effectiveness of reroute decisions, and 3. potential impacts on procedures and decision making based on observations of IDRP use in the field. The evaluation concludes with suggestions for future enhancements to improve the performance and realization of potential benefits in operational use of IDRP.
Summary
The Integrated Departure Route Planning (IDRP) tool combines convective weather impact forecasts from the Route Availability Planning Tool (RAPT) with departure demand forecasts from the MITRE tfmCore system to aid traffic managers in formulating plans to mitigate volume congestion in fair weather and during convective weather impacts. An initial prototype...
READ MORE
A tree-based ensemble method of the prediction and uncertainty quantification of aircraft landing times
January 22, 2012
Conference Paper
Published in:
10th Conf. on Artificial and Computational Intelligence, 22 January 2012.
Topic:
R&D area:
R&D group:
Summary
Accurate aircraft landing time predictions provide situational awareness for air traffic controllers, enable decision support algorithms and gate management planning. This paper presents a new approach for estimation of landing times using a tree-based ensemble method, namely Quantile Regression Forests. This method is suitable for real-time applications, provides robust and accurate predictions of landing times, and yields prediction intervals for individual flights, which provide a natural way of quantifying uncertainty. The approach was tested for arrivals at Dallas/Fort Worth International Airport over a range of days with a variety of operational conditions.
Summary
Accurate aircraft landing time predictions provide situational awareness for air traffic controllers, enable decision support algorithms and gate management planning. This paper presents a new approach for estimation of landing times using a tree-based ensemble method, namely Quantile Regression Forests. This method is suitable for real-time applications, provides robust and...
READ MORE
Evaluation of Consolidated Storm Prediction for Aviation (CoSPA) 0-8 hour convective weather forecast using the airspace flow program blockage-based capacity forecast ("The Matrix")
November 9, 2011
Project Report
Published in:
MIT Lincoln Laboratory Report ATC-385
Topic:
R&D area:
R&D group:
Summary
The CoSPA 0-8 hour convective weather forecast provides deterministic forecast products that can be used by strategic traffic management planners, and can be readily translated into forecasts of aviation capacity impacts for use in automated decision support tools. An operational CoSPA prototype was evaluated at several FAA Air Traffic Control facilities during the summer of 2010. As part of this evaluation, CoSPA forecasts were translated into forecasts of capacity impacts on traffic flows through two Flow Constrained Areas (FCAA05 and FCAA08) commonly used to control arrival traffic into the highly congested northeastern United States. This report describes an objective and operationally relevant evaluation of the accurancy of CoSPA-based forecasts of FCA capacity.
Summary
The CoSPA 0-8 hour convective weather forecast provides deterministic forecast products that can be used by strategic traffic management planners, and can be readily translated into forecasts of aviation capacity impacts for use in automated decision support tools. An operational CoSPA prototype was evaluated at several FAA Air Traffic Control...
READ MORE
Concept of operations for the Integrated Departure Route Planning (IDRP) tool
September 12, 2011
Project Report
Published in:
MIT Lincoln Laboratory Report ATC-379
Topic:
R&D area:
R&D group:
Summary
A concept of operations for the Integrated Departure Route Planner (IDRP) tool is proposed to address issues in the area of departure route management. By combining information about weather and departure demand, IDRP can both identify potential demand/capacity imbalances and recommend a rerouting option, if appropriate. To effectively implement IDRP into the operational environment, a twophase approach is suggested. The first phase appends IDRP functionality onto the CIWS/RAPT platform, combining departure demand information with the convective weather information, creating a live prototype. This initial phase allows a gradual introduction of functionality into an existing display and enables the gathering of operational data to appropriately evolve IDRP to phase 2. The second phase involves introducing airline route preferences, along with any operational improvements discovered during the initial phase.
Summary
A concept of operations for the Integrated Departure Route Planner (IDRP) tool is proposed to address issues in the area of departure route management. By combining information about weather and departure demand, IDRP can both identify potential demand/capacity imbalances and recommend a rerouting option, if appropriate. To effectively implement IDRP...
READ MORE
Collaborative Decision Making (CDM) Weather Evaluation Tool (WET) operational bridging for convective weather: demonstrations and implementation plans
January 25, 2011
Conference Paper
Published in:
2nd Aviation, Range and Aerospace Meteorology Special Symp. on Weather-Air Traffic Management Integration, 22-27 January 2011.
Topic:
R&D area:
R&D group:
Summary
The purpose of this manuscript is twofold. First, it provides a review of the activities of the Weather Evaluation Team (WET), which is part of a joint Industry and Federal Aviation Administration (FAA) effort called Collaborative Decision Making (CDM). Over ten years ago, the predecessor to the WET, the Weather Action Group (WAG), developed a process that involved industry and government participants in the production of the Collaborative Convective Forecast Product (CCFP). The CCFP was developed in response to the need of industry and government Air Traffic Managers to have a common forecast of convective information used in their decision making processes. In light of the concepts introduced by the Next Generation Air Transportation System (NextGen), the CCFP could be viewed as one predecessor to the Single Authoritative Source. During the period 2008 through 2010, the WET worked on a task to increase the amount of detail as well as extend its forecast time period. At the same time, new automated convective forecasts were developed and introduced to both the WET and Traffic Flow Management (TFM) community. The manuscript includes a description of how the WET has strived to integrate both Human-in-the-Loop (HITL) and fully automated products, including the Localized Aviation Model Output Statistics (MOS) Product (LAMP)/CCFP Hybrid (LCH), the Aviation Impact Guidance for Convective Weather, the Corridor Integrated Weather System (CIWS) and the Consolidated Storm Prediction for Aviation (CoSPA). The second purpose of this manuscript is to introduce the new concept called Operational Bridging. The WET first discussed Operational Bridging at the 2010 Friends and Partners of Aviation Weather (FPAW) Vision Meeting in July, 2010. Foundational materials such as a Concept of Operations (CONOPS) and a demonstration plan are now being developed by the WET. Operational Bridging is first described from within a meteorology-centric view of the CCFP forecast process. Not only does this allow the new concept to be further defined, it also lays out a transition path for the current CCFP. Operational Bridging is next described from the broader conceptual perspective of Air Traffic Management (ATM)/Weather Integration, and two key areas are explored: 1) the role of the CDM weather community in the area of automated probabilistic and deterministic convective weather forecast information and 2) the integration of probabilistic forecast information into both strategic and deterministic (tactical) ATM decision making process.
Summary
The purpose of this manuscript is twofold. First, it provides a review of the activities of the Weather Evaluation Team (WET), which is part of a joint Industry and Federal Aviation Administration (FAA) effort called Collaborative Decision Making (CDM). Over ten years ago, the predecessor to the WET, the Weather...
READ MORE
Modeling convective weather avoidance of arrivals in the terminal airspace
January 25, 2011
Conference Paper
Published in:
2nd Aviation, Range, and Aerospace Meteorology Special Symp. on Weather-Air Traffic Management Integration, 22-27 January 2011.
Topic:
R&D area:
R&D group:
Summary
For several years the NASA sponsored Convective Weather Avoidance Model (CWAM) has been under development at Lincoln Lab to correlate pilot behavior with observable weather parameters available from convective weather systems. To date, the current CWAM has focused primarily on the enroute airspace used by aircraft at cruise altitude. At these altitudes there is a strong correlation between the observable echo tops from the Corridor Integrated Weather System (CIWS) and the probability that a pilot will deviate around weather. The CWAM has lead to the development of a Weather Avoidance Field (WAF) that combines the echo tops and vertically integrated liquid (VIL) into a probabilistic forecast of the likelihood of pilot deviation. In recent years the WAF has become widely acceptance as a reliable indicator of the impact of convective weather on air traffic operations. This paper will explore the adaptation of the CWAM into the terminal airspace with a focus on the weather impact on arrival decision making. A database of convective weather impacts on several major terminals from 2009 has been collected and identification of the impact on arriving aircraft has begun. Past studies of terminal weather impact have identified aircraft that penetrated severe weather or made clear deviations around convective cells within the terminal. This study will expand the definition of an impact to identify pilot decision making occurring outside of the terminal with regard to the expected weather impact upon arrival in the terminal. Examples include rerouting to an alternate corner post, holding in enroute airspace, or diverting to an alternate airport when weather is expected along the planned terminal trajectory. These types of terminal weather avoidance decisions can often be made many miles outside of the terminal. The enroute CWAM uses spatial filters applied to the echo tops and VIL to obtain the best correlation between the weather and pilot behavior. This paper will evaluate the current CWAM filters and identify alternate spatial filters or additional weather products that may best correlate pilot decision making in the terminal. Ultimately the goal of this work is provide ATC managers and automated decision supports tools with a weather avoidance field for effective management of convective weather in terminal airspace.
Summary
For several years the NASA sponsored Convective Weather Avoidance Model (CWAM) has been under development at Lincoln Lab to correlate pilot behavior with observable weather parameters available from convective weather systems. To date, the current CWAM has focused primarily on the enroute airspace used by aircraft at cruise altitude. At...
READ MORE
The Route Availability Planning Tool (RAPT): evaluation of departure management decision support in New York during the 2008 convective weather season
June 29, 2009
Conference Paper
Published in:
8th USA/Europe Air Traffic Management Research and Development Sem., ATM 2009, 29 June - 2 July 2009.
Topic:
R&D area:
R&D group:
Summary
Severe weather avoidance programs (SWAP) due to convective weather are common in many of the busiest terminal areas in the US National Airspace System (NAS). In order to make efficient use of available airspace in rapidly evolving convective weather, it is necessary to predict the impacts of the weather on key resources (e.g., departure and arrival routes and fixes), with frequent updates as the weather changes. Currently, this prediction is a mental process that imposes a significant cognitive burden on air traffic managers. As a result, air traffic management in SWAP is often inconsistent and decisions result in less than optimal performance. The Route Availability Planning Tool (RAPT) is a prototype automated decision support tool, intended to help air traffic managers in convective weather SWAP, by predicting the impacts of convective weather on departure routes. Originally deployed in New York in August, 2002, RAPT has recently undergone two field evaluations (2007 and 2008) in order to test and refine its concept of operations, evaluate the accuracy and usefulness of its decision guidance, and estimate observed and potential delay reduction benefits that may be achieved as a result of its use. This paper presents the results of the 2008 performance evaluation, focusing on the concept of operations and the quality of decision support guidance. A second paper [1] presents analyses of delay reduction benefits and the operational decision making environment in which RAPT is deployed.
Summary
Severe weather avoidance programs (SWAP) due to convective weather are common in many of the busiest terminal areas in the US National Airspace System (NAS). In order to make efficient use of available airspace in rapidly evolving convective weather, it is necessary to predict the impacts of the weather on...
READ MORE