Publications

Refine Results

(Filters Applied) Clear All

Artificial intelligence: short history, present developments, and future outlook, final report

Summary

The Director's Office at MIT Lincoln Laboratory (MIT LL) requested a comprehensive study on artificial intelligence (AI) focusing on present applications and future science and technology (S&T) opportunities in the Cyber Security and Information Sciences Division (Division 5). This report elaborates on the main results from the study. Since the AI field is evolving so rapidly, the study scope was to look at the recent past and ongoing developments to lead to a set of findings and recommendations. It was important to begin with a short AI history and a lay-of-the-land on representative developments across the Department of Defense (DoD), intelligence communities (IC), and Homeland Security. These areas are addressed in more detail within the report. A main deliverable from the study was to formulate an end-to-end AI canonical architecture that was suitable for a range of applications. The AI canonical architecture, formulated in the study, serves as the guiding framework for all the sections in this report. Even though the study primarily focused on cyber security and information sciences, the enabling technologies are broadly applicable to many other areas. Therefore, we dedicate a full section on enabling technologies in Section 3. The discussion on enabling technologies helps the reader clarify the distinction among AI, machine learning algorithms, and specific techniques to make an end-to-end AI system viable. In order to understand what is the lay-of-the-land in AI, study participants performed a fairly wide reach within MIT LL and external to the Laboratory (government, commercial companies, defense industrial base, peers, academia, and AI centers). In addition to the study participants (shown in the next section under acknowledgements), we also assembled an internal review team (IRT). The IRT was extremely helpful in providing feedback and in helping with the formulation of the study briefings, as we transitioned from datagathering mode to the study synthesis. The format followed throughout the study was to highlight relevant content that substantiates the study findings, and identify a set of recommendations. An important finding is the significant AI investment by the so-called "big 6" commercial companies. These major commercial companies are Google, Amazon, Facebook, Microsoft, Apple, and IBM. They dominate in the AI ecosystem research and development (R&D) investments within the U.S. According to a recent McKinsey Global Institute report, cumulative R&D investment in AI amounts to about $30 billion per year. This amount is substantially higher than the R&D investment within the DoD, IC, and Homeland Security. Therefore, the DoD will need to be very strategic about investing where needed, while at the same time leveraging the technologies already developed and available from a wide range of commercial applications. As we will discuss in Section 1 as part of the AI history, MIT LL has been instrumental in developing advanced AI capabilities. For example, MIT LL has a long history in the development of human language technologies (HLT) by successfully applying machine learning algorithms to difficult problems in speech recognition, machine translation, and speech understanding. Section 4 elaborates on prior applications of these technologies, as well as newer applications in the context of multi-modalities (e.g., speech, text, images, and video). An end-to-end AI system is very well suited to enhancing the capabilities of human language analysis. Section 5 discusses AI's nascent role in cyber security. There have been cases where AI has already provided important benefits. However, much more research is needed in both the application of AI to cyber security and the associated vulnerability to the so-called adversarial AI. Adversarial AI is an area very critical to the DoD, IC, and Homeland Security, where malicious adversaries can disrupt AI systems and make them untrusted in operational environments. This report concludes with specific recommendations by formulating the way forward for Division 5 and a discussion of S&T challenges and opportunities. The S&T challenges and opportunities are centered on the key elements of the AI canonical architecture to strengthen the AI capabilities across the DoD, IC, and Homeland Security in support of national security.
READ LESS

Summary

The Director's Office at MIT Lincoln Laboratory (MIT LL) requested a comprehensive study on artificial intelligence (AI) focusing on present applications and future science and technology (S&T) opportunities in the Cyber Security and Information Sciences Division (Division 5). This report elaborates on the main results from the study. Since the...

READ MORE

Adversarial co-evolution of attack and defense in a segmented computer network environment

Published in:
Proc. Genetic and Evolutionary Computation Conf. Companion, GECCO 2018, 15-19 July 2018, pp. 1648-1655.

Summary

In computer security, guidance is slim on how to prioritize or configure the many available defensive measures, when guidance is available at all. We show how a competitive co-evolutionary algorithm framework can identify defensive configurations that are effective against a range of attackers. We consider network segmentation, a widely recommended defensive strategy, deployed against the threat of serial network security attacks that delay the mission of the network's operator. We employ a simulation model to investigate the effectiveness over time of different defensive strategies against different attack strategies. For a set of four network topologies, we generate strong availability attack patterns that were not identified a priori. Then, by combining the simulation with a coevolutionary algorithm to explore the adversaries' action spaces, we identify effective configurations that minimize mission delay when facing the attacks. The novel application of co-evolutionary computation to enterprise network security represents a step toward course-of-action determination that is robust to responses by intelligent adversaries.
READ LESS

Summary

In computer security, guidance is slim on how to prioritize or configure the many available defensive measures, when guidance is available at all. We show how a competitive co-evolutionary algorithm framework can identify defensive configurations that are effective against a range of attackers. We consider network segmentation, a widely recommended...

READ MORE

Classifier performance estimation with unbalanced, partially labeled data

Published in:
Proc. Machine Learning Research, Vol. 88, 2018, pp. 4-16.

Summary

Class imbalance and lack of ground truth are two significant problems in modern machine learning research. These problems are especially pressing in operational contexts where the total number of data points is extremely large and the cost of obtaining labels is very high. In the face of these issues, accurate estimation of the performance of a detection or classification system is crucial to inform decisions based on the observations. This paper presents a framework for estimating performance of a binary classifier in such a context. We focus on the scenario where each set of measurements has been reduced to a score, and the operator only investigates data when the score exceeds a threshold. The operator is blind to the number of missed detections, so performance estimation targets two quantities: recall and the derivative of precision with respect to recall. Measuring with respect to error in these two metrics, simulations in this context demonstrate that labeling outliers not only outperforms random labeling, but often matches performance of an adaptive method that attempts to choose the optimal data for labeling. Application to real anomaly detection data confirms the utility of the approach, and suggests direction for future work.
READ LESS

Summary

Class imbalance and lack of ground truth are two significant problems in modern machine learning research. These problems are especially pressing in operational contexts where the total number of data points is extremely large and the cost of obtaining labels is very high. In the face of these issues, accurate...

READ MORE

Development and application of spherically curved charge-coupled device imagers

Summary

Operation of a CCD imager on a curved focal surface offers advantages to flat focal planes, especially for lightweight, relatively simple optical systems. The first advantage is that the modulation transfer function can approach diffraction-limited performance for a spherical focal surface employed in large field-of-view or large-format imagers. The second advantage is that a curved focal surface maintains more uniform illumination as a function of radius from the field center. Examples of applications of curved imagers, described here, include a small compact imager and the large curved array used in the Space Surveillance Telescope. The operational characteristics and mechanical limits of an imager deformed to a 15 mm radius are also described.
READ LESS

Summary

Operation of a CCD imager on a curved focal surface offers advantages to flat focal planes, especially for lightweight, relatively simple optical systems. The first advantage is that the modulation transfer function can approach diffraction-limited performance for a spherical focal surface employed in large field-of-view or large-format imagers. The second...

READ MORE

Analysis of delay causality at Newark International Airport

Published in:
4th USA/Europe Air Traffic Management R&D Seminar, 3-7 December 2001.

Summary

Determining causes of aviation delay is essential for formulating and evaluating approaches to reduce air traffic delays. An analysis was conducted of large weather-related delays at Newark International Airport (EWR), which, located in the heart of the congested northeast corridor of the United States, is an airport with a significant number of delays. Convective weather and reduced ceiling and visibility were found to be the leading contributors to large delays at EWR between September 1998 and August 2001. It was found that 41% of the cumulative arrival delay (delay relative to schedule) on days in this period averaging more than 15 minutes of delay per arrival occurred on days characterized by convective weather either within or at considerable distances from the New York terminal area. Of the remaining delays, 28% occurred on days characterized by low ceiling/visibility conditions, while 14% occurred on fair weather days with high surface winds, and 2% were caused by distant non-convective storms. Known causes other than weather accounted for 9% of the delays, and causes were unknown for 6%. When delay types (airborne, gate, taxi -out etc.) were categorized by the type of weather causing the delay, it was found that: (1) departure delays (gate + taxi-out) were much larger than arrival delays for thunderstorms in the NY terminal area and (2) taxi-out delays were the dominant type when delays were caused by distant convective weather. The fraction of total delay time explained by pre-planned Ground Delay Programs (GDP) rose sharply during 2000, accounting for over 40% of total the arrival delay that year, and then decreased slightly in 2001. On days with thunderstorms in the NY TRACON, arrival and departure delays were significantly higher during the year (2000) that GDPs were used most frequently.
READ LESS

Summary

Determining causes of aviation delay is essential for formulating and evaluating approaches to reduce air traffic delays. An analysis was conducted of large weather-related delays at Newark International Airport (EWR), which, located in the heart of the congested northeast corridor of the United States, is an airport with a significant...

READ MORE

Tactical convective weather decision support to complement "strategic" traffic flow management for convective weather

Author:
Published in:
46th Annual Air Traffic Control Association Conf. Proc., 4-8 November 2001, pp. 98-102.

Summary

Delay increases during the months of the year characterized by thunderstorms have been the principal cause of the dramatic delay growth in the US aviation system over the past 3 years, as shown in Figure 1. In 2000, the key new initiative for reducing these convective weather delays was "strategic" traffic flow management (TFM) through the Collaborative Convective Forecast Product (CCFP), the Strategic Planning Team, and Collaborative Routing (CR). This "strategic" approach has been quite successful in improving operations. However, in congested airspace, the inability to accurately forecast convective weather impacts requires a complementary tactical weather decision support capability. This paper describes terminal and enroute weather prediction systems plus traffic flow management and automation decision support tools to complement the strategic approach.
READ LESS

Summary

Delay increases during the months of the year characterized by thunderstorms have been the principal cause of the dramatic delay growth in the US aviation system over the past 3 years, as shown in Figure 1. In 2000, the key new initiative for reducing these convective weather delays was "strategic"...

READ MORE

Designing a terminal area bird detection and monitoring system based on ASR-9 data

Published in:
3rd Joint Annual Meeting Bird Strike Committee, 27-30 August 2001.

Summary

Conflicts between birds and commercial aircraft are a noteworthy problem at both large and small airports [Cleary, 1999]. The risk factor for United States airports continues to increase due to the steady rise in take-off/landings and bird populations. There is a significant bird strike problem in the terminal area as shown by the incidents reported in the National Bird Strike Database [Cleary and Dolbeer, 1999]. The focus of bird strike mitigation in the past has centered primarily on wildlife management techniques. Recently, an Avian Hazard Advisory System (AHAS) has been developed to reduce the risks of bird strikes to military operations [Kelly, 1999]. This system uses a mosaic of data obtained from the Next Generation Weather Radar (NEXRAD). This sensor serves as an excellent tool for enroute bird advisories due to the radar coverage provided across the majority of the United States. However, its utility in the airport terminal environment is limited due to the slow update rate and the fact that the distance of most NEXRADs from the airport results in beam heights that are too high to detect low-altitude birds over the airport. The Federal Aviation Administration (FAA) operates two radar systems – the Terminal Doppler Weather Radar (TDWR), and the Airport Surveillance Radar (ASR-9) -- that could be used to help monitor bird activity at an airport in order to: 1. Provide continuously updated information on locations and approximate numbers of birds in flocks roosting or feeding on or near an airfield; 2. Generate real-time warnings of bird activity for dissemination to pilots of landing or departing aircraft by air traffic controllers or by direct data link. The TDWR provides wind shear warnings in the terminal area to enhance safety, while the ASR-9’s primary function is air traffic control. Both of these systems have been shown to detect biological echoes as well. Characteristics of the two radar systems have been examined and compared to determine capabilities for bird detection. Amongst other favorable factors, the high update rate and on-airport locale makes the ASR-9 a highly desirable platform for a bird detection and warning system for the terminal area. Data from an ASR-9 at Austin TX (AUS) equipped with a Weather Systems Processor (WSP) have been analyzed to assess the ASR-9's capability to detect and monitor bird activity. The WSP add-on provides a variety of radar base data products similar to those that would be available on all ASR-9s as part of an ASR-9 Service Life Extension Program (SLEP) currently underway. The Austin airport area is subject to large flocks of wintering migratory birds as well as a resident population of bats in close proximity to the airport. Radar data, visual observations and bird strike information during periods of active bird/bat movements have been collected for this study. An automated processing algorithm called the Terminal Avian Hazard Advisory System (TAHAS) is being developed to detect and track roosting and migratory birds using ASR-9 data. A key challenge will be the ability to discriminate biological from non-biological targets based on variables such as vertical continuity, variance or spectral width, and horizontal velocity distribution.
READ LESS

Summary

Conflicts between birds and commercial aircraft are a noteworthy problem at both large and small airports [Cleary, 1999]. The risk factor for United States airports continues to increase due to the steady rise in take-off/landings and bird populations. There is a significant bird strike problem in the terminal area as...

READ MORE

Positive charge in the stratiform cloud of a mesoscale convective system

Published in:
J. Geophys. Res., Vol. 106, No. D1, 16 January 2001, pp. 1157-1163.

Summary

A balloon sounding of electric field in the trailing stratiform cloud of a bow echo mesoscale convective system reveals only two substantial in-cloud positive charge regions. These charge regions are located at altitudes of 5.1-5.6 km and 6.4-6.8 km, above the level of 0 degree C at 4.2 km. The two positive charge regions are the likely sources of six positive cloud-to-ground flashes with large peak currents (>32 kA) that occurred within 60 km of the balloon during its flight. The amount of charge transferred by three of these positive flashes that made Q bursts is calculated in the range of 97-196 C. Flashes of this sort are known to produce sprites and elves in the mesosphere. The positive charge regions in this stratiform cloud are substantially lower than the 10-km altitude commonly assumed for the positive charge in many sprite modeling studies.
READ LESS

Summary

A balloon sounding of electric field in the trailing stratiform cloud of a bow echo mesoscale convective system reveals only two substantial in-cloud positive charge regions. These charge regions are located at altitudes of 5.1-5.6 km and 6.4-6.8 km, above the level of 0 degree C at 4.2 km. The...

READ MORE

A statistical analysis of approach winds at capacity-restricted airports

Published in:
19th AIAA/IEEE Digital Avionics Systems Conf., Vol. 1, 7-13 October 2000, pp. 3.E.4-1 - 3.E.4-7.

Summary

Many major airports in the U.S. rely on simultaneous approaches to closely-spaced parallel (CSP) runways to maintain a high airport acceptance rate. During Visual Meteorological Conditions (VMC), aircraft are able to utilize both runways by making side-by-side landings and are able to meet the demands of heavy volume. However, when conditions deteriorate to marginal-VMC or Instrument Meteorological Conditions (IMC), side-by-side approaches are not possible due to the inherent safety concerns associated with lowered ceilings and visibilities. This situation is severely limiting to an airport's capacity and can create large delays and increased costs. Various ideas have been suggested that would facilitate the simultaneous use of CSP runways during low ceiling and visibility (LCV) conditions at capacity-restricted airports. This report addresses the specific scenario of a pair of approaching aircraft being staggered by some longitudinal distance. This situation alleviates the collision hazard presented by LCV conditions, but also introduces the hazard of a wake vortex encounter, particularly if the following aircraft is downwind of the leading aircraft.
READ LESS

Summary

Many major airports in the U.S. rely on simultaneous approaches to closely-spaced parallel (CSP) runways to maintain a high airport acceptance rate. During Visual Meteorological Conditions (VMC), aircraft are able to utilize both runways by making side-by-side landings and are able to meet the demands of heavy volume. However, when...

READ MORE

Learning from incidents - what the machine can learn

Published in:
Int. Society of Air Safety Investigators Conf., ISASI, 2-6 October 2000.

Summary

Aviation weather refers to any type of weather that can affect the operation of an aircraft – anything from a brief delay in departure to a catastrophic accident during flight. Wind shear and events associated with convective weather were recognized as an aviation hazard long before Dr. Theodore Fujita began publishing his now-famous treatises. On July 28, 1943, American Airlines Flight 63 from Cleveland, Ohio, USA to Nashville, Tennessee crashed after the pilot lost control of the Douglas DC3. The pilots and numerous passengers were fatally injured. The aircraft was destroyed by impact and post crash fire. The weather report at the time included warnings for storms, heavy rain, lightning and severe turbulence. The Civil Aeronautics Board found that the probable cause was a loss of control of the aircraft due to unusually severe turbulence and violent downdraft caused by a thunderstorm. In the ten-year period from 1987 through 1996, 24% of all U.S. accidents were judged to be "weather related". For the twenty-year period 1976 to 1996 fully 43% of U.S. accidents were judged to have involved wind or wind shear, and 2.3 % thunderstorm, although the two data elements are not mutually exclusive. In the U.S., approximately 82% of accidents are general aviation; the rest are air carriers and commuters of various types. When general aviation accidents are negated, and only air carriers are considered, wind and wind shear issues account for 9.5% of accidents. The Weather Systems Processor (WSP) has been developed to reduce the impact of severe weather conditions on air traffic by providing information concerning weather conditions in the airport terminal environment. WSP provides warnings to air traffic controllers and supervisors of hazardous wind shear and microburst events in the terminal area, forecasts the arrival of gust fronts, and tracks thunderstorms, providing a complete picture of current and future terminal area hazardous weather conditions that may impact runway and airport usage. Common weather situation awareness allows Terminal Approach, Tower Controllers and other traffic management personnel to jointly plan with confidence and safely manage more arrivals and departures with less delay. Knowledge of the location, severity and movement of hazardous weather allows dynamic adjustments to be made in routing aircraft to runways, approach and departure corridors, terminal arrival and departure transition areas (i.e. gate-posts) and other air routes.
READ LESS

Summary

Aviation weather refers to any type of weather that can affect the operation of an aircraft – anything from a brief delay in departure to a catastrophic accident during flight. Wind shear and events associated with convective weather were recognized as an aviation hazard long before Dr. Theodore Fujita began...

READ MORE