In this paper, we summarize contemporary approaches to quantifying convective weather delay reduction benefits. We outline a program to develop a significantly improved capability that can be used to assess benefits of specific systems. This program may potentially accomplish weather impact normalization for studies of National Airspace System (NAS) performance in handling convective weather. Benefits quantification and NAS performance assessment have become very important topics for the aviation weather community. In an era of significant federal government and airline budget austerity for civil aviation investments, it is essential to quantitatively demonstrate delay reduction benefits of improved weather decision support systems. Major FAA initiatives stress the importance of quantitative system performance metrics that are related to aviation weather. For example, the new FAA Air Traffic Organization (ATO) and the FAA Flight Plan 2004-08 both have quantitative performance metrics that are closely related to reducing convective weather delays. The Flight Plan metrics include: "Improving the percentage of all flights arriving within 15 minutes of schedule at the 35 OEP airports by 7%, as measured from the FY2000-02 baseline, through FY08," and "Maintaining average en route travel times among the eight major metropolitan areas." The ATO metrics include the percentage of on time gate arrivals and the fraction of departures that are delayed greater than 40 minutes. However, these metrics currently do not account for the differences in convective weather severity and changes in the NAS. The dramatic increase in convective season delays in 2004 (Figure 1) due to a combination of severe weather, increases in overall demand, and specific airport issues has demonstrated that one needs to consider these other factors. Approaches to delay reduction quantification that were viewed as successful and valid several years ago are no longer considered to be adequate by either by the FAA investment analysis branch or by the Office of Management and Budget (OMB). The paper proceeds as follows. We first discuss at some length the mechanisms by which convective weather delay occurs in the NAS and highlight challenges in delay reduction assessment. We consider this to be very important since one needs to understand how the system operates if one is to design an effective, accurate performance assessment system. We then consider benefits quantification based on feedback from experienced users of a system. Feedback on "average" benefits from a system at the end of a test period was used to generate delay reduction estimates for the Integrated Terminal Weather System (ITWS) and the Weather and Radar Processor (WARP). This end-of-season interview approach was not viable in highly congested en route airspace. Hence, a new approach was developed for Corridor Integrated Weather System (CIWS) benefits assessment that uses real time observations of product usage during convective weather events coupled with in depth analysis of specific cases. Next, we discuss the problems that arise when one attempts to quantify delay reduction benefits by comparing flight delays before and after the Integrated Terminal Weather System (ITWS) system was deployed at Atlanta Hartsfield International Airport (ATL). This seemingly simple approach has proven very difficult in practice because the convective weather events in the different time periods are virtually never identical and because other aspects of the NAS may also have changed (e.g., user demand, fleet mix, and other systems that impact convective weather delays). It has become clear that one needs a quantitative model for the NAS that would permit adjustment of measured delay data to account at least for the differences in convective weather and changes in user demand (i.e., flight scheduling). The paper concludes with recommendations for measuring near term benefits of various classes of convective weather decision support systems.