Goal 4. Provide Robust, Accessible and Innovative Information Services and Tools
NCAR Strategic Priority: Conducting Computer Science, Computational Science, Applied Mathematics, Statistics, and Numerical Methods R&D
Extreme Value Methods as an Integrative Element in Weather and Climate Impacts Assessment
Figure Legend. Trend in lowest summer minimum temperature at Phoenix, AZ.
Extreme events are the focus of much attention in assessments of the impacts of weather and climate. Yet the methods used still tend to be rather ad hoc, lacking any theoretical justification. This project calls for a more sustained effort to establish extreme value theory as the appropriate foundation on which to base statistical aspects of such assessments, thus playing an integrative role in impact assessment.
FY2008 Accomplishments
Work continued to address the question of how best to extend the concepts of “return period” and “return level,” used to communicate the uncertainty about an extreme event, to a changing climate. Whether the extension was based on the expected waiting time or on the expected number of events did not necessarily have much effect on the actual return level. But either extension results in some rather undesirable features under climate change, suggesting that an entirely new concept is needed instead.
Work was initiated by SOARS protégé Marcus Walter to examine whether extreme value theory could be used to devise more appropriate methods for detecting trends in the characteristics of weather spells, such as heat waves. The statistical modeling of the frequency of occurrence, duration, and peak intensity of heat waves was essentially completed. As a test, the methods were applied to Phoenix, AZ, a city which has experienced marked trends in temperature extremes as part of the urban heat island. We learned that extreme value theory does provide a satisfactory statistical model for these characteristics of heat waves at Phoenix.
FY2009 Plans
Future work includes the completion of the development of better statistical methods for detecting trends in heat waves. It remains to develop a statistical model, based on extremal modeling with covariates, for the clustering of high temperatures within a hot spell. One constraint is that the methods be simple enough to implement within the Extremes Toolkit (www.isse.ucar.edu/extremevalues/evtk.html). The toolkit is currently used by researchers from a variety of disciplines, including atmospheric sciences and statistics, to detect trends in weather and climate extremes. But it can only treat simple extreme events, such as a single hot day, not more complex forms of extreme events such as heat waves whose societal impacts are potentially much greater.
Geographical Information Systems (GIS)
Monthly mean temperature for January 1896 from the CCSM AR4 20th Century Experiment before and after statistical downscaling.
The Geographic Information Systems (GIS) Strategic Initiative at the National Center for Atmospheric Research (NCAR) is an interdisciplinary effort to foster collaborative science, spatial data interoperability, and knowledge sharing with GIS. Working toward the definition, standards and interoperability of atmospheric information for usable science, the GIS Initiative is: 1) Conducting research integrating the Earth system and social sciences through spatial analysis and interoperability of georeferenced information; 2) Supporting the use of GIS as both an analysis, and an infrastructure tool in atmospheric research; 3) Improving usable science and knowledge sharing between science groups, educators and stakeholders; and 4) Addressing broader issues of spatial data management, interoperability, and geoinformatics within the geosciences.
FY2008 Accomplishments
- Many NCAR research projects, where spatial analysis and accurate georeferenced data are important, benefited from GIS expertise provided by the initiative staff. With more than a hundred GIS users in all NCAR laboratories, the projects range from atmospheric chemistry to homeland security to societal impacts of climate change.
- The GIS Climate Change Scenarios project continues to provide access to global data sets of climate change scenarios generated for the Intergovernmental Panel on Climate Change, Assessment Report 4 (IPCC AR4) by the Community Climate System Model (CCSM-3). Near 3000 users from 108 countries have accessed the CCSM-3 climate predictions through the GIS data portal. In FY08 we developed a tutorial for portal users as a demonstration of appropriate use of the CCSM projections.
- Responding to users’ needs, the GIS Initiative, in collaboration with CISL-IMAGe, added a downscaled product of CCSM climate change projections to the GIS portal. Statistically downscaled temperature and precipitation projections for the United States at 4.5 km spatial resolution are now available to a GIS community (see figure).
- The GIS Initiative continued to work towards improvement of compatibility, accessibility and accuracy of atmospheric data and models across scales with regard to the GIS environment. We investigated methods of conversion between sphere-based models and ellipsoid-based data for GIS analysis. This work will continue in FY09. In FY2008, the GIS Initiative, together with ESRI conducted a survey of GIS professionals about their uses of weather and climate information. A manuscript describing results of the survey is now in preparation.
- Collaboration with EPA, University of Wisconsin, Arizona State University and Stratus Consulting on exploring spatial patterns of societal vulnerability to extreme heat in Phoenix, AZ and Philadelphia, PA continues through preparation of a manuscript for submission to Environmental Health Perspectives. In FY08 we initiated a pilot project on improved assessment of societal vulnerability and adaptive capacity through integration of quantitative and qualitative data and information in a GIS. This Phoenix-based project will continue in FY09.
- The 5th Atmospheric Data Modeling workshop, conducted in January 2008, was focused on preparation of an edited volume on the Atmospheric Data Model. This compendium will demonstrate a range of GIS applications in the atmospheric and oceanographic sciences.
- The GIS Initiative continued to support Open Geospatial Consortium (OGC) standards-based developments with the goal to improve interoperability between atmospheric and GIS data and tools.
FY2009 Plans
The GIS Initiative will be working on: 1) integrating social and natural sciences through integration of quantitative and qualitative data in a GIS; 2) improving spatial accuracy and usability of atmospheric models for terrestrial and societal applications; and3) building capacity through GIS-focused education ladder for UCAR community, usable GIS lab, community workshops, conferences, joint publications and enabling interdisciplinary research.
The main focus in FY09 will be on the following tasks:
Integration of quantitative and qualitative data and information in a GIS. Societal vulnerability depends on local contexts and interests, including socio-economic, political and cultural factors. Quantitative vulnerability assessments are limited by available geocoded census data and cannot reflect individual conceptions and actions, interactions within neighborhoods, communities and places, and governance processes. Examples from the literature indicate that integration of quantitative and qualitative information can strengthen our understanding of societal vulnerability and provide a more nuanced analysis of the local environment and social processes. Through a pilot project, based in Phoenix, AZ and in collaboration with ASU and Arizona public health organizations, we will develop methods to integrate quantitative and qualitative data in a GIS framework and investigate how transferable this integration across locales and applicability of the method for integrated regional adaptation strategies.
3rd Community Workshop on GIS in Weather, Climate and Impacts. The GIS Initiative will be hosting the 3rd community workshop on GIS in Weather, Climate and Impacts. The theme of this workshop will be "Integrating social and natural systems in GIS." The first two NCAR GIS workshops (http://www.gis.ucar.edu/workshops.jsp) were focused on integration of GIS with atmospheric data across scales, setting research directions for GIScience that are relevant for atmospheric, weather and climate research, and tackling spatial data management challenges. 3rd workshop will review the progress that has been made in AtmoGIS research, applications and data standards, and will add another dimension to the discussion: quantitative and qualitative social science data across scales. Topics of presentations and discussions will include: 1) atmospheric data needs for spatial societal research and applications; 2) Spatial data needs for integrative assessments and Earth System modeling; 3) methodologies for integration of natural and social science data (both quantitative and qualitative) for weather hazards preparedness and climate change adaptation.
WRF-GIS. Earlier work of the GIS initiative ensured compatibility of WRF model outputs, available in NetCDF CF format, with the GIS software. In FY09, the GIS Initiative will explore use of web services and web GIS technologies with regard to distribution of WRF model outputs in the context of societal impacts of weather. Methods of conversion between sphere-based models and ellipsoid-based data for GIS analysis will be investigated further to determine the full potential of the sphere-ellipsoidal shift and its impacts upon the output positional accuracy for numeric weather models as well as, the use of these models to determine impacts upon society, and infrastructure.
Atmospheric Data Model. Since 2003, the GIS Initiative has been leading a community effort in developing an Atmospheric Data Model for GIS. This effort enabled many research projects, NWS operations, national and international collaborations and adopting NetDCF data format by ESRI. Most recent activity of the atmospheric data modeling working group (participants from NOAA labs and NWS regional offices, NASA JPL, NCAR, Unidata, universities and private sector) focuses on preparation of an edited volume for publication through ESRI Press. This publication will summarize atmospheric data modeling concepts and use cases and provide GIS professional community with a number of cases illustrating GIS applications in meteorology, climatology, atmospheric remote sensing, and marine applications.
NCAR science support. Approximately, 150 scientific, engineering and educational NCAR/UCAR staff members use services and facilities provided by the GIS Initiative. GIS methods, tools and data have been used in a wide range of projects and contributed to a number of peer-reviewed publications. Interactions among the staff through the workshops, UCAR GIS User Group meetings and GIS seminars resulted in increased collaboration and knowledge transfer between research groups and divisions. In FY09 GIS initiative will continue supporting NCAR staff, being involved in interdisciplinary research projects, and identifying future directions that would most effectively contribute to NCAR’s core mission and usability of NCAR’s science.
Modeling Weather Extremes
Median (1980-1999) annual maximum Wmax (m/s) * shear (m/s) from the reanalysis (left) and CCSM3 (right). CAPE is the convective available potential energy and Shear is the magnitude of the vector difference between the surface and 6-km estimated wind.
In the third and fourth Assessment Reports of the Intergovernmental Panel on Climate Change, discussions of the impacts of climate change on severe thunderstorms have been limited to comments regarding the difficulty of using storm report databases to determine if changes have taken place historically. Because convective storms occur on very fine spatial scales, it is not possible to directly resolve such phenomena from coarse-scale global datasets. However, large-scale indicators can be employed to study trends in environments that are conducive to such severe weather. Initial work in this area has focused on identification of useful measures of large-scale environments that are relevant for severe thunderstorm formation based on NCAR global model reanalysis data. These data have been used to investigate trends in the large scale environmental characteristics, as well as spatial and extreme value distribution attributes. The approaches developed for the reanalysis data will be extended to Global Climate Model (GCM) projections of future climate, to determine the expected characteristics of severe weather environments associated with future climate change scenarios. Large-scale indicators of severe weather analyzed so far involve the product of 0-6 km wind shear (Shear) and Convective Available Potential Energy (CAPE).
This work has led to identification of several statistical challenges as well as new areas for research. Statistical challenges include methods for modeling extreme values in a spatial context; addressing the issue of multiple comparisons inherent in working with gridded data; and making inferences about changes in distribution parameters.
FY2008 Accomplishments
Initial analysis was started using the output of the NCAR Community Climate System Model (CCSM3), AB1 scenario being used to represent the current climate as well as comparison of results associated with the reanalysis data. Results to date were presented by Eric Gilleland at the annual Joint Statistical Meetings (JSM) of the American Statistical Association (ASA) in an invited session. Some important feedback about the analyses was gleaned from R. L. Smith and H. E. Brooks regarding (i) difficulties in estimating the parameters of the generalized extreme value (GEV) for these data, and (ii) the weak relationship between the product of CAPE and shear and severe weather. For the first issue, it was suggested that Bayesian estimation should be tried as the GEV-estimated return levels were everywhere severely under estimating the observed empirical return levels, though the empirical level estimates cannot be fully trusted. Although new to Bayesian estimation, Gilleland has begun to implement a Gibbs sampling procedure in order to estimate the parameters under the Bayesian paradigm. For the second point, Brooks very recently discovered a stronger connection between concurrently high values of shear and a transformation of CAPE, Wmax, with severe weather. This new large-scale indicator will allow for stronger and more meaningful conclusions from the project. Results presented at the JSM were submitted to the conference proceedings, which was possible because the presentation session was an invited one. These analyses continue to be in collaboration with H. Brooks at the NOAA National Severe Storms Laboratory and P. Marsh at the University of Oklahoma.
FY2009 Plans
Bayesian estimation will be explored for finding reasonable estimates of the GEV parameter, and trend analysis for the CCSM3 will be performed to determine whether the characteristics of CAPE and Shear based on GCM output for an unchanged climate are consistent with the characteristics of these parameters in the reanalysis data. Initial diagnostics suggest that there are large discrepancies. However, these discrepancies are less pronounced after transformation of CAPE to Wmax, and the overall spatial structure of the two fields is similar (see Figure).
NextGeneration Network Enabled Weather (NNEW)
Since its inception in 1998, the Aviation Digital Data Service (ADDS) has emphasized user-friendly and intuitive weather graphics to provide users with enhanced weather situational awareness. Within the ADDS system, there exists a fundamental infrastructure for serving weather data in a network-centric manner. The ADDS Flight Path Tool (FPT), for example, enables human users to visualize specific portions from immense volumes of data using a highly interactive software application and an Internet browser.
While graphical presentations of weather data are useful to human users, there is also a need for machine-to-machine data dissemination to provide data to decision support tools and systems that manage air traffic. This need has been identified as a critical element in the U.S. Joint Planning and Development Office’s drive to transform the nation’s air transportation system by 2025. The research and development challenge here is to provide four-dimensional weather data using standard formats for the request and delivery of important weather information.
To address this problem, NCAR/RAL has teamed with MIT-Lincoln Labs and NOAA’s Global Systems Division to explore and implement standards-based, net-centric data access. The goal of this effort is to create a virtual weather database spanning more than one physical location, organization, and data system. Starting in late 2007, each laboratory has been conducting research-to-operations work on open standards and technologies to provide uniform access to weather information.
FY2008 Accomplishments
Efforts in FY2008 focused on development of a standards-based implementation of a service to distribute traditional non-gridded products to aviation users. These new products, as well as gridded products from previous years, are now accessible via a single NNEW registry/repository that allows them all to be treated transparently as a virtual database. Additionally, the registry/repository includes ontological mappings and enhanced standards based metadata for service endpoints and data products.
FY2009 Plans
Future work will include the participation of all three labs in developing reference implementations for web services for these new four-dimensional weather capabilities. The NNEW labs will also continue to produce demonstrations of basic capabilities, including data retrieval, registry/repository utility, and flight hazard avoidance.
Each lab will also take responsibility for a portion of the data products included in the Initial Operating Capability (IOC) plan for the NextGeneration air transit system. They are also tasked with assisting in developing data formats and operational capabilities for the organizations that will be operationally providing these products. Additionally, the labs will participate in broad national and international collaborations to develop an extensible, flexible, and efficient non-gridded data model with associated formats.
Tools for Studying Biocomplexity in the Environment
Soybean field in Argentine Pampas
The goal of this activity is to develop tools that facilitate end-to-end uncertainty analysis in assessments of the economic impacts of seasonal to decadal variations in climate on agriculture. It is part of two broader multidisciplinary projects, the first on “Climate, Agriculture, and Complexity in the Argentine Pampas” and the subsequent one on “Interactions between Changing Climate and Technological Innovations in Agricultural Decision-Making: Implications for Land use and Sustainability of Production Systems”, involving collaboration among researchers from various institutions in the U.S. and Argentina from a wide range of disciplines including agronomy, economics, hydrology, political science, psychology, and statistics.
FY2008 Accomplishments
Previous work included the development of an improved stochastic weather generator for producing scenarios of daily weather, based on the statistical approach known as generalized linear modeling (GLM). During the past year, this framework was used to implement methods, consistent with the results of extreme value theory, to improve the simulation of extreme high precipitation amounts by stochastic weather generators. For more information on the GLM weather generator, see www.image.ucar.edu/~eva/GLMwgen/index.shtml.
FY2009 Plans
Future work will focus on the development of tools to produce scenarios of daily weather for input to agronomic models that reflect the variation in climate on annual to decadal time scales. These tools will be based in part on hidden Markov models to allow for persistent regime shifts in climate. Apparent regime shifts in climate have contributed to decisions by farmers to switch from one crop to another (e.g., corn to soybeans).
Verification Research and Development
Example of “3-dimensional” precipitation objects identified by the MODE forecast evaluation method. For this example, the time dimension is in the vertical. Colors change from West to East.
Example of a “quilt” plot showing precipitation forecast performance as a function of scale – where the scale is represented by the convolution radius and threshold values used by MODE to define spatial objects. Warm colors indicate better performance. Values for Threshold-Radius combinations in the upper right are not meaningful due to small sample sizes. The plot suggests that performance is best for scales with objects defined using a radius greater than 10 grid squares and thresholds less than 0.15 inches.
Forecast verification and evaluation activities typically are based on relatively simple metrics that measure the meteorological performance of forecasts and forecasting systems. Metrics such as the Probability of Detection, Root Mean Squared Error, and Equitable Threat Score provide information that is useful for monitoring changes in performance of single aspects of forecast performance with time. However, they generally do not provide information that can be used to improve the forecasts, or that can be used by end users (including forecasters) for decision making. Moreover, it is possible for forecasts that are quite useful – including high resolution forecasts – to have very poor scores when evaluated by using these standard metrics. In response to these limitations, the RAL Verification Group develops improved verification approaches and tools that provide more meaningful and relevant information about forecast performance. The focus of this effort is on diagnostic, statistically valid approaches, including object-based evaluation of precipitation and convective forecasts and other approaches (e.g., distribution-based) that can provide more useful information – for forecast developers as well as forecast users – about forecast performance; and the development and application of methods (e.g., confidence intervals) to estimate the uncertainty associated with verification measures. In addition, the RAL Verification Group develops forecast evaluation tools that are available for use by members of the operational, model development, and research communities.
Development and dissemination of new forecast verification approaches requires research and application in several areas, including statistical methods, exploratory data analysis, statistical inference, pattern recognition, and evaluation of user needs.
FY2008 Accomplishments
Versions 1.0 and 1.1 of the Model Evaluation Tools, developed by RAL’s Developmental Testbed Center (DTC), were released to the community in January and July, respectively. This state-of-the-art set of model evaluation tools includes traditional verification methods as well as new methods that have been developed for spatial forecasts. MET has been widely implemented by the university community and by government and commercial users.
One of the new forecast evaluation methods included in MET is the Method for Object-based Diagnostic Evaluation (MODE), which was developed by NCAR scientists. MODE has recently been extended to examine forecast performance as a function of scale and to consider temporal attributes of forecasts (see figures).
The ongoing intercomparison project (ICP) for spatial forecast verification methods involves scientists from around the world who are developing new methods for evaluation of spatial forecasts. The ICP focuses on the evaluation of several real and idealized forecast cases, and was facilitated by a workshop among participants in April 2008. Results of the applications of each method will be described in papers to be included in an upcoming special collection of papers for the journal, Weather and Forecasting. The ICP is also expected to lead to discussions within the verification and NWP communities regarding development of a protocol for judging when new verification methods are ready to be applied in operational settings.
We also continued advocacy and outreach efforts through participation in, and leadership of the WMO’s Joint Working Group on Verification, numerous conferences and workshops, statistical support for forecast evaluation studies undertaken by the RAL Developmental Testbed Center (DTC), and applications of MODE by various scientists at NCAR and in the wider atmospheric science community. The RAL Verification Group also organized and hosted a verification workshop on state-of-the-art verification methods in April 2008, with participation by international verification and numerical weather prediction experts.
FY2009 Plans
Version 2.0 of MET will be implemented in winter 2008. The new version will include methods for probabilistic forecasts, new data formats, and additional spatial verification approaches. A workshop will be held to identify new methods that should be included in future MET versions, for example, to facilitate the evaluation of ensemble forecasts.
Attributes of the MODE approach will be more thoroughly investigated, including extensive examination of the impacts of variations in spatial scale, as represented by the parameters used to define objects. New diagnostic methods will be developed to summarize object attribute comparisons, to provide greater understanding of model performance. The approach for incorporating the time dimension in MODE analyses will be further investigated and enhanced.
The results of the verification method intercomparison project (ICP) will be summarized in journal articles. A special collection of papers will be created for the journal, Weather and Forecasting. In addition, we will investigate the desirability of follow-on projects and possible establishment of a “verification method testbed.”