Goal 4, Priority 2: Developing and Providing Advanced Services and Tools
For nearly 50 years, NCAR has provided a computational environment to satisfy the institution's overall mission of providing robust, reliable, accessible, innovative, and advanced services to the university community and the broader scientific community. NCAR also advances atmospheric science by creating integrated software and data system services that are at the heart of today's state-of-the-art observing systems. We provide classic computational services through our high performance computing production environment with mass storage, network connectivity, data analysis, visualization, research data stewardship, and data distribution services. In addition, NCAR continually evaluates opportunities to provide services designed to enable Earth System science for the entire community of users such as TeraGrid integration, experimental computing systems, and an Earth system knowledge environment. NCAR continually improves its observational data products and services to increase overall functionality, accessibility, reliability, and ease of maintenance. This work often has broad community impacts when the technologies are shared and distributed to outside users and agencies.
FY2007 Accomplishments
Click to enlarge.
Blueice is a supercomputer based on the IBM POWER5+ processor and the High Performance Switch communication technology. When it was installed, it effectively doubled the high-end computing resources available at NCAR. CISL bears the ongoing responsibility to provide the numerical simulation community with resources that offer the optimal combination of computational capability and capacity that can be used effectively. This facility supports the NCAR strategic priority of "Enhancing capability and capacity of NCAR supercomputing.
NCAR’s TeraGrid activities in FY2007 have already resulted in a number of new and interesting technical and scientific collaborations. NCAR demonstrated co-scheduling of Blue Gene/L systems with SDSC at Supercomputing 2006 in Tampa, Florida. CISL’s TeraGrid system administration staff has conducted tests of GPFS-WAN filesystem performance with SDSC, and has set up a Storage Resource Broker (SRB) instance with SDSC that enabled the first cross archiving of data between SDSC and NCAR to provide a semblance of disaster-recovery capability. This is being applied first to CISL’s research datasets. This activity is also helping to prototype Chronopolis, a strategic collaboration in long-term digital preservation between NCAR, SDSC, and others.
NCAR also continued to develop the next-generation user interface for its new metadata database tool and enhance its links to the Community Data Portal. For more information on this effort, see Priority 4 below.
Data Assimilation (DA) refers to combining observations with a numerical model to give an improved estimate of a process or physical variable. NCAR has a large number of researchers for whom DA is an essential part of their ongoing or planned research. IMAGe’s goal is to develop general methods and software to improve the DA usability for researchers. A centerpiece of this effort is the Data Assimilation Research Testbed (DART), a software system that allows a scientist to rapidly incorporate new models or new kinds of observations for assimilation.
An important activity is applying DART to specific science problems. During FY2007, work continued on assimilating GPS occultation observations (COSMIC program) and the application to the WRF single-column model. A highlight is the successful assimilation of MOPITT instrument radiance observations using the NCAR Community Atmosphere Model. The most important aspect of this accomplishment is integration of the tracer transport within the model so the advection is consistent with the model winds. Also, assimilation is directly based on a version of the satellite radiances, not on retrieved CO concentrations.
The NCAR Command Language (NCL) is a free, interpreted language designed specifically for scientific data processing and visualization. NCL has robust file input and output. It can ingest data in netCDF, HDF4, HDF4-EOS, GRIB1, GRIB2, binary, and ASCII formats. The world-class graphical outputs are highly customizable.
CISL officially released version 4.3.0 of NCL, and it is now available for download from the Earth System Grid (ESG). The GRIB2 reader is the significant new feature in this release. In addition, there are 14 new functions, 12 new color tables, updates to 15 functions, four new resources, and many bug fixes. NCL is a data processing and visualization tool of global importance to our user community. For example, the NCL team recently trained NOAA staff on its use, and just received an all-expenses-paid request to train scientific users in Istanbul, Turkey.
Like CISL, EOL provides the university research community with many tools and services it might not otherwise be able to afford or find. In 2007, development continued on the EOL Metadata Database and Cyber Infrastructure (EMDAC). A next-generation user interface has been created, which enhances links between EMDAC and the UCAR Community Data Portal (CDP). EOL has also developed tools to make it easier for EOL facilities users to manage their datasets via EMDAC. EOL also provides users with information on using these tools. To improve its efficiency, stability, and ease of use, EOL’s Mass Store Retrieval tool will be incorporated into EMDAC.
Among other upgrades, EOL installed a Satcom system onboard the GV that provides a new paradigm for operating and communicating with research aircraft. The installation of the new Satcom system on the GV underscored the importance of such communications capabilities on the C-130, which currently is involved in the majority of field research requiring aircraft. We were recently able to install a state-of-the-art Inmarsat Satcom system on the C-130, which creates a bandwidth comparable to the GV.
Click to enlarge.
Concept drawing of the COSMO 1.5-meter coronagraph and dome. The telescope is a simple tube structure on an equatorial mount. The diameter of the dome is 12.2 meters.
In FY2007, the ESSL/HAO Community Spectro-polarimetric Analysis Center (CSAC) completed development of the GRID Milne-Eddinton inversion code (MERLIN), and the development and implementation of data calibration/conditioning software for the Diffraction Limited Spectro-polarimeter (DLSP), the SST Spectro-polarimeter, and the Hinode Spectro-polarimeter. These efforts will provide data services – including data reduction, inversions, and data visualization – to those involved in the measurement and interpretation of solar vector magnetic fields, a rapidly growing community.
This year also saw improvements to the data services offered through the ESSL/HAO to the Mauna Loa Solar Observatory (MLSO). Observing and processing software was modified to take high time-resolution (10 seconds) of Mauna Loa Halpha images to meet the needs of a community joint-observing campaign with Hinode, STEREO and other NASA missions. All of the Solar Maximum Mission (SMM) coronagraph data were made available online, and Mauna Loa observations were also added to community-based software packages including SolarSoft, Solar Weather Browser and FESTIVAL, and the community-based Virtual Observatory known as AstroGrid.
FY2008 Plans for Strategic Priority 2
CISL views its TeraGrid-related activities as strategic areas of development. Testing and deployment of wide area, global parallel file systems will continue using the storage cluster in NCAR’s TeraGrid node. Funding for the Visualization and Analysis Platform for Ocean, Atmosphere, and Solar Researchers (VAPOR) project runs out at the end of FY2008. Since VAPOR is considered strategic to a petascale supercomputing environment that will produce multi-terabyte history files, CISL is working to broaden the utility and appeal of VAPOR to new communities, and we have included some support for VAPOR in the NSF Track-1 proposal. We will continue to pursue other proposal opportunities with the VAPOR team.
The application of DART to estimating concentrations of atmospheric constituents will be expanded to the radiances measured by the MOPITT instrument. This is the first step in DART becoming the data analysis core for a satellite simulation facility, which will be pursued in collaboration with ESSL’s Atmospheric Chemistry Division.
Plans for NCL and Python software development and support are largely based on continuing dialogs with the scientific community and on results from a survey conducted in FY2006. FY2008 goals are to release NCL, PyNGL, and PyNIO under an open source license, release new features and science enablers, and continue to provide high-quality consulting support and training services for the user community.
In FY2008 and 2009, EOL will solicit feedback from the community to continue evolving EMDAC to meet user requirements, and will enhance current infrastructure to streamline overall internal data management activities. EOL will also continue developing a single, integrated codebase for the Field Catalog, which provides a secure and robust platform from which to plan future projects.
In the coming year the HAO CSAC will:
- Begin the Milne-Eddington inversion of Hinode Data, including implementing the genetic algorithm initialization for MERLIN, and the GUI, as well as integrating the HINODE database
- Implement LILIA (LTE Inversion based on the Lorien Iterative Algorithm) detailed inversion
- Generalize AZAM (azimuth ambiguity) to accept data from any data source
- Implement simulated annealing azimuth ambiguity resolution for automatic processing
- Host a community workshop to address community needs
- Sponsor graduate student visits
Related Lab Annual Report Sections:
Goal 4, Priority 2