Goal 4, Priority 4: Creating an Earth System Knowledge Environment
NCAR continues to develop an Earth System Knowledge Environment (ESKE), new cyberinfrastructure that combines models, data, experiments, collaborative tools, and information resources in a way that fosters knowledge sharing and accelerates scientific workflow. NCAR is creating integrated, collaborative problem-solving environments that advance the community's ability to engage in research and scientific discovery. Efforts span modeling frameworks, data and knowledge management and access, collaboration, and analysis and visualization.
FY2007 Accomplishments
Click to enlarge.
This is a snapshot of the new Community Data Portal (CDP) homepage, which now reflects a more thematic interface to our broad digital asset holdings. We are currently developing a next-generation Semantic Web-based system, and this thematic approach is one step along that path. The CDP has now grown into what is commonly referred to as a "science gateway," where data, models, tools, and knowledge are managed and made available for community access. The CDP serves as a primary foundation for our development of an Earth System Knowledge Environment (ESKE).
The Community Data Portal serves as our premier institutional repository for data, models, frameworks, IPCC analyses, and more. This portal is cyberinfrastructure—a core capability on which the whole institution relies. It provides sustainability by allowing individual groups to publish and manage their holdings while reducing redundant data service development efforts across our organization. At the recent CISL Strategic Retreat, we identified the CDP as a core foundation framework for the ESKE.
The CDP currently hosts a number of models, datasets, frameworks, analyses, and other scientific holdings such as Daymet, ESMF, Megan, Mozart, Rose, Socrates, TUV, the Earth System Curator, and WACCM. In FY2007, we expanded CDP capabilities so it now supports remote data upload/management for field projects, and it has been used to good effect in MILAGRO (Megacity Initiative: Local and Global Observations), with ACME coming up soon. The Cyberinfrastructure Strategic Initiative (CSI) funding has served as either a strategic core or a contributing element of CISL’s part in various funded and unfunded CDP projects, including the:
- Earth System Grid (DOE, climate data, IPCC)
- Virtual Solar Terrestrial Observatory (NSF, solar-terrestrial)
- Cooperative Arctic Data and Information Service (NSF, polar data)
- Global WMO Information System (WIS)
- THORPEX Interactive Grand Global Ensemble (TIGGE)
- GIS IPCC Portal
- GoogleEarth/CDP Opportunity Fund project (CISL+ACD)
The CDP is an important part of outstanding proposals as well, such as the Virtual Operations Center (NSF) and Chronopolis (NSF).
Click to enlarge.
Concept drawing of the COSMO 1.5-meter coronagraph and dome. The telescope is a simple tube structure on an equatorial mount. The diameter of the dome is 12.2 meters. The Earth System Modeling Framework collaboration is building high-performance, flexible software infrastructure to increase ease of use, performance portability, interoperability, and reuse in climate, numerical weather prediction, data assimilation, and other Earth science applications. The ESMF defines an architecture for composing complex, coupled modeling systems, and includes data structures and utilities for developing individual models.
Led by ESSL/HAO computer scientist Peter Fox, the Virtual Solar Terrestrial Observatory and Data Systems and Solar-Terrestrial Informatics efforts are an integrative activity that supports solar, solar-terrestrial, and space physics science carried out by HAO and its communities. To date, emphasis has been placed on providing virtual observatory access to HAO and community data holdings including data integration, robust and high-performance web services for data query and access, and re-use of existing underlying data infrastructures.
The Earth System Curator will be a set of tools, based on this extended ESMF schema that allows modelers to archive and query models, experiments, model components, and model output to test the technical compatibility of model components, and to assemble and run multi-component models.
During FY2007, the ESMF and Earth System Curator efforts focused on second-round ESMF goals, on developing a system architecture, and on supporting collaborations for the Earth System Curator. In particular:
- ESMF build rework and performance optimization was completed to satisfy the CCSM Stage 1 evaluation plan: quantitative criteria were met.
- National Unified Operational Prediction Capability (NUOPC) operational partnership identified ESMF as the basis for the next global numerical weather prediction system, target 2015.
- The MAPL software toolkit, a usability layer for ESMF developed at NASA, was used to couple GEOS-5, Modular Ocean Model 4 (MOM4), and the MIT General Circulation Model (MITgcm).
- The Earth System Curator project defines a distributed architecture based on the Geophysical Fluid Dynamics Laboratory (GFDL) Run-time Environment, plus the NCAR CDP. The Curator project and the Numerical Model Metadata (NMM) group at the University of Reading collaborate to extend the NMM schema for the ESMF hierarchical architecture.
Click to enlarge. This map shows the geographic locations of the sites that use the Earth System Grid (ESG) to acquire climate simulation data, IPCC coupled model results, the CCSM climate model, and an array of analysis and visualization tools. ESG provides services to over 6,000 registered users around the world.
The Earth System Grid (ESG) integrates large-scale data and analysis servers located at numerous national labs and research centers to create a powerful environment for next-generation climate research.
The ESG enterprise now:
- Supports more than 6,000 registered users from around the globe
- Manages more than 180 TB of data, models, and tools
- Has delivered more than 300 TB of data
ESG is a source for the CCSM source code distribution and for analysis and visualization tools. Both the ESG and CDP have been refactored to leverage Java frameworks such as Spring and Hibernate, which will provide significant improvements in maintainability and extensibility.
More importantly, ESG represents an opportunity. It is NCAR’s first service to the TeraGrid: a Science Gateway. Leveraging the TeraGrid, we hope to expand the community of users to include the Purdue Climate Change Research Center.
FY2008 Plans for Strategic Priority 2
CISL intends to leverage the CDP institutional repository to continue pursuing and supporting various strategic endeavors such as WMO Information System (WIS), THORPEX Interactive Grand Global Ensemble (TIGGE), and national digital preservation efforts. We will continue to unify the cyberinfrastucture base where we can harmonize technology across many different labs, projects, and agencies to avoid stovepiping and redundant efforts. CDP is a cornerstone of our ESKE effort, and it will be used in the definition and development of that unifying vision.
The collaboration between CDP and the Earth System Curator is mutually beneficial, and is laying the architectural groundwork for a future Earth System Knowledge Environment.
The ESG project will continue, thanks to a recently funded DOE SciDAC-2 proposal. The follow-on project, called ESG Center for Enabling Technologies (ESG-CET), is an aggressive thrust that expands the original ESG to a global scale, encompasses computation, and includes NOAA as a collaborating partner.
Throughout 2008, the ESSL/HAO Virtual VSTO project will continue to develop and implement modern computer science and software engineering approaches. These include, for example, rapid prototyping, agile development and design iteration, extensive partnering with scientists, computer scientists, and community developers, open-source development, publication, and dissemination of current efforts, and best practices in solar-terrestrial research and data sharing.
Related Lab Annual Report Sections:
Goal 4, Priority 4