Message from CISL Director Al Kellie
Al Kellie by the liquid coolant reservoir for the 650-KW supercomputer bluefire. Welcome to CISL's FY2008 Annual Report. CISL integrates world-class high performance computing with research and development and applied mathematics to extend the reach of the atmospheric and related sciences community. CISL aligns itself with the overarching priorities of the NSF Strategic Plan and Cyberinfrastructure (CI) Vision and with the NCAR Strategic Plan. Each section of this report describes a CISL project, specifies the strategic goals and priorities it supports, then lists all of its funding sources.
FY2008 has been a year of growth and achievement for CISL, but it also marks a turning point in our approach to our upcoming challenges. Scientific needs are demanding petascale computing resources, and the technology to provide them is arriving now. Large additional investments and developments in facilities, tools, problem-solving environments, education, applied math and statistics, numerical methods, and computer science are all required to produce new discoveries and fulfill the promise of science at the petascale. In FY2008, CISL began refocusing its efforts to facilitate and speed this transition.
In alignment with NCAR's strategic plan, CISL is advancing into the petascale era on three fronts: facilities, science, and education.
- Facility designs and infrastructure deployments need to provide the computation, storage, and connectivity necessary for a secure, Grid-enabled center that will grow to meet rapidly increasing demands.
- Research throughout CISL, especially in its math institute and computational science efforts, aims to provide the geosciences and related communities with mathematical and statistical tools and the numerical methods necessary to solve petascale science challenges.
- Education for the next generation of U.S. scientists and engineers who will use and develop petascale systems continues to be a high priority for CISL.
Our work to advance the goals of 21st-century computational geoscience research is supported by our diverse and talented staff, and is continually informed by fresh input from our collaborators, users, and panels of community advisors.
Developments in facilities and cyberinfrastructure
Each node in bluefire contains thirty-two 4.7-GHz processors that require liquid coolant carried by copper tubes to heat sinks mounted over the processors. This annual report describes our top achievements and near-term plans for facilities and cyberinfrastructure in three sections: increasing the capability and capacity of NCAR's supercomputing facilities; procuring, deploying, and operating NCAR's hardware cyberinfrastructure; and developing software cyberinfrastructure to create problem-solving environments. Highlights for FY2008 include developing the NCAR Supercomputing Center with Wyoming partners, deploying the bluefire supercomputer, serving our first full year as a TeraGrid resource provider, and partnering with NCAR's Advanced Study Program for training graduate students, model developers, and scientists to analyze atmospheric dynamical cores being considered for next-generation models.
Our top accomplishments also include procuring a next-generation mass storage archive, strengthening NCAR's data stewardship role, using ESMF for the TeraGrid Build Test Environment, producing a new Science Gateway Framework, deploying a new TIGGE portal for international data archive and access, and continuing to update the VAPOR and NCL visualization tools.
CISL progress in science and research
The CCSM simulation that produced this image proves that current supercomputers can simulate the Earth System at resolutions 100 times as computationally demanding as the production CCSM. This work shows that upcoming petascale systems will be able to resolve important physical processes that have to be parameterized now. Descriptions of our primary research accomplishments and plans appear in the section "Research in computational science and math for geophysics: TDD and IMAGe" and in the CISL research catalog. Directed by CISL's Technology Development Division (TDD) and its Institute for Mathematics Applied to Geosciences (IMAGe), our contributions support scientific computation, numerical methods, geophysical modeling, and the analysis of geophysical data and model experiments. The top highlight for FY2008 was the successful integration of HOMME's spectral element dynamical core with almost every physics algorithm in the Community Atmosphere Model. When it ran on a 57,600-processor computer, this prototype model simulated a little-understood feature of the atmosphere's kinetic energy spectrum with unprecedented fidelity.
Our top science and research advances also include continuous improvements in data assimilation and statistical methods for summarizing and interpreting complex model output. Ongoing work in numerical methods for improving geophysical models is progressing in GASPAR, GHOST, CAM-HOMME, RBFs, and adaptive mesh refinement. The evolution of our ability to simulate multiscale processes continues in the areas of magnetohydrodynamics, helical flows, planetary boundary layer processes, and multiresolution statistical models. And of extreme value to the future of geophysical modeling, we continue advancing the computational science technologies needed for simulation at the petascale.
Some of the participants in NCAR's 1-13 June 2008 ASP Colloquium. Participants performed hundreds of test runs to analyze 13 dynamical cores for atmospheric models, yielding insights into the advantages and disadvantages of each core. The results from the workshop are available to the research community via a new science gateway developed collaboratively by the Earth System Curator and Earth System Grid projects in CISL. This work was supported by NCAR's Cyberinfrastructure Strategic Initiative, which is developing a Science Gateway Framework that can be used to build a variety of custom portals. A high priority for CISL is our work to prepare the next generation of U.S. scientists and engineers for the challenges of scientific computing in the 21st century. We operate several recurring programs, conduct summer schools, support NCAR and UCAR education efforts, and collaborate with other institutions to help satisfy this critical need. CISL's FY2008 education highlight was co-hosting the NCAR Advanced Study Program (ASP) colloquium entitled Numerical Techniques for Global Atmospheric Models. Jointly supported by NSF, NASA, and DOE, this event immersed 38 students, 13 modeling mentors, 18 lecturers, and 4 organizers in weeks of intensive work that produced significant benefits for both the attendees and the atmospheric research community.
Renderings of a 3D Kelvin-Helmholtz instability using the VAPOR visualization and analysis platform. This image was created by students participating in the IMAGe Theme-of-the-Year 2008 Summer School on Geophysical Turbulence. Students of the summer school received hands-on experience performing the end-to-end processes of numerical simulation, visualization, and analysis using state-of-the-art resources. CISL initiated and operates several education and outreach programs. SIParCS offers 10-12-week internships for exceptional students to gain practical experience with a wide variety of parallel computational science problems by working with the HPC systems and applications related to NCAR's Earth System science mission. The IMAGe Theme of the Year (TOY) assembles multidisciplinary teams of young geoscientists and applied mathematicians to immerse them in the challenges of the geosciences' large computational problems. Each year, CISL provides multiple trainings in NCL data analysis and visualization both here in Boulder and internationally. And weekly, CISL's Visualization Laboratory uses collaborative technologies and stereo 3D scientific visualizations to demonstrate NCAR science to visitors and colleagues.
I invite you to read more about CISL's educational programs in the education and outreach section of this report.
CISL's priorities for FY2009
Our plans for the future are organized by our three paths into the era of petascale science: facilities and infrastructure, science and research, and education of the future workforce. Specifically, we will:
- Establish new ways to exploit massively parallel computing systems
- Provide robust, reliable, secure supercomputers for the geosciences
- Develop NCAR's Grid cyberinfrastructure capabilities
- Develop NCAR's software cyberinfrastructure
- Continue developing the NCAR Supercomputing Center in Wyoming
- Strengthen NCAR's research data stewardship role
- Continue to expand the TIGGE international data archive and access system
- Expand numerical methods for improving geophysical models
- Enhance data assimilation methods
- Improve techniques for simulating multiscale processes
- Continue upgrading the VAPOR and NCL visualization tools
- Enhance the SIParCS and TOY internship programs
- Modernize the Visualization Lab to help NCAR tell its story
- Continue training users in scientific data analysis and visualization
CISL provides levels of support for science that are without peer anywhere in the country. Our ongoing goal is to provide a balanced computing environment that supports our traditional base of atmospheric sciences researchers and integrates with the larger geosciences community via distributed facilities such as the TeraGrid.
As we look to the future, we continually adapt ourselves and our organization to maximize our contribution to understanding the complexities of the Earth System. As you read this report, I hope you share our sense of expectation for significant progress in the future.
I invite you to review our accomplishments and plans in the FY2008 CISL Annual Report.