High-Performance Data Analysis for the ABoVE Field Campaign

Overview

The effects of climate change are being revealed at alarming rates in the Arctic and Boreal regions of the North America. NASA’s Terrestrial Ecology Program has launched a major field campaign to study these effects over the next 5 to 8 years. The Arctic Boreal Vulnerability Experiment (ABoVE) will challenge scientists to take measurements in the field, study remote observations, and run models to better understand the impacts of a rapidly changing climate on Alaska and western Canada.

The NASA Center for Climate Simulation (NCCS) at Goddard Space Flight Center (GSFC) has partnered with the Terrestrial Ecology Program to construct a data-centric High-Performance Data Analysis Platform (HPDAP) to address the demands of the ABoVE field campaign. The platform combines traditional high-performance computing (HPC) with emerging network and data management technologies to create an infrastructure specifically designed for large-scale climate analytics.

Project Details

The platform is currently in the operational prototype stage undergoing final development prior to going fully operational in early 2015. The ABoVE HPDAP features:

• A mix of physical and virtual compute, storage, and network assets.
• Virtual machines (VM) customized for data-intensive science applications and VM management using a combination of the applications oVirt and ManageIQ.
• Quad Data Rate/Fourteen Data Rate InfiniBand networks to hypervisors and VMs.
• High-performance file systems and data storage—a combination of Gluster and Lawrence Livermore National Laboratory’s native port of ZFS for Linux.
• The open-source Integrated Rule-Oriented Data System (iRODS), Ramadda, and ArcGIS for distributed data management.

Results and Impact

Our data-centric, virtual system approach significantly lowers the barriers and risks to organizations that require access to HPC solutions on demand. Such a service enables the NCCS to be extremely agile and responsive in meeting its customers’ rapidly increasing and evolving requirements.

Why HPC Matters

At the center of the HPDAP architecture is a multi-petabyte open source Gluster storage environment. Surrounding the storage are multi-core, large-memory, HPC nodes interconnected via InfiniBand. VMs can be persistent or transient. Persistent VMs provide ongoing data delivery services, whereas scientists interact with HPDAP through the tailored ManageIQ interface to provision resources and launch transient VMs as the demand arises.