Multi-Protocol High-Performance Centralized Storage

Overview

The NASA Center for Climate Simulation (NCCS) has multiple platforms for creating and analyzing scientific data. These include the Discover supercomputer cluster and the Advanced Data Analytics Platform (ADAPT). Both Discover and ADAPT have large amounts of local storage, but at present there is no single storage repository that can be computed against from the two systems. This situation means that scientists whose workflows leverage both systems need to move data back and forth between them. That process is not only time-consuming but also results in significant duplication of data. As datasets increase in size, storing duplicate copies of petabyte-sized datasets is not economically feasible both in the time it takes to move the data and in the amount of storage capacity required. Accordingly, the NCCS designed and is implementing a Centralized Storage infrastructure to enable computation on datasets from both systems.

Project Details

The initial installation of the NCCS Centralized Storage infrastructure will contain approximately 20 petabytes of raw storage, a high-speed internal InfiniBand network, 40 Gigabit Ethernet connections to ADAPT, and direct connectivity to the Discover’s InfiniBand and Omni-Path high-speed networks. It will use IBM’s Spectrum Scale General Parallel File System (GPFS) filesystem to store the data.

Results and Impact

Research collaborations such as NASA’s Arctic-Boreal Vulnerability Experiment (ABoVE) are using large volumes of Earth observations, including high-resolution visible-wavelength imagery, on Discover to create digital elevation models (DEMs) and store them in the shared Centralized Storage. In turn, this data is mounted on virtual hosts in ADAPT for investigations using customized analytics environments.

The shared Centralized Storage also contains data collections such as 30 years of Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) data, lidar altimetry data from the recently launched ICESat-2 spacecraft, historical data from the first ICESat (2003–2009), and airborne observations from the Operation IceBridge field campaign (2009–2018). ABoVE scientists (and others) are taking advantage of the close proximity of the DEMs, observations, and atmospheric analysis data to closely examine how the Arctic-boreal environment (e.g., permafrost) is changing. High-resolution ABoVE analysis data generated on ADAPT can in turn be used in processing on Discover, for example, in investigations of high-mountain snowmelt and ice sheet-ocean interactions.

Why HPC Matters

Access to tens of petabytes of complementary Earth observations and analysis data—coupled with Discover’s powerful 5-petaflop high-performance computing (HPC) environment and ADAPT’s hundreds of customizable virtualized hosts—is proving valuable to the NASA science community. Having data collections accessible from Centralized Storage by varied computational resources is facilitating investigations that would otherwise require much more disk space, network infrastructure, and time to move data between systems. This proximity of related data, processing, and services is already demonstrating the important roles these NASA resources play for multiple science communities (e.g., Earth scientists studying cryospheric processes, the carbon cycle, and biomass).

More Information

• See the poster for this demonstration (PDF, 1.9MB). As shown in the NASA booth at SC18.