Supercomputing: A Ground-Based Instrument for Exploration

Overview

NASA's supercomputing requirements are projected to grow exponentially over time, as the agency continues to leverage high-end computing (HEC) technology to pursue its diverse scientific and engineering activities. At the NASA Advanced Supercomputing (NAS) facility, HEC systems experts must ensure that the supercomputing environment continually evolves to meet the present and future computational requirements of the agency's challenging missions. Currently, the heterogeneous supercomputing environment at NAS includes a variety of compute, storage, and filesystem resources interconnected via a large InfiniBand network:

• Pleiades—a distributed memory system comprising 162,496 cores with four generations of Intel Xeon processors.
• Endeavour—a cache-coherent, global shared-memory system comprising 1,536 cores with 6 terabytes total shared memory.
• Merope—5,120 cores with Intel Xeon 5400 (Harpertown) processors, repurposed from Pleiades's original configuration.
• Two testbed systems—a 128-node Intel Xeon Phi cluster based on Intel's Many Integrated Core (MIC) architecture and a 128-node NVIDIA graphics processing unit (GPU) cluster.
• Globally accessible filesystems totaling over 10 petabytes.
• Multiple front-end systems, storage systems, a 60-petabyte tape archive, and ten dedicated data analysis nodes.
• The hyperwall-2 visualization system.

Project Details

As the systems evolve, so do their usage models and characteristics. NAS experts have developed an operational model that successfully maintains a productive supercomputer with maximum availability. This is accomplished through several methods that allow most system updates to occur transparently:

• Rolling updates in which new default system images are pushed as user jobs finish.
• Filesystem maintenance that is scheduled one filesystem at a time so only those user jobs requiring the resource are held while others proceed, resulting in full system utilization.
• Systemwide suspend/resume capability for maintenance activities that might otherwise cause disruption, such as firmware updates, server software updates, network topology changes, and equipment addition or removal.

Results and Impact

Since 1983, NAS systems experts have developed increasingly effective tools and processes that have become industry standards. As the industry moves toward exascale computing, our operational model will help us meet new challenges while continuing to support NASA missions in aeronautics, climate science, space exploration, and the study of our Earth and the universe.