HECC: Evolving to Meet Tomorrow’s Requirements
Overview
NASA’s challenging mission to explore space and understand the universe and the Earth within it requires the agency to innovatively apply and extend humankind's most advanced capabilities, technologies, and knowledge. The High-End Computing Capability (HECC) Portfolio provides a suite of such powerful leading-edge tools for the benefit of all NASA mission directorates.
The HECC Portfolio accelerates scientific and engineering discoveries by providing resources and services that address NASA’s current high-performance computing (HPC) and data analytics needs, and engages in research and development that will create the productive environments of the future.
Project Details
HECC meets NASA’s current HPC requirements through the provision of world-class computing architectures both on the cloud and at the NASA Advanced Supercomputing facility at Ames Research Center, designed to maximize return on investment, as well as services that multiply the impacts of those assets.
The state of the art in physics-based simulations is changing, largely due to the Department of Energy’s Exascale Computing Project (ECP), which will spend $1.7 billion over seven years to “deliver exascale simulation and data science innovations and solutions to national problems.” The path to exascale will employ GPU-accelerated nodes; preparing applications for these new architectures is difficult and time consuming.
To meet these needs, HECC will shape its activities along the three main technical axes of ECP’s organization:
- Application Development: Work with NASA mission directorates to identify the agency’s most important applications for the future, and partner with subject matter experts to upgrade them to ensure they will be ready for exascale.
- Software Technology: Investigate approaches and packages that facilitate porting existing applications or developing new ones; and work with academia and industry to exploit the power of data analytics to provide better insight into complex data sets.
- Hardware and Integration: Evaluate and provide access to exascale-like systems; explore arrangements with other government agencies for access to their systems or cloud-based resources; and develop in-house pathfinding systems.
Results and Impact
HECC’s experiment of housing supercomputers in modular facilities exceeded expectations, providing growth opportunities that would not have been possible in our older facilities. The modular facilities increased power available for computing from 5.8 megawatts (MW) to 10.9 MW, with the potential to grow to 37 MW. For the three existing modules housing Electra and Aitken, the power usage effectiveness (PUE)—the metric used to determine data center energy efficiency—is near the top of the industry at 1.05. Next year, a fourth module will be installed with an additional 3.7 MW of power to expand Aitken—now NASA’s most powerful supercomputer—even further.
HECC’s rapid growth over the past several years—expanding to multiple facilities and systems—resulted in significant improvements to its InfiniBand network, which will evolve from a single-system, focused multi-dimensional hypercube to a data island connected by a fat tree and accessed by all the compute clusters. This approach maintains expandability while taking improving performance.
In the past year, technical staff successfully transitioned a science team running computational fluid dynamics from HECC on-premises resources to our AWS cloud resources. HECC staff also worked with several science teams to burst in from the cloud to on-premises resources for compute intensive components of their workflows.
In 2023 a new HECC cluster, Cabeus, will provide scientists and engineers with three times more GPU capacity than currently available. Installation of a 128-node GPU expansion containing 512 NVIDIA A100 GPUs will begin in December 2022, and will be joined with the existing GPUs to build Cabeus—a new system that will exceed 7.6 petaflops (quadrillion floating-point operations per second).
In addition, staff are in the process of upgrading the 128-screen hyperwall at the NASA Advanced Supercomputing (NAS) facility at NASA’s Ames Research Center, which will grow from 245 million to 1 billion pixels. Its physical size will increase to 32’ x 9’3”, requiring a redesign of the display room. The new facility will be available to HECC users in early 2023.
What's Next
HECC will continue to optimize the environment on-site as well as facilitate access to cloud resources. Additionally, a new data module is being deployed to contain filesystems outside the NAS facility, which will enable users to continue their work during an unplanned outage in the fixed facility. Planned improvements to the NAS infrastructure by replacing all chillers with newer, more energy efficient models will improve the facility’s PUE from 1.34 to 1.28.
More Information
Researchers
- William Thigpen, NASA Ames Research Center Please enable Javascript to see email address.
