NASA’s Module-Based Approach to Greener Computing

Overview

For many years, NASA had a straightforward process to manage the replacement of hardware operated by the agency’s High-End Computing Capability (HECC) Project: when the three-year cost of maintaining a group of compute nodes exceeded the cost of replacing them with fewer, new nodes that did the same work, we replaced the old hardware. However, this process had to change once we reached the limits of the power and cooling capacity of the NASA Advanced Supercomputing (NAS) facility. Because of the limitations, when we installed new hardware, we had to simply remove as many of the least productive nodes as needed to free up enough power and cooling capability for the new ones. As a result, we could not provide our users with as much compute capability as we could have without the facility’s limitations.

In addition, with one-quarter of our total power going to cool the systems and 50,000 gallons of water per day being evaporated, we needed a solution that would expand the NAS facility while being sensitive to the impact on the local environment. These issues led to the deployment of our successful prototype Modular Supercomputing Facility (MSF). Subsequently, we began construction of the NAS Facility Expansion (NFE) project—a one-acre site that will house NASA’s future HPC systems.

Project Details

The HECC systems are located at NASA’s Ames Research Center, on the southern portion of the San Francisco Bay. The temperate climate in this area opened up the possibility for solutions that are well suited to this area, and allow for exceptionally energy-efficient solutions. The first MSF module housing the agency’s newest supercomputer, Electra, was installed in 2016, followed by a second module to expand Electra in 2017. In 2018, we installed additional computers to fully populate the second module, and bringing Electra to over 8 petaflops peak performance. The energy efficiency of the prototype modules exceeded expectations and paved the way for the NFE project.

Results and Impact

In Fiscal Year 2018, the module approach saved approximately 2 million kilowatt-hours of power and over 3 million gallons of water, when compared to what would be used if the hardware had been installed in the primary NAS facility. The modules’ power usage effectiveness (PUE; the ratio of energy used to power the entire facility to energy used by the computers), at 1.025 PUE for Module 1 and 1.036 PUE for Module 2, exceeded our goals of 1.06 and 1.08 PUE and far surpassed industry standards. The technology facilitates an operational approach to new system deployments that are not constrained by facility limitations; each expansion is housed in a module that is customized to best support the systems deployed. The NFE will be able to provide up to 30 megawatts of power for future expansion.

Why HPC Matters

Moving to this technology allows NASA to continue providing essential high-end compute resources to accelerate and enhance the agency’s mission of space exploration, scientific discovery, and aeronautics research by continually creating and ensuring optimal use of one of the most productive high-end computing environments in the world.

What’s Next

Next year will mark the start of operations of the first modular compute systems on the NAS Facility Expansion site.

More Information

• HECC Electra Resource Page
• NASA Ames Kicks off Pathfinding Modular Supercomputing Facility
• See the poster for this demonstration (PDF, 1.9MB). As shown in the NASA booth at SC18.