NASA Ames Kicks off Pathfinding Modular Supercomputing Facility
When NASA’s concept for an innovative modular supercomputing facility (MSF) with the potential to save millions of gallons of water a year was announced in November 2015, Silicon Valley was four years into California’s long-running drought. The drought has been alleviated in the Bay Area thanks to abundant rain in January 2017—just as the first module of the MSF was turned over to scientists and engineers around the country who rely on high-performance computing for their research supporting NASA missions.
Now that the prototype module is operational, the system—named Electra—is projected to save 1,300,000 gallons of water annually, according to facility engineers at NASA’s Ames Research Center. And although water levels have risen in many parts of the state, Bay Area environmental experts caution that we still need to conserve water and be aware of long-range climate trends.
Reducing Environmental Impact
These savings translate to money that we’ll use to buy more supercomputing systems to better serve more than 1,500 users—which is the whole reason behind this experiment.
“One of NASA’s key science goals is to expand our knowledge of Earth systems. So we have a responsibility to do our part to lessen the impact of our technologies on the environment over the long term,” said Bill Thigpen, chief of the Advanced Computing Branch in the NASA Advanced Supercomputing (NAS) Division on the Ames campus, situated about a half‐mile from the south end of San Francisco Bay.
In addition to reducing water usage, the MSF will also save electrical power, Thigpen noted. By installing Electra in the MSF rather than in the main NAS facility, NASA will save a million kilowatt‐hours of energy each year—equivalent to the annual usage of 90 households, conservatively.
“These savings translate to money that we’ll use to buy more supercomputing systems to better serve more than 1,500 users—which is the whole reason behind this experiment,” said Thigpen. “This is a different way for NASA to do supercomputing in a cost-effective manner. It makes it possible for us to be flexible and add computing resources as needed, and we can save about $35 million dollars—about half the cost of building another big facility.”
Powering up for Future Research & Discovery
Electra’s power consumption has been measured at 1.03 PUE, or power usage effectiveness. This measurement reflects the ratio of energy used by the computing equipment to energy used to power the entire data center, including cooling, lighting, and staff workstations. The system falls well below the computing industry standard of 1.7 PUE.
Built and installed by NASA partners SGI/CommScope, the 1,152‐node Electra system will provide NASA users an additional 280 million hours of computing per year. Electra currently ranks as the 39th most powerful system in the U.S. on the TOP500 list, with a sustained performance of 1.09 petaflops—possibly the fastest supercomputer in a modular setup at this time, according to Thigpen. This is good news for users of the NAS facility resources, where computer floor space and power capacity have reached their limits. The agency’s most powerful supercomputer, Pleiades, takes up most of the space on the facility’s 14,000 square‐foot computer floor.
So far, users testing Electra have given it two thumbs up: it’s now faster and easier to run jobs in the heavily utilized NAS computing environment, where time on Pleiades is always in demand. During the test period, a team of researchers at Massachusetts Institute of Technology (MIT) and NASA Jet Propulsion Laboratory ran a numerical model used for studying the state of the Earth’s oceans over time in order to test the performance of very large, complex applications on Electra. In collaboration with software experts in the NAS Division to optimize some of the MIT code’s input/output routines, the researchers achieved “heroic” performance numbers—an ocean simulation with 31 billion grid cells—which is two-and-a-half times larger than the largest previously attempted computation on Pleiades. “Electra proves the concept that we can deploy significant modular systems to give our user community more computing capability,” Thigpen said.
Building in Flexibility Down the Road
Based on what NAS systems engineers have learned from initial testing and user feedback on the prototype module, NASA has approved a plan to further expand the MSF. A second module type is now being evaluated, which, when combined with Electra would exceed Pleiades in peak performance while maintaining highly efficient water and power usage.
Plans are also moving forward to deploy an expanded modular environment at Ames with up to 16 times the capability of the current Electra configuration. As usage on Pleiades has increased more than 20% in the last two years alone, these plans for expansion will help keep up with the demands for supercomputing time to support NASA engineers and researchers as they continue to study our planet, seek to invent the next breakthrough in aerospace technology, discover new planets, and explore the origin of the universe.
—Jill Dunbar, NASA Ames Research Center
