Teaching Your Legacy Code New (GPU) Tricks

Overview

Accelerated computing using graphics processing units (GPUs) has become an industry standard for high-performance, energy-efficient computing. However, adapting the technology is challenging, especially for large, long-developed legacy codes maintained by domain scientists. The OpenACC parallel programming model allows GPU acceleration through the use of special code comments. This greatly eases the porting process and produces a single source code that’s able to run on both CPUs and GPUs.

CORona-HELiosphere (CORHEL), a state-of-the-art modeling suite developed by Predictive Science Inc., is used extensively to generate solar wind and solar eruption simulations for research and space weather applications. It is also used as the backbone of the Solar Particle Event Threat Assessment Tool (STAT). Adding GPU acceleration to CORHEL would greatly benefit these applications, paving the way towards the eventual goal of real-time forecasting.

Project Details

CORHEL is a great example of a long-standing legacy code package. Developed by domain scientists for over 15 years, it contains multiple model applications, totaling over 100,000 lines of code. Rewriting CORHEL with GPU-specific APIs is simply not feasible due to time, money, and developer learning curve considerations. By using OpenACC, we can rapidly add GPU acceleration with minimal impact on the original code base, and maintain full portability—the code still works as before on older CPU systems.

We approach adding OpenACC to CORHEL on a model-by-model incremental basis. Occasionally, it is necessary (or advantageous) to refactor and/or modify the original CPU code to better position it for OpenACC acceleration. Almost always, this refactoring is minimal and improves the CPU code in both performance and clarity. After each implementation, the resulting code is extensively validated on CPUs and GPUs, and then optimizations are explored.

Results and Impact

Using the above approach, we successfully added OpenACC to several models within CORHEL. Timing results on various GPU systems, including the NVIDIA V100 GPU nodes on NASA’s Pleiades supercomputer, show very good performance compared to CPUs. For example, a simplified solar eruption simulation that takes over eight hours on a Sandy Bridge 16-core CPU node, and over three hours on a Skylake 40-core CPU node, takes only about 12 minutes to complete on a single 8xV100 GPU node. These results demonstrate the low-cost, high-performance advantages of using GPUs on legacy applications.

In addition to the value of running CORHEL on GPUs, this project also demonstrates the potential of using OpenACC to port other applications run at NASA—both legacy and new—to GPUs.

Why HPC Matters

In order to develop and test our OpenACC GPU implementation of CORHEL, we utilized many high-performance computing and local resources including the recently deployed NVIDIA 32GB V100 GPU nodes on NASA’s Pleiades system. Running on Pleiades’ four-GPU and eight-GPU nodes resulted in large speedups over using the CPU nodes.

What's Next

We plan to continue implementing OpenACC into the remaining models in CORHEL, most notably its thermodynamic magnetohydrodynamic model used for large realistic coronal background and solar eruption simulations. This will greatly improve the time-to-science in studying these solutions in detail, as well as improve time-to-solution for advanced space weather applications.

More Information

• OpenACC programming model, tutorials, examples, and compilers
• Using OpenACC to Port Solar Storm Modeling Code to GPUs (NVIDIA Developer Blog)
• GPU Acceleration of an Established Solar MHD Code using OpenACC
• See the poster for this demonstration (PDF, 652KB). As shown in the NASA booth at SC19.
• Download the demonstration slide deck (PPSX, 63.5MB) as presented in the NASA booth at SC19. Click "Read Only" after launching presentation.