Optimizing Code for NASA's High-Performance Computing Systems

Overview

Many factors must fall into place for calculations to run optimally on modern high-performance computing systems. For example, the layout of memory and the ordering of arithmetic operations can greatly affect data movement, which typically has a significant effect on performance. Throwing more compute resources at the problem can often speed things up, but leads to a question: what is meant by "optimization?" Is the goal to minimize the time to calculate the solution, or to reduce the amount of compute resources required? As part of the services provided to users of the high-performance computing systems at the NASA Advanced Supercomputing (NAS) facility, the Application Performance and Productivity (APP) team provides the tools and assistance needed to optimize their codes.

Project Details

Part of NASA's High End Computing Capability (HECC) Project, the APP team provides a range of consulting services for users. In addition to helping them port codes or find causes for job failures on NAS supercomputers, the group also works closely with users to understand and improve code performance. This process starts by acquiring the user's code, a representative dataset, and the expected results. Then, the team begins an iterative process to identify computationally intensive code segments (hotspots) that may be causing performance issues, which they will then attempt to reduce or eliminate.

Performance analysis tools are essential to help in this process. For example, identification of bottlenecks is facilitated when tools report findings such as where the hotspots are, whether a hotspot has been compiled using scalar or vector instructions, and when there is an imbalance between different threads of execution.

Results and Impact

In the past year, the APP team has assisted HECC users from across NASA and the agency's university partners with optimizing the performance of their codes to run on agency systems. Codes optimized include:

• SFILTER – The APP team more than doubled the speed of the job runs and allowed the science team to perform a simulation that previously would not run. The modified code, using a new Intel compiler and job partitioning, resulted in a savings of more than 2.47 million standard billing units (SBUs). One SBU is the equivalent to one hour on a Westmere node on the Pleiades supercomputer.
• VULCAN – The team's optimization work resulted in a code that runs 46% faster than before. Aside from the ability to complete more simulations, this effort was part of an ongoing collaboration between NASA's Ames Research Center and NASA's Langley Research Center to share knowledge regarding different ways to improve application performance.
• MULTIA – Over a span of three weeks, we were able to increase code performance by a factor of four. Work is still continuing to attain the 25x speedup the user needs to achieve his science goals.
• Fortran CUDA wrapper – The APP team developed a wrapper that makes a subset of the existing, highly optimized CUDA libraries accessible to Fortran programmers who are using NASA high-performance computing systems.

What's Next

In addition to investigating codes at users' request, the APP team will continue collaborating with other NASA centers to share knowledge of optimization fixes and best practices, which could be useful at other agency high-performance computing centers. The team will also continue to work closely with the administrators at NAS to identify user codes that trigger instabilities within the systems.