Algorithm Development for a Many-Core HPC Landscape

Overview

Large-scale fluids simulations are routinely used to determine the viability and performance of new vehicle concepts for numerous space, aeronautics, and science missions at NASA. In the field of computational fluid dynamics (CFD), the Navier-Stokes equations are often solved using an unstructured-grid approach to accommodate geometric complexity. Furthermore, turbulent flows encountered in aerospace applications generally require highly anisotropic meshes, driving the need for implicit solution methodologies to efficiently solve the discrete equations. These approaches require frequent construction and solution of large, tightly-coupled systems of block-sparse linear equations.

Practical computations of complex flow phenomena using tools such as NASA Langley Research Center’s FUN3D solver require efficient use of leadership-class high performance computing systems. The new generation of many-core HPC architectures presents tremendous challenges for software developers—applications are likely to perform worse if not refactored for new hardware paradigms. The goal of our effort is to identify key programming strategies to achieve optimal performance on Intel Xeon Phi-Knights Landing (KNL) and NVIDIA Pascal and Volta GPUs.

Project Details

Exascale simulations impose dramatically lower power and memory-per-core constraints. Legacy multi-core MPI codes may be rendered woefully inefficient by the coming paradigm shift to exascale. To prepare NASA Langley’s FUN3D CFD solver for the future HPC landscape, we ported two representative kernels to the Intel Xeon Phi-KNL and the NVIDIA Pascal and Volta GPU many-core architectures. Data layout strategies and a broad range of programming models are explored, including explicit domain decomposition with message passing, shared-memory approaches based on compiler directives, and assembly-level techniques. Their impact on vectorization and memory performance are studied, and results are compared with those of a multi-core Intel Xeon Broadwell node, along with the performance comparisons.

Results and Impact

We observed considerably higher performance for linear algebra kernels on the many-core architectures. The NVIDIA GPU hardware also accelerates matrix construction substantially. The KNL architecture benefits greatly from explicit prefetching and vectorization. Where thread safety is required, a coloring approach outperforms OpenMP atomics on the KNL processors, while atomic operations excel on the GPUs.

This work will ultimately enable more timely and cost-effective solutions for NASA missions across the speed range, including development of the Space Launch System, unmanned science missions, and numerous aeronautics applications spanning fixed-wing vehicles, rotorcraft, and supersonic configurations. The benefits will also be realized by hundreds of groups outside NASA, including all branches of the Department of Defense, commercial entities such as SpaceX, and academic institutions around the country.

Why HPC Matters

Accurate and efficient simulations of complex aerodynamic flows require frequent solutions of billions of simultaneous equations. These intensive calculations can be efficiently performed using emerging high-performance computing hardware.

More Information

• NASA’s FUN3D website
• See the poster for this demonstration (PDF, 4.2MB), as shown in the NASA booth at SC17.