Maia: An Early Evaluation of an Intel Xeon Phi-Based System

Overview

Intel recently introduced its new Xeon Phi coprocessor, which is based on the company's Many Integrated Core (MIC) architecture. The Xeon Phi features 60 cores with a peak performance of 1.0 teraflops. To gain early experience with a Phi-based system, researchers in the NASA Advanced Supercomputing (NAS) Division deployed a 128-node Xeon Phi-based testbed cluster, named Maia, in early 2013. Using micro-benchmarks, we measured memory, network latency, bandwidth, and performance of OpenMP directives and MPI functions. We also used OpenMP and MPI versions of the NAS Parallel Benchmarks, along with several computational fluid dynamics applications, to examine the performance of different programming modes both within a single node and across multiple nodes.

Project Details

Maia is a 128-node SGI Rackable system, with each node comprising two 8-core Intel Xeon E2670 (Sandy Bridge) processors along with two Xeon Phi 5110P coprocessors. Since its installation earlier this year, NAS teams have been evaluating the system in collaboration with SGI and Intel. The goal of this project is to gain early experience with the Phi-based system and to examine its effectiveness for NASA applications. To conduct the evaluation, we selected a set of benchmarks and applications that are representative of NASA workloads and conducted functionality and performance tests at various levels.

Results and Impact

As the industry drives toward the development of exascale computing, examining maturing technology such as next-generation processors and networks is vital to the modeling and simulation activities that support NASA missions. Our early experience with the Phi-based system showed promise but also identified issues. We are now working closely with vendors and other institutions to improve the technology.

Preliminary results include:

• Overhead of system software such as MPI and OpenMP is very high on the Xeon Phi, so the software must be optimized.
• For native applications tested, one Xeon Phi provides about half the performance of two host processors. NASA's OVERFLOW code, in symmetric mode with two Phi coprocessors, achieved a 1.9x boost compared to native host mode.
• The offload mode has significant overhead associated with data transfer. Careful consideration of offload granularity is critical to performance improvement.
• Software on the Xeon Phi is maturing gradually. We observed significantly enhanced performance and stability with system software revisions.

Role of High-End Computing Resources

The project uses the recently acquired high-end computing resource, Maia, a 128-node SGI Pyramid cluster with two Intel Xeon E5-2670 (Sandy Bridge) processors and two Intel Xeon Phi processors. We will continue working with experts from SGI and Intel in preparation for more general user access of the system.