Domain-Specific Language (DSL) Adoption into NASA’s Goddard Earth Observing System (GEOS) Model Code

Overview

NASA’s Global Modeling and Assimilation Office (GMAO) is undertaking an effort to develop a next-generation Goddard Earth Observing System (GEOS) model. Recognizing that accelerator-based high-performance computing (HPC) systems provide a promising platform to meet future GEOS requirements, the GMAO is looking to leverage such systems by incorporating a domain-specific language (DSL) into GEOS. DSL adoption brings an opportunity for code portability of GEOS across multiple architectures. It also enables developers to use a higher-level language, which improves development productivity by abstracting the details of the computing architecture.

To jump-start DSL adoption, the GMAO leveraged a GridTools for Python (GT4Py) DSL port of the Finite-Volume Cubed-Sphere Dynamical Core (FV3) from the Allen Institute for AI (AI2). The GMAO is developing an approach to incorporate the GT4Py-port of FV3 into GEOS and determine whether incorporating the DSL-ported FV3 makes it tractable to achieve performance gains on accelerators over CPUs.

Project Details

The GMAO is taking incremental steps to create an accelerator-based GEOS codebase. The first step involved the integration of the GT4Py port of FV3 (gtFV3) into GEOS by creating a code interface layer that connects the Fortran-based GEOS codebase with the Python-based gtFV3. Since Fortran can interoperate with C-based code, the interface was implemented using the C Foreign Function Interface (CFFI) for Python, which allows C code to interface with Python. This interface passed relevant data between gtFV3 and the GEOS framework and also executed gtFV3. Developers successfully integrated the interface into the GEOS framework, demonstrating that gtFV3 can accurately execute an atmospheric model by comparing averaged zonal wind values from a CPU-based GEOS run.

Results and Impact

The GMAO verified that gtFV3 was successfully integrated into the GEOS framework by comparing 30- and 90-day U-zonal wind averages between gtFV3 running on NVIDIA graphical processor units (GPUs) and a CPU-based FV3 GEOS run. Currently, the performance of executing gtFV3 in GEOS is comparable to the current CPU-based FV3 implementation. AI2 has shown that gtFV3 executes about 3.5 times faster on GPUs versus CPUs, so the GMAO will try to achieve the same performance with gtFV3 within GEOS.

The project is forward-looking for the GMAO, which develops the GEOS model used to simulate and predict actions of the Earth's atmosphere and oceans. Since the DSL hides the details of the computing architecture from the developer, it provides GEOS with the capability to utilize current architectures—such as general-purpose GPUs (GPGPUs) and potential future architectures such as vector engines and ARM processors—without having to rewrite the codebase.

Why HPC Matters

Global atmospheric and ocean simulations such as those running with GEOS require signficant HPC resources to accurately resolve global climate patterns. Currently, advancements in HPC compute capabilities are being provided by accelerator-based computing such as GPUs. Enabling GEOS to utilize these architectures will provide the capability to tackle higher-resolution and more complex global simulations accurately.

What's Next

With heterogeneous architectures, properly managing data movement is key to achieving optimal performance, and this project will explore how to expand the GEOS software framework to support accelerator data transfers and refactor GEOS to minimize data movement. Creating a true global climate model will require porting additional physics schemes to GPU architectures. The GMAO will aim to create an aquaplanet simulation as a next verification model.

More Information

• NASA Global Modeling and Assimilation Office
• Public repository containing GEOS framework with gtFV3 and Held-Suarez running via OpenACC
• Public repository for GT4Py

Researchers

• Christopher Kung, NASA Goddard Space Flight Center
  Please enable Javascript to see email address.
• Purnendu Chakraborty, NASA Goddard Space Flight Center
  Please enable Javascript to see email address.