Supercomputing

Artificial Intelligence/Machine Learning in HPC Environments

Image composite from Landsat 8 data
Image composite from Landsat 8 data captured September 22, 2014 shows a phytoplankton bloom (green and blue swirls) near the Pribilof Islands off the coast of Alaska in the Bering Sea. The turquoise waters are likely colored by a type of phytoplankton called coccolithophores. Norman Kuring, NASA/Goddard

The NASA Advanced Supercomputing (NAS) Division’s Data Science team at Ames Research Center supports artificial intelligence (AI) and machine learning (ML) efforts on NAS high-performance computing systems. The team maintains the PyData software stack, which includes Python packages with JupyterLab, Pandas, Dask, Scikit-Learn, TensorFlow, PyTorch, and many others used to conduct data science and machine learning work, and makes them available to NAS users via Anaconda environments. Support for Horovod using TensorFlow or PyTorch for multi-node GPU ML model training on the NAS facilty's GPU nodes provides advantages over TensorFlow or PyTorch’s native multi-GPU-node implementations, such as easy distribution of training across multiple nodes at scale; use of MPI to communicate between nodes; and up to 30% faster performance. In addition to software maintenance and support, the team’s expertise in using ML and AI methods for scientific purposes helps NAS users to advance their research, including the development of the Spectral Water Inversion Processor and Emulator (SWIPE) and photovoltaic (PV) cell materials.

Quick Facts

The high-performance computing systems provided by the NAS facility enable data science research in subjects ranging from Earth science to material sciences.

Thaddeus Norman,
NASA Ames Research Center
  • Since 2020, more than 250 NAS users have used the AI/ML software stack provided by the Data Science team to support their scientific research, and have created over 260 unique Conda environments.
  • The team helped researchers reduce, explore, and organize more than 31,000 current-voltage (IV) curve data files from different PV cell materials, reducing the dataset to approximately 6,500 IV curves and metadata. Subsequent analysis suggested that PV cell materials can grouped into four types, and neural networks were developed to predict electrical properties and material type.
  • The NASA Ames Biospheric Science team’s SWIPE tool helps researchers understand water quality in Earth’s inland waterways. One of the tool’s components uses a two-layer Mie scattering code to calculate the optical properties of various plankton and phytoplankton species. The NAS Data Science team modified the code using Dask to reduce the calculation time from over two hours to about 18 minutes.
  • The Data Science team has directly supported five research projects that have resulted in a peer reviewed publication and several conference presentations.

Researchers

  • Shubha Ranjan, NASA Ames Research Center
  • Thaddeus Norman, NASA Ames Research Center

More Information