Advancing Geospatial Data Structures on GPUs: Mapping the Earth at Fine Scale
Overview
The growing number of Earth observation satellite constellations is producing exponential increases in the amount of data produced. Petabytes of Earth imagery are coming down from hundreds of Earth-observing sensors in orbit. Our planet’s satellite constellation alone generates more than 11 terabytes of data per day. The scientific community is dealing with a fast-moving Big Data problem that necessitates multi-disciplinary engineering, high-performance computing (HPC), and software intervention to succeed. We present an end-to-end, GPU-accelerated pipeline for inference of machine learning (ML) and deep learning (DL) models on very high-resolution geospatial data.
Project Details
The Data Science Group (DSG) at NASA’s Computational and Information Sciences and Technology Office (CISTO) is working with scientists and industry to accelerate the development of innovative science powered by artificial intelligence and machine learning (AI/ML). The goal of this project is to accelerate the inference of individual very high-resolution (< 3 meter) scenes by removing the CPU-dependency of traditional software and by efficiently leveraging HPC hardware.
The DSG builds GPU-accelerated Earth science data structures powered by NVIDIA RAPIDS and distributed by Xarray + Dask. With complete support for cuML, PyTorch, and TensorFlow ML and DL frameworks, we perform all calculations and processing directly on the GPUs, avoiding bottlenecks from in-memory offloading. Multi-node and multi-GPU strategies are implemented for greater acceleration and parallelization. Once inference is performed, we can use NVIDIA’s cuCIM library for additional GPU-accelerated postprocessing. Since executing this software uses a large amount of compute resources, we containerize it for greater portability.
Results and Impact
The DSG has processed thousands of visible and near-infrared very high-resolution images for the segmentation of clouds and land cover and individual object detection across the tropics and Arctic regions. Performing Earth observations at more local scales is important to better understand and monitor Earth dynamics. However, previous efforts using traditional computing methods have greatly underperformed in the presence of diverse landscapes and study areas smaller than regional scales. By using ML and DL models combined with HPC resources, we can monitor Earth at a level of detail that will accelerate the scientific study of regional dynamics using NASA’s massive Earth data holdings and overcome existing Big Data challenges.
Why HPC Matters
Development occurs in the Explore/Advanced Data Analytics PlaTForm computing environment at the NASA Center for Climate Simulation (NCCS). All of the GPU compute is done in the Promoting Research in Science using Machine Learning (PRISM) cluster and the new NCCS Discover Scalable Compute Unit 16 expansion. The PRISM cluster has 22 nodes each containing 40 Intel cores, 4 NVIDIA V100s, and 768 gigabytes (GB) of RAM; it includes an additional NVIDIA DGX system with 8 NVIDIA A100s and 1 terabyte of RAM. The Discover SCU expansion includes an additional 12 GPU nodes comprised of 2 x 24 AMD Epyc Rome cores, 4 NVIDIA A100 cores, and 512 GB of RAM.
What's Next
NASA Goddard’s Data Science Group plans to continue scaling up the inference of large extents of Earth at very-high resolution. This includes exploring integration of NVIDIA’s Triton system into the inference framework to better support cloud-native applications, while allowing the team to perform cloud bursting when needed.
More Information
Researchers
- Jordan A. Caraballo-Vega, NASA Goddard Space Flight Center Please enable Javascript to see email address.
- Mark L. Carroll, NASA Goddard Space Flight Center Please enable Javascript to see email address.
- Christopher S.R. Neigh, NASA Goddard Space Flight Center Please enable Javascript to see email address.
- Caleb S. Spradlin, NASA Goddard Space Flight Center Please enable Javascript to see email address.