SatVision: A Large Vision Transformer Model Trained with MODIS Satellite Imagery
Remote sensing images from NASA’s fleet of Earth-observing satellites are pivotal for applications as broad as land cover mapping, disaster monitoring, urban planning, and environmental analysis. The potential of AI-based geospatial foundation models for performing visual analysis tasks on these remote sensing images has garnered significant attention. To realize that potential, the crucial first step is to develop foundation models—computer models that acquire competence in a broad range of tasks, which can then be specialized with further training for specific applications. In this case, the foundation model is based on a large-scale visual transformer model trained with satellite imagery.
Vision transformers employ AI/deep learning techniques to fine-tune the imagery. Through training on extensive remote sensing datasets, vision transformers can learn general visual representations, as well as capture high-level visual patterns, semantics, and spatial relationships that can be leveraged for a wide range of analysis tasks. Trained vision transformers can handle large-scale, high-resolution data; learn global reorientations; extract robust features; and support multi-modal data fusion—all with improved performance.
The Data Science Group at NASA Goddard Space Flight Center’s Computational and Information Sciences and Technology Office (CISTO) has implemented an end-to-end workflow to generate a training dataset from NASA’s Moderate Resolution Imaging Spectroradiometer (MODIS) surface reflectance products, train a SwinV2 vision transformer called SatVision, and conduct a few visual analysis tasks with the trained transformer.
Quick Facts
Like other transformer models, SatVision comes with a high computational cost due to its demands for a substantial number of parameters and an extensive dataset for training and deployment. The NCCS Centralized Storage System provides on-premises access to abundant MODIS surface reflectance products, ensuring comprehensive training samples. Meanwhile, the NCCS Prism GPU cluster, especially its DGX node, has significantly expedited the training process and enabled the model to be trained within a reasonable timeframe.
- Using the NASA Center for Climate Simulation (NCCS) Prism GPU cluster, the Data Science Group accomplished two tasks: 1) implementing an end-to-end workflow generating a 2-million-image training dataset from MODIS surface reflectance products and 2) training an 84-million-parameter transformer, SatVision.
- Results show that training a large-scale visual transformer model reduces the amount of labelled training data needed to train a competitive model in various downstream tasks.
- Large, trained vision transformers can help researchers process vast amounts of remote sensing data with few classification labels, allowing the extraction of valuable information and results with limited observational input data and the gaining of insights into various Earth and space science phenomena.
- Next steps include moving towards generating a 500-million-image training dataset, scaling up to nearly three billion model parameters, and extending SatVision to incorporate additional imagery.
Researchers
- Jian Li, NASA Goddard Space Flight Center Please enable Javascript to see email address.
- Caleb Spradlin, NASA Goddard Space Flight Center Please enable Javascript to see email address.
