Foundation Models for Generalist Geospatial Artificial Intelligence
Significant advances in developing adaptable and reusable artificial intelligence (AI) models will greatly influence Earth science and remote sensing. Foundation models — which are pre-trained on extensive, unlabeled datasets through self-supervision — can be fine-tuned for downstream tasks using small, labeled datasets. In addition, the scientific community is increasingly interested in constructing generalist AI models that leverage multi-sensor data for Earth observation applications. Prithvi is a transformer-based geospatial foundational model pre-trained on over seven years of multispectral satellite imagery from NASA’s Harmonized Landsat Sentinel-2 (HLS) global dataset. The Prithvi team used approximately 4.2 million samples for training and 45,568 samples for validation. Utilizing Harmonized Landsat Sentinel (HLS) Land Use and Land Cover (LULC) classes ensures representation of all land use categories, with more than 3,000 tiles covering approximately 60% of all urban regions. The team also samples from each LULC class separately and makes sure that the roughly 800 incorporated ecoregions are each represented by at least three tiles. This approach results in a dataset that is more than three times larger than the previous model’s, significantly increasing global coverage and the diversity of landscapes and ecoregions.
Jülich Supercomputing Centre systems are quite good with hosting advanced package stacks and providing access to 4 A100 GPUs with 40 gigabytes of memory on each node. The support team helps you to check on system performance and remove any access bottlenecks like bad nodes.
Sujit Roy,
NASA Marshall Space Flight Center
Quick Facts
- The team used Germany’s Jülich Supercomputing Centre systems, which offer four A100 GPUs with 40 gigabytes (GB) of memory on each node. The initial project phase used 16 GPUs for 24 hours to execute each experiment. For pretraining the model, the team scaled to 80 GPUs and ran for about 15 days. Subsequently, they used one to four GPU nodes (each node containing four A100s and 80 GB of memory) at the NASA Advanced Supercomputing (NAS) facility to finetune the model for five downstream use cases.
- Overall, the model performed better than its previous version, which was trained over the continental U.S. region for the year 2017 only.
- The pretrained model with 600 million parameters and associated fine-tuning workflows is publicly available as an open-source contribution via Hugging Face.
- The Prithvi team is going to further advance the models in terms of multi-modality and sensor fusion.
Researchers
- Amy Lin, NASA Marshall Space Flight Center Please enable Javascript to see email address.
- Sujit Roy, NASA Marshall Space Flight Center Please enable Javascript to see email address.
