Estimating Trees and Shrubs in the Sub-Sahara Using Cloud Computing

Overview

As remote sensing observations and model outputs have grown, scientists are increasingly burdened with both the necessity and complexity of large-scale data analysis. Scientists are increasingly applying traditional high-performance computing (HPC) solutions to solve their "big data" problems. While this approach has the benefit of limiting data movement, HPC systems are optimized for large-scale simulations and not for data analysis. To address these issues and to alleviate some of the strain on the HPC environment, the NASA Center for Climate Simulation (NCCS) has created the Advanced Data Analytics Platform (ADAPT). ADAPT combines HPC and virtualization technologies to create an on-site private cloud specifically designed for large-scale data analytics. To extend this approach beyond ADAPT, the NCCS has coupled this private environment with a commercial cloud—namely Amazon Web Services (AWS)—by leveraging Cycle Computing's cloud software tools. The result is a highly extensible and elastic hybrid cloud environment where users can start using systems and services at NASA Goddard Space Flight Center and burst into commercial clouds as resource demands dictate.

Project Details

The NCCS partnered with Compton Tucker (NASA's Goddard Space Flight Center) and Paul Morin (University of Minnesota) on a project to estimate the tree and bush biomass over the entire arid and semi-arid zone on the south side of the Sahara. Tucker and Morin submitted a proposal to the Intel "Head in the Clouds Challenge" and were subsequently awarded a research grant for compute and storage resources in AWS. To estimate the amount of carbon stored and create a baseline for future research, their effort will process commercial satellite sub-meter resolution imagery. Given that ADAPT cannot be dedicated to this project, it is estimated that it would take approximately 10 months to analyze all the data using NASA systems exclusively. By using the hybrid cloud approach of combining local resources with AWS, the goal is to finish a single iteration of the analysis in one month.

Results and Impact

Efforts to date have focused on Niger (Universal Transverse Mercator [UTM] zone 32 for the latitude range of 12° N to 24° N) for this proof of concept test, which involved:

• Mosaicking 2,500+ QuickBird satellite images into 100-kilometer (km) x 100-km tiles, which are then broken into 25-km x 25-km sub-tiles for processing.
• Orthorectifying and mosaicking all satellite data in ADAPT, to prevent double or multiple counting, and transferring data to AWS using Cycle Computing's Dataman utility.
• Identifying trees and shrubs using adaptive vegetation classifier algorithms, determining their height and crown areas, and estimating their biomass. Cycle Computing utilities automated the data transfer to and processing in AWS West, a zero carbon footprint facility.

Why HPC Matters

The tile-oriented processing is "embarrassingly" parallel and efficiently leverages concurrent processing on multiple virtual machines (VMs) in ADAPT or instances in AWS. The combined resources of ADAPT and AWS potentially reduce total processing time to less than one month.

What's Next

With UTM Zone 32 completed, the next objective is to incorporate GeoEye and WorldView satellite data, which necessitates inter-calibrating imagery among multiple sensors. Then, the research team will process the entire Sahel region—UTM zones 28 through 38—incorporating algorithms to calculate tree and shrub height for biomass estimates.