iRODS-Based Climate Data Services and Virtualization-as-a-Service at NCCS

Overview

Scientific data services are becoming an important part of the NASA Center for Climate Simulation's (NCCS's) mission. Our technological response to this expanding role is built around the concept of a Virtual Climate Data Server (vCDS), repetitive provisioning, image-based deployment and distribution, and virtualization-as-a-service (VaaS).

Project Details

The vCDS is an Integrated Rule-Oriented Data System (iRODS) server, specialized for a particular data-centric application. iRODS is data grid middleware that provides policy-based control over collection-building, managing, querying, accessing, and preserving large scientific datasets.

In vCDS, we have developed a version of iRODS that manages Network Common Data Form (NetCDF) products from NASA's Intergovernmental Panel on Climate Change (IPCC) 5^th Assessment Report (AR5) simulations. We use Resource Package Manager scripts to build vCDS images in the following virtualized resource classes:

• NCCS local computing environment
• NCCS local Virtual Machine Environment
• NASA's Nebula Cloud Services
• Amazon's Elastic Compute Cloud

Once provisioned into one or more of these virtualized resource classes, the vCDSs can use the iRODS federation and realized object capabilities to create an integrated data ecosystem that is scalable and adaptable to changing resource requirements.

Results and Impact

This approach enables platform- or software-as-a-service deployment of the vCDS and allows NCCS to offer VaaS. VaaS provides a capacity to respond in an agile way to new customer requests for data services, and also provides a path to migrate existing services into the cloud.

Role of High-End Computing Resources

We have registered Moderate Resolution Imaging Spectroradiometer (MODIS) atmospheric data products in a vCDS that contains 54 million registered files, 630 terabytes of data, and over 300 million metadata values. We have registered the entire catalog (360 files, 47 gigabytes) of monthly Modern Era Retrospective-analysis for Research and Applications (MERRA) products into a vCDS. We are now assembling IPCC AR5 data into a vCDS that will provide the platform upon which NCCS's Earth System Grid Data Node publishes data to the extended science community.