IOPS Galore: Upgrading a Supercomputer's Metadata with Solid-State Drives

Overview

The NASA Center for Climate Simulation (NCCS) continually strives to provide faster, more advanced technology for its users. A part of the most recent wave of upgrades is a refresh of the metadata storage, which houses information such as inodes and directory tree structures. This refresh has increased the metadata capacity of the Discover supercomputer's storage system to 16 terabytes, significantly expanded user and administrator capabilities in metadata operations, and introduced new technology into the storage system.

Project Details

Discover currently contains nearly 3,300 compute nodes and over 25 petabytes of usable storage. To support such a thriving operation, the NCCS physically separates data storage from metadata storage. Separating these storage types has many benefits, one of which is using different technologies that may not be feasible for the entire storage cluster.

The NCCS has taken advantage of this method by upgrading the previous spinning disk metadata storage to solid-state drives (SSDs). These drives are technologically superior to spinning disk for this purpose, with benefits including significantly higher capacity for input/output operations per second (IOPS) and avoiding common issues like “head thrashing” (and the slow file access that comes with it).

Results and Impact

System administrators successfully transitioned all of Discover's metadata over to SSDs. They not only see benefits in administrative operations, but have multiple success stories from users as well. Direct consequences include metadata performance improvement of up to 10x and zero noticeable impact during rebuilds after failed disks.

Why HPC Matters

A high-performance computing environment gives scientists and engineers a place to generate and analyze large quantities of data within a reasonable amount of time. By providing newer and more advanced storage technologies, the NCCS enables these researchers to work with their data at a fraction of the time cost and tackle scientific problems of greater size and complexity.