A Distributed Administration System for Ocean Simulation Data

Abstract

The current growth in supercomputing performance is allowing scientists to run larger simulation experiments, generally leading to an increase in the quantity of data to be stored and analysed. Conventional, manual data administration practices are becoming impractical in efficiently dealing with such data sets.

This paper presents a prototype data administration system developed at the Alfred Wegener Institute for managing the output data from ocean simulation model runs. The system is inherently distributed in nature, and leaverages the technology provided by database systems, fileservers and remote procedure calls, bringing them together within a single visualisation environment.

Typical use of the system is as follows: A scientist setting up their model run will also enter additional data that describes the experiment. During the model run each output data set is automatically transferred from the compute server to long term storage on the fileserver. As each dataset is transferred, information is read from its header and combined with the previously input experiment description to provide full meta information about each dataset. This metadata is then placed in a database. When the scientist wants to view or analyse the data, the database can be searched from within the visualisation environment, which then provides pointers to the relevant datasets on the fileserver. This information can be used by the visualisation program to collect data sets and display them.