Evaluation of a Long Term Viable Dynamic Data Archive Format

Abstract

Although, the Universal File Format (UFF) has functioned effectively as a de facto standard for cross platform data interchange and archiving for nearly 40 years, with the progression of technology, the aging nature of its 80 character ASCII FORTRAN card image based format has become problematic. This paper reviews the various (and sometimes conflicting) issues involved in the development of a new flexible, open definition file format. To help facilitate early evaluation of format function and performance, an intermediate pseudo-prototypical hybrid UFF-DSA format has been implemented. This evaluation format effectively encodes the UFF dataset record information in a XML format (similar to the draft DSA definition) and then stores the individual XML records as discrete entries in the DSA archive container (ZIP format). In addition, the resulting principal features necessary for viable long term archiving of data and results, together with an evaluation of an initial implementation of the design, are presented.