A database benchmark is a reproducible experimental framework for characterizing and comparing the performance (time, memory, or quality) of database systems or algorithms on those systems. Such an experimental framework defines the system under test, the workload, metrics, and experiments.
The system under test comprises a database system and its execution environment, composed of operating system services on top of hardware components. The benchmark may define constraints for the execution environment to ensure that the performance of different database systems is comparable.
The workload typically consists of a synthetic data set, which is either given explicitly as a file or defined by intension with a data generation algorithm. For instance, a relational database benchmarks may rely on a well-defined schema with data distribution rules for generating attribute values as well as a scale factor to create instances of various sizes. An XML database benchmarks may define a...
- 2.Anon et al, Bitton D, Brown M, Catell R, Ceri S, Chou T, DeWitt D, Gawlick D, Garcia-Molina H, Good B, Gray J, Homan P, Jolls B, Lukes T, Lazowska E, Nauman J, Pong M, Spector A, Trieber K, Sammer H, Serlin O, Stonebraker M, Reuter A, Weinberger P. A measure of transaction processing power. Datamation. 1985;31(7):112–8.Google Scholar
- 4.Gray J. Benchmark handbook: for database and transaction processing systems. San Francisco: Morgan Kaufmann; 1992. http://research.microsoft.com/en-us/um/people/gray/BenchmarkHandbook/TOC.htmzbMATHGoogle Scholar