Benchmark; Performance benchmark
An application benchmark is a suite of tasks that are representative of typical workloads in an application domain.
Unlike a MICROBENCHMARK, an application benchmark specifies broader tasks that are aimed at exercising most components of a system or tool. Each individual task in the benchmark is assigned a relative weight, usually reflecting its frequency or importance in the application being modeled. A meaningful interpretation of the benchmark results has to take these weights into account.
The Transaction Processing Performance Council (TPC) is a body with a long history of defining and published benchmarks for database systems. For instance, it has defined benchmarks for Online Transaction Processing applications (TPC-C and TPC-E), Decision Support applications (TPC-H), and for an Application Server setting (TPC-App).
Other examples of application benchmarks are: the OO1 and OO7 benchmarks, developed for...
- 1.Carey MJ, DeWitt DJ, Naughton JF. The OO7 benchmark. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 1993. p. 12–21.Google Scholar
- 3.Nicola M, Kogan I, Schiefer B. An XML transaction processing benchmark. In: Proceedings of the ACM SIGMOD International Conference on Management of Data; 2007. p. 937–48.Google Scholar
- 5.Transaction Processing Performance Council. Available at: http://www.tpc.org/default.asp
- 6.Yao BB, Özsu MT, Khandelwal N. XBench benchmark and performance testing of XML DBMSs. In: Proceedings of the 20th International Conference on Data Engineering; 2004. p. 621–33.Google Scholar