Skip to main content
SpringerLink
Log in
Book cover

Encyclopedia of Big Data Technologies pp 1–5Cite as

Benchmark Harness

  • Living reference work entry
  • First Online:

  • 558 Accesses

Synonyms

Test harness

Definition

A benchmark harness is a software that provides the infrastructure to conduct benchmarks of a software and/or hardware system, typically with the goal to quantitatively assess the system’s characteristics and capabilities or to compare the characteristics and capabilities of multiple systems relative to each other. It facilitates the development and execution of benchmarks and the analysis of benchmark results. Typical components of a big data benchmark harness include a tool to generate data, an execution environment to run benchmarks, a data collection component to monitor the benchmark run, and a reporting component to calculate and summarize benchmark results.

Overview

In computing, benchmarking is the process of assessing a system’s quantitative characteristics and capabilities by running a benchmark workload (or set of workloads) against it. The assessed system, also referred to as the system under test (SUT), may be a software and/or hardware...

This is a preview of subscription content, log in via an institution.

References

  • Baru C, Bhandarkar M, Nambiar R, Poess M, Rabl T (2012) Setting the direction for big data benchmark standards. In: Technology conference on performance evaluation and benchmarking. Springer, Berlin, pp 197–208

    Google Scholar 

  • Capotă M, et al (2015) Graphalytics: a big data benchmark for graph-processing platforms. Proceedings of the GRADES’15. ACM

    Google Scholar 

  • Ghazal A, Rabl T, Hu M, Raab F, Poess M, Crolotte A, Jacobsen HA (2013) BigBench: towards an industry standard benchmark for big data analytics. In: Proceedings of the 2013 ACM SIGMOD international conference on management of data. ACM, pp 1197–1208

    Google Scholar 

  • Han R, Lu X, Jiangtao X (2014) On big data benchmarking, Workshop on big data benchmarks, performance optimization, and emerging hardware. Springer, Cham

    Google Scholar 

  • Michael N, et al (2017) CloudPerf: a performance test framework for distributed and dynamic multi-tenant environments. Proceedings of the 8th ACM/SPEC on international conference on performance engineering. ACM

    Google Scholar 

  • Ming Z et al (2013) BDGS: a scalable big data generator suite in big data benchmarking, Workshop on big data benchmarks. Springer, Cham

    Google Scholar 

  • Rabl T, Poess M (2011) Parallel data generation for performance analysis of large, complex RDBMS. In: Proceedings of the fourth international workshop on testing database systems. ACM, p 5

    Google Scholar 

  • Wang L, et al (2014) Bigdatabench: a big data benchmark suite from internet services. High Performance Computer Architecture (HPCA), 2014 IEEE 20th international symposium on. IEEE

    Google Scholar 

  • Zhan J, et al (2016) BigDataBench tutorial. ASPLOS. http://www.bafst.com/events/asplos16/BigDataBench_tutorial_16/wp-content/uploads/BigDataBench_First_Part-asplos16.pdf

Download references

Author information

Authors and Affiliations

  1. Oracle, San Francisco, CA, USA

    Nicolas Michael

Authors
  1. Nicolas Michael
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicolas Michael .

Editor information

Editors and Affiliations

  1. School of Comp. Sci. and Engineering, University of New South Wales School of Comp. Sci. and Engineering, Eveleigh, New South Wales, Australia

    Sherif Sakr

  2. Sch of Info Techno, Building J12, University of Sydney Sch of Info Techno, Building J12, Sydney, Australia

    Albert Zomaya

Section Editor information

  1. Server Technologies, Oracle, Redwood Shores, California, USA

    Meikel Poess

  2. Database Systems and Information Management Group, Technische Universität Berlin, Berlin, Germany

    Tilmann Rabl

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Michael, N. (2018). Benchmark Harness. In: Sakr, S., Zomaya, A. (eds) Encyclopedia of Big Data Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-63962-8_134-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-63962-8_134-1

  • Received:

  • Accepted:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-63962-8

  • Online ISBN: 978-3-319-63962-8

  • eBook Packages: Springer Reference MathematicsReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Search

Navigation