Experiment Setup and Runtime

Abstract

The previous chapter described the objectives and challenges for implementing a cloud service benchmark that has already been designed. Now having an implementation at hand, it can be used to run actual experiments. In this chapter, we discuss how to deploy, set up, and run such experiments. For this purpose, we start by outlining the typical process underlying experiment setup and execution. Afterwards, we discuss an important precondition for running experiments, namely, ensuring that the required resources are readily available when needed. We then dive into addressing challenges that occur directly before, during, and after running an experiment, including challenges associated with collecting benchmarking data, data provenance, and storing data.