Abstract
Workflow engines are frequently used in the service-oriented and cloud computing domains. Since engines have significant impact on the quality of service provided by hosted applications, it is desirable to compare and select the most appropriate engine for a given task. To enable such a comparison, approaches for benchmarking workflow engines have emerged. Although these approaches deal with different quality properties, such as performance or standard conformance, they face many reoccurring problems during the design and implementation phase, which they solve in similar ways. In this paper, we describe such common solutions to reoccurring problems in the area of workflow engine benchmarking as patterns. Our aim is to present pattern candidates that help benchmark authors to design and implement proper and valid workflow engine benchmarks and benchmarking tools.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alexander, C.: A Pattern Language, August 1978
Bianculli, D., Binder, W., Drago, M.L.: SOABench: performance evaluation of service-oriented middleware made easy. In: ICSE, pp. 301–302. ACM (2010)
Ferme, V., Ivanchikj, A., Pautasso, C.: A framework for benchmarking BPMN 2.0 workflow management systems. In: Motahari-Nezhad, H.R., Recker, J., Weidlich, M. (eds.) BPM 2015. LNCS, vol. 9253, pp. 251–259. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23063-4_18
Ferme, V., Ivanchikj, A., Pautasso, C., Skouradaki, M., Leymann, F.: A container-centric methodology for benchmarking workflow management systems. In: CLOSER (2016)
Gamma, E., Helm, R., Johnson, R.E., Vlissides, J.: Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, Amsterdam (1995)
Geiger, M., Harrer, S., Lenhard, J.: Process engine benchmarking with betsy - current status and future directions. In: ZEUS, pp. 37–44, January 2016
Geiger, M., Harrer, S., Lenhard, J., Casar, M., Vorndran, A., Wirtz, G.: BPMN conformance in open source engines. In: SOSE, March 2015
Geiger, M., Harrer, S., Lenhard, J., Wirtz, G.: On the evolution of BPMN 2.0 support and implementation. In: SOSE, pp. 120–128, March 2016
Harrer, S.: Process engine selection support. In: Meersman, R., et al. (eds.) OTM 2014. LNCS, vol. 8842, pp. 18–22. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-45550-0_3
Harrer, S., Lenhard, J., Wirtz, G.: BPEL conformance in open source engines. In: SOCA, pp. 237–244. IEEE, December 2012
Harrer, S., Lenhard, J., Wirtz, G., van Lessen, T.: Towards uniform BPEL engine management in the cloud. In: CloudCycle, LNI. GI e.V., September 2014
Harrer, S., Nizamic, F., Wirtz, G., Lazovik, A.: Towards a robustness evaluation framework for BPEL engines. In: SOCA, pp. 199–206. IEEE, November 2014
Harrer, S., Preißinger, C., Wirtz, G.: BPEL conformance in open source engines: the case of static analysis. In: SOCA, pp. 33–40. IEEE, November 2014
Harrer, S., Röck, C., Wirtz, G.: Automated and isolated tests for complex middleware products: the case of BPEL engines. In: ICSTW (2014)
Huppler, K.: The art of building a good benchmark. In: Nambiar, R., Poess, M. (eds.) TPCTC 2009. LNCS, vol. 5895, pp. 18–30. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-10424-4_3
ISO/IEC. ISO/IEC 19510:2013 - Information technology - Object Management Group Business Process Model and Notation, November 2013. v2.0.2
Kohls, C.: The structure of patterns. In: PLOP. ACM (2010)
Kohls, C.: The structure of patterns - part ii - qualities. In: PLOP. ACM (2011)
Meszaros, G., Doble, J.: A pattern language for pattern writing. Pattern Lang. Program Des. 3, 529–574 (1998)
OASIS. Web Services Business Process Execution Language, April 2007. v2.0
Papazoglou, M.P., Traverso, P., Dustdar, S., Leymann, F.: Service-oriented computing: a research roadmap. IJCIS 17(2), 223–255 (2008)
Peltz, C.: Web services orchestration and choreography. IEEE Comput. 36(10), 46–52 (2003)
Skouradaki, M., Ferme, V., Pautasso, C., Leymann, F., van Hoorn, A.: Micro-benchmarking BPMN 2.0 workflow management systems with workflow patterns. In: Nurcan, S., Soffer, P., Bajec, M., Eder, J. (eds.) CAiSE 2016. LNCS, vol. 9694, pp. 67–82. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39696-5_5
Skouradaki, M., Roller, D.H., Leymann, F., Ferme, V., Pautasso, C.: On the road to benchmarking BPMN 2.0 workflow engines. In ICPE. ACM (2015)
van der Aalst, W.M.P.: Business process management: a comprehensive survey. ISRN Softw. Eng. 2013, 1–37 (2013)
van der Aalst, W.M.P., ter Hofstede, A.H.M., Kiepuszewski, B., Barros, A.P.: Workflow patterns. Distrib. Parallel Databases 14(1), 5–51 (2003)
WfMC. The Workflow Reference Model, January 1995. v1.1
Wohed, P., Dumas, M., ter Hofstede, A.H.M., Russell, N.: Pattern-based Analysis of BPMN - An extensive evaluation of the Control-flow, the Data and the Resource Perspectives (revised version). BPM Center Report BPM-06-17 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Harrer, S., Kopp, O., Lenhard, J. (2018). Patterns for Workflow Engine Benchmarking. In: Lazovik, A., Schulte, S. (eds) Advances in Service-Oriented and Cloud Computing. ESOCC 2016. Communications in Computer and Information Science, vol 707. Springer, Cham. https://doi.org/10.1007/978-3-319-72125-5_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-72125-5_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-72124-8
Online ISBN: 978-3-319-72125-5
eBook Packages: Computer ScienceComputer Science (R0)