Skip to main content
SpringerLink
Log in

Interactive Dashboard for Workflow Engine Benchmarks

  • Conference paper
  • First Online:
Advances in Service-Oriented and Cloud Computing (ESOCC 2016)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 707))

Included in the following conference series:

  • 834 Accesses

Abstract

Today, more and more companies model their business processes using languages such as Business Process Model and Notation (BPMN) or Business Process Execution Language (BPEL), and automate their processes by executing these models on appropriate workflow engines. To help choose the best fitting engine among different alternatives, several benchmarking initiatives have emerged, e.g., the BPEL/BPMN Engine Test System (betsy) that benchmarks conformance and expressiveness, and BenchFlow that benchmarks performance. However, their results are hard to analyze and compare for the typical end-user, developer or researcher. This paper tries to solve this issue by introducing (a) a common data model which can hold data of both betsy and BenchFlow tests and their results, (b) a transformer that can automatically transform the results of betsy and BenchFlow runs into the common data model, and (c) an interactive dashboard that visualizes the results according to the most important use cases. Hence, it enables the end-users, developers and researchers to analyze and compare the engines in a straight-forward manner.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.techempower.com/benchmarks.

  2. 2.

    http://kangax.github.io/compat-table.

  3. 3.

    http://node.green.

  4. 4.

    http://jmeter.apache.org/usermanual/generating-dashboard.html.

  5. 5.

    https://www.mirantis.com/blog/rally-as-an-openstack-performance-dashboard/.

  6. 6.

    https://www.openstack.org/.

  7. 7.

    http://www.workflowpatterns.com/.

  8. 8.

    https://github.com/uniba-dsg/betsy/tree/master/src/main/groovy/betsy/common/model.

  9. 9.

    https://github.com/benchflow/docker-images/blob/dev/cassandra/data/benchflow.cql.

  10. 10.

    The license is given according to the enumeration at http://spdx.org/spdx-license-list/license-list-overview.

  11. 11.

    See http://www.json.org/.

  12. 12.

    See https://github.com/peace-project/transformer.

  13. 13.

    See https://github.com/peace-project/dashboard.

  14. 14.

    See peace-project.github.io.

References

  1. Delgado, A., Calegari, D., Milanese, P., Falcon, R., García, E.: A systematic approach for evaluating BPM systems: case studies on open source and proprietary tools. In: Damiani, E., Frati, F., Riehle, D., Wasserman, A.I. (eds.) OSS 2015. IFIP AICT, vol. 451, pp. 81–90. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17837-0_8

    Google Scholar 

  2. Elias, M., Bezerianos, A.: Exploration views: understanding dashboard creation and customization for visualization novices. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) INTERACT 2011. LNCS, vol. 6949, pp. 274–291. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23768-3_23

    Chapter  Google Scholar 

  3. Ferme, V., Ivanchikj, A., Pautasso, C.: A framework for benchmarking BPMN 2.0 workflow management systems. In: Motahari-Nezhad, H.R., Recker, J., Weidlich, M. (eds.) BPM 2015. LNCS, vol. 9253, pp. 251–259. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23063-4_18

    Chapter  Google Scholar 

  4. Few, S.: Information Dashboard Design: The Effective Visual Communication of Data. O’Reilly, Massachusetts (2006)

    Google Scholar 

  5. Geiger, M., Harrer, S., Lenhard, J.: Process engine benchmarking with betsy – current status and future directions. In: ZEUS, pp. 37–44, January 2016

    Google Scholar 

  6. Geiger, M., Harrer, S., Lenhard, J., Casar, M., Vorndran, A., Wirtz, G.: BPMN conformance in open source engines. In: SOSE, March 2015

    Google Scholar 

  7. Geiger, M., Harrer, S., Lenhard, J., Wirtz, G.: On the evolution of BPMN 2.0 support and implementation. In: SOSE, pp. 120–128, March 2016

    Google Scholar 

  8. Harrer, S., Lenhard, J., Wirtz, G.: BPEL conformance in open source engines. In: SOCA, pp. 237–244, December 2012

    Google Scholar 

  9. Harrer, S., Lenhard, J., Wirtz, G.: Open source versus proprietary software in service-orientation: the case of BPEL engines. In: Basu, S., Pautasso, C., Zhang, L., Fu, X. (eds.) ICSOC 2013. LNCS, vol. 8274, pp. 99–113. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-45005-1_8

    Chapter  Google Scholar 

  10. Harrer, S., Nizamic, F., Wirtz, G., Lazovik, A.: Towards a robustness evaluation framework for BPEL engines. In: SOCA, pp. 199–206, November 2014

    Google Scholar 

  11. Harrer, S., Preißinger, C., Wirtz, G.: BPEL conformance in open source engines: the case of static analysis. In: SOCA, pp. 33–40, November 2014

    Google Scholar 

  12. ISO/IEC. ISO/IEC 25010:2011; Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models (2011)

    Google Scholar 

  13. ISO/IEC. ISO/IEC 19510:2013 – Information technology - Object Management Group Business Process Model and Notation, November 2013. v2.0.2

    Google Scholar 

  14. OASIS. Web Services Business Process Execution Language, April 2007. v2.0

    Google Scholar 

  15. Oppenheimer, D., Brown, A.B., Traupman, J., Broadwell, P., Patterson, D.A.: Practical issues in dependability benchmarking. In: EASY, p. 7 (2002)

    Google Scholar 

  16. Pauwels, K., Ambler, T., Clark, B.H., LaPointe, P., Reibstein, D., Skiera, B., Wierenga, B., Wiesel, T.: Dashboards as a service: why, what, how, and what research is needed? J. Serv. Res. 12, 175–189 (2009)

    Article  Google Scholar 

  17. Röck, C., Harrer, S., Wirtz, G.: Performance benchmarking of BPEL engines: a comparison framework, status quo evaluation and challenges. In: SEKE, pp. 31–34, July 2014

    Google Scholar 

  18. Skouradaki, M., Ferme, V., Pautasso, C., Leymann, F., van Hoorn, A.: Micro-benchmarking BPMN 2.0 workflow management systems with workflow patterns. In: Nurcan, S., Soffer, P., Bajec, M., Eder, J. (eds.) CAiSE 2016. LNCS, vol. 9694, pp. 67–82. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39696-5_5

    Google Scholar 

  19. van der Aalst, W.M.P., ter Hofstede, A.H.M., Kiepuszewski, B., Barros, A.P.: Workflow patterns. Distribu. Parallel Databases 14(1), 5–51 (2003)

    Article  Google Scholar 

  20. WfMC. Terminology & Glossary. WfMC, February 1999. v3.0

    Google Scholar 

  21. Yigitbasioglu, O.M., Velcu, O.: A review of dashboards in performance management: implications for design and research. IJAIS 13(1), 41–59 (2012)

    Google Scholar 

Download references

Acknowledgment

We would like to express our gratitude to Jörg Lenhard and Matthias Geiger for fruitful discussions and feedback regarding the dashboard, and both, Marigianna Skouradaki and Vincenco Ferme, for helping in bringing the BenchFlow data into this dashboard.

Author information

Authors and Affiliations

  1. Distributed Systems Group, University of Bamberg, Bamberg, Germany

    David Bimamisa, Mathias Müller, Simon Harrer & Guido Wirtz

Authors
  1. David Bimamisa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mathias Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Simon Harrer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guido Wirtz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simon Harrer .

Editor information

Editors and Affiliations

  1. University of Groningen, Groningen, The Netherlands

    Alexander Lazovik

  2. Vienna University of Technology, Vienna, Austria

    Stefan Schulte

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bimamisa, D., Müller, M., Harrer, S., Wirtz, G. (2018). Interactive Dashboard for Workflow Engine Benchmarks. In: Lazovik, A., Schulte, S. (eds) Advances in Service-Oriented and Cloud Computing. ESOCC 2016. Communications in Computer and Information Science, vol 707. Springer, Cham. https://doi.org/10.1007/978-3-319-72125-5_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-72125-5_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-72124-8

  • Online ISBN: 978-3-319-72125-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation