Skip to main content
SpringerLink
Log in

A Calibration Framework for Capturing and Calibrating Software Performance Models

  • Conference paper
Computer Performance Engineering (EPEW 2008)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 5261))

Included in the following conference series:

Abstract

Software performance engineering could benefit from combining modeling and testing techniques, if performance models could be derived more cheaply and more easily. This work investigates how known testing and estimation methodologies can be combined in a calibration framework, to provide and maintain performance models in sync with a developing product or component library. There are two main aspects. The first addresses a major barrier in practice, the calibration of model parameters that represent quantities that cannot easily be measured directly. This work calibrates these “hidden parameters” efficiently using a Kalman Filter. The second is the exploitation of the filter estimator to control the calibration framework, for example to terminate a test when accuracy is sufficient, and to design tests for parameter coverage. The technique is demonstrated on simulated data and on an implemented Voice-over-IP (VoIP) system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Avritzer, A., Weyuker, E.J.: The Role of Modeling in the Performance Testing of E-Commerce Applications. IEEE Trans. Software Eng. 30(12) (December 2004)

    Google Scholar 

  2. Avritzer, A., Kondek, J., Liu, D., Weyuker, E.J.: Software performance testing based on workload characterization. In: Proc. 3rd Int. Workshop on Software and Performance (WOSP 2002), Rome, pp. 17–24 (2002)

    Google Scholar 

  3. Balsamo, S., DiMarco, A., Inverardi, P., Simeoni, M.: Model-based Performance Prediction in Software Development. IEEE Trans. Software Eng. 30, 295–310 (2004)

    Article  Google Scholar 

  4. Barber, S.: Creating Effective Load Models for Performance Testing with Incomplete Empirical Data. In: Proc. Sixth IEEE International Workshop on Web Site Evolution (WSE 2004) (2004)

    Google Scholar 

  5. Bertolino, A., Mirandola, R.: CB-SPE Tool: Putting Component-Based Performance Engineering into Practice. In: Proc. 7th Int. Symposium on Component-Based Software Engineering, Edinburgh (2004)

    Google Scholar 

  6. Bondarev, E., Muskens, J., de With, P., Chaudron, M., Lukkien, J.: Predicting Real-Time Properties of Component Assemblies. In: Proc. 30th Euro. Micro. Conf. IEEE, Los Alamitos (2004)

    Google Scholar 

  7. Dang, L., Jennings, C., Kelly, D.: Practical VoIP Using VOCAL. O’Reilly Media, Sebastopol (2002)

    Google Scholar 

  8. Denaro, A.P.G., Emmerich, W.: Early Performance Testing of Distributed Software Applications. In: Proc. 4th Int. Workshop on Software and Performance, Redwood Shores, California, January 2004, pp. 94–103 (2004)

    Google Scholar 

  9. Eskenazi, E., Fioukov, A.V., Hammer, D.K., Obbink, H.: Performance prediction for industrial software with the APPEAR method. In: Proc. 4th Progess Symp. on Embedded Systems (2003)

    Google Scholar 

  10. Franks, R.G., et al.: Performance Analysis of Distributed Server Systems. In: Proc. 6th Int. Conf. on Software Quality, Ottawa, October 28-30, 1996, pp. 15–26 (1996)

    Google Scholar 

  11. Franks, R.G., et al.: Layered Queueing Network Solver and Simulator User Manual, Dept. of Systems and Computer Engineering, Carleton University (December 2005)

    Google Scholar 

  12. Hissam, S.A., et al.: Packaging Predictable Assembly. In: Bishop, J.M. (ed.) CD 2002. LNCS, vol. 2370, pp. 108–124. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  13. Israr, T., Woodside, M., Franks, R.G.: Interaction tree algorithms to extract effective architecture and layered performance models from traces. J. Systems and Software 80(4), 474–492 (2007)

    Article  Google Scholar 

  14. Jazwinski, A.H.: Stochastic Processes and Filtering Theory. Academic Press, London (1970)

    MATH  Google Scholar 

  15. Kutner, M.H., Nachtsheim, C.J., Neter, J., Li, W.: Applied Linear Statistical Models, 5th edn. McGraw-Hill, New York (2005)

    Google Scholar 

  16. Liu, Y., Fekete, A., Gorton, I.: Design-Level Performance Prediction of Component-Based Applications. IEEE Trans. on Software Eng. 31(11) (November 2005)

    Google Scholar 

  17. Miller, B.P., Callaghan, M.D., Cargille, J.M., Hollingsworth, J.K., Irvin, R.B., Karavanic, K.L., Kunchithapadam, K., Newhall, T.: The Paradyn Parallel Performance Measurement Tool. IEEE Computer 28(11) (November 1995)

    Google Scholar 

  18. Musa, J.D.: The Operational Profile in Software Reliability Engineering: An Overview. IEEE Software 10, 14–32 (1993)

    Article  Google Scholar 

  19. Muskens, J., Chaudron, M.: Prediction of Run-Time Resource Consumption in Multi-task Component-Based Software Systems. In: Proc. 7th Int. Symp. on Component-Based Software Engineering, Edinburgh, May 24-25 (2004)

    Google Scholar 

  20. Object Management Group, UML Profile for Schedulability, Performance, and Time Specification, Version 1.1, OMG document formal/05-01-02 (January 2005)

    Google Scholar 

  21. Rosenberg, J., et al.: The Session Initiation Protocol (SIP), IETF RFC 3261 (June 2002), http://www.ietf.org/rfc/rfc3261.txt

  22. Smith, C.U., Williams, L.G.: Performance Solutions. Addison-Wesley, Reading (2002)

    Google Scholar 

  23. Woodside, C.M., Neilson, J.E., Petriu, D.C., Majumdar, S.: The Stochastic Rendezvous Network Model for Performance of Synchronous Client-Server-Like Distributed Software. IEEE Trans. on Computers 44(1), 20–34 (1995)

    Article  MATH  Google Scholar 

  24. Woodside, M., Petriu, D.C., Petriu, D.B., Shen, H., Israr, T., Merseguer, J.: Performance by Unified Model Analysis (PUMA). In: Proc. 5th Int. Workshop on Software and Performance, Palma de Mallorca, pp. 1–12 (2005)

    Google Scholar 

  25. Woodside, M., Franks, R.G., Petriu, D.C.: The Future of Software Performance Engineering. In: Briand, L., Wolf, A. (eds.) Proc. Future of Software Engineering 2007, May 2007, vol. P2829, pp. 171–187. IEEE Computer Society, Los Alamitos (2007)

    Chapter  Google Scholar 

  26. Woodside, M., Zheng, T., Litoiu, M.: Service system resource management based on a tracked layered performance model. In: Proc. 3rd Int. Conf. on Autonomic Computing (ICAC 2006), pp. 123–133 (2006)

    Google Scholar 

  27. Woodside, M.: The Relationship of Performance Models to Data. In: Proc. SPEC Int. Performance Evaluation Workshop (SIPEW 2008). LNCS, vol. 5119, Darmstadt. Springer, Heidelberg (2008)

    Google Scholar 

  28. Wu, X., Woodside, M.: Performance Modeling from Software Components. In: Proc. 4th Int. Workshop on Software and Performance, Redwood Shores, CA, pp. 290–301 (2004)

    Google Scholar 

  29. Zheng, T., Woodside, M., Litoiu, M.: Performance Model Estimation and Tracking using Optimal Filters. IEEE Trans. on Software Eng. (June 2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Systems & Computer Eng., Carleton University, Ottawa, ON, Canada, K1S 5B6

    Xiuping Wu & Murray Woodside

Authors
  1. Xiuping Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Murray Woodside
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Nigel Thomas Carlos Juiz

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Wu, X., Woodside, M. (2008). A Calibration Framework for Capturing and Calibrating Software Performance Models. In: Thomas, N., Juiz, C. (eds) Computer Performance Engineering. EPEW 2008. Lecture Notes in Computer Science, vol 5261. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87412-6_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-87412-6_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-87411-9

  • Online ISBN: 978-3-540-87412-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation