Skip to main content
SpringerLink
Log in

Testing in Multi-Agent Systems

  • Conference paper
Agent-Oriented Software Engineering X (AOSE 2009)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 6038))

Included in the following conference series:

Abstract

Testing software agents and Multi-Agent Systems (MAS) needs suitable techniques to evaluate agent’s autonomous behaviours as well as distribution, social and deliberative properties, which are particular to these systems. This paper reviews testing methods and techniques with respect to the MAS properties they are able to address. For this purpose, we provide a reference framework that provides a classification of MAS testing levels (such as unit, agent, integration, system, and acceptance) and of testing approaches along the development artefact they exploit (namely, design and code artefacts). Open issues in testing MAS are then discussed providing a basis for a research roadmap.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bernon, C., Gleizes, M.P., Picard, G.: Enhancing self-organising emergent systems design with simulation. In: O’Hare, G.M.P., Ricci, A., O’Grady, M.J., Dikenelli, O. (eds.) ESAW 2006. LNCS (LNAI), vol. 4457, pp. 284–299. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  2. Brazier, F.M.T., Dunin-Keplicz, B., Jennings, N.R., Treur, J.: DESIRE: Modelling Multi-Agent Systems in a Compositional Formal Framework. Int. J. Cooperative Inf. Syst. 6(1), 67–94 (1997)

    Article  Google Scholar 

  3. Caire, G., Cossentino, M., Negri, A., Poggi, A., Turci, P.: Multiagent Systems Implementation and Testing. In: Proc. of the 4th From Agent Theory to Agent Implementation Symposium, AT2AI-4 (2004)

    Google Scholar 

  4. Chella, A., Cossentino, M., Sabatucci, L., Seidita, V.: Agile passi: An agile process for designing agents. International Journal of Computer Systems Science & Engineering (2006)

    Google Scholar 

  5. Coelho, R., Kulesza, U., von Staa, A., Lucena, C.: Unit testing in multi-agent systems using mock agents and aspects. In: SELMAS 2006: Proceedings of the 2006 International Workshop on Software Engineering for Large-scale Multi-agent Systems, pp. 83–90. ACM Press, New York (2006), http://dx.doi.org/http://doi.acm.org/10.1145/1138063.1138079

    Chapter  Google Scholar 

  6. Cossentino, M., Fortino, G., Garro, A., Mascillaro, S., Russo, W.: PASSIM: A Simulation-based Process for the Development of Multi-Agent Systems. Int. Journal of Agent-Oriented Software Engineering 2(2), 132–170 (2008)

    Article  Google Scholar 

  7. De Wolf, T.: Analysing and Engineering Self-organising Emergent Applications, PhD thesis, Katholieke Universiteit Leuven (2007)

    Google Scholar 

  8. De Wolf, T., Samaey, G., Holvoet, T.: Engineering self-organising emergent systems with simulation-based scientific analysis. In: Brueckner, S., Serugendo, D.M., Hales, D., Zambonelli, F. (eds.) Third International Workshop on Engineering Self-Organising Application, sUtrech, Netherlands, pp. 146–160 (2005)

    Google Scholar 

  9. Dikenelli, O., Erdur, R.C., Gumus, O.: Seagent: a platform for developing semantic web based multi agent systems. In: AAMAS 2005: Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 1271–1272. ACM Press, New York (2005), http://dx.doi.org/http://doi.acm.org/10.1145/1082473.1082728

    Chapter  Google Scholar 

  10. Ekinci, E.E., Tiryaki, A.M., Cetin, O., Dikenelli, O.: Goal-Oriented Agent Testing Revisited. In: Proc. of the 9th Int. Workshop on Agent-Oriented Software Engineering, pp. 85–96 (2008)

    Google Scholar 

  11. Fortino, G., Garro, A., Russo, W., Caico, R., Cossentino, M., Termine, F.: Simulation-Driven Development of Multi-Agent Systems. In: Workshop on Multi-Agent Systems and Simulation, Palermo, Italia (2006)

    Google Scholar 

  12. Gamma, E., Beck, K.: JUnit: A Regression Testing Framework (2000), http://www.junit.org

  13. Gardelli, L., Viroli, M., Omicini, A.: On the Role of Simulations in the Engineering of Self-Organising MAS: The Case of an Intrusion Detection System in TuCSoN. In: 3rd International Workshop Engineering Self-Organising Applications, pp. 161–175 (2005)

    Google Scholar 

  14. Gómez-Sanz, J.J., Botía, J., Serrano, E., Pavón, J.: Testing and debugging of MAS interactions with INGENIAS. In: Luck, M., Gomez-Sanz, J.J. (eds.) AOSE 2008. LNCS, vol. 5386, pp. 199–212. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  15. Gutknecht, O., Ferber, J., Michel, F.: Integrating tools and infrastructures for generic multi-agent systems. In: AGENTS 2001: Proceedings of the Fifth International Conference on Autonomous Agents, pp. 441–448. ACM, New York (2001)

    Chapter  Google Scholar 

  16. Kaddoum, E., Gleizes, M.-P., Georg, J.-P., Picard, G.: Characterizing and evaluating problem solving self-* systems. In: Dini, P., Gentzsch, W., Geraci, P., Lorenz, P., Singh, K. (eds.) Computation World: Future Computing, Service Computation, Cognitive, Adaptive, Content, Patterns (Adaptive 2009), pp. 137–147. IEEE Computer Society, Los Alamitos (2009)

    Google Scholar 

  17. Kephart, J.O., Chess, D.M.: The vision of autonomic computing. IEEE Computer 36(1), 41–50 (2003)

    Article  Google Scholar 

  18. Kidney, J., Denzinger, J.: Testing the limits of emergent behavior in mas using learning of cooperative behavior. In: Proceeding of the 2006 Conference on ECAI 2006, pp. 260–264. IOS Press, Amsterdam (2006)

    Google Scholar 

  19. Lam, D.N., Barber, K.S.: Debugging agent behavior in an implemented agent system. In: Bordini, R.H., Dastani, M.M., Dix, J., El Fallah Seghrouchni, A. (eds.) PROMAS 2004. LNCS (LNAI), vol. 3346, pp. 104–125. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  20. Lomuscio, A., Raimondi, F.: mcmas: A Model Checker for Multi-agent Systems. In: Hermanns, H., Palsberg, J. (eds.) TACAS 2006. LNCS, vol. 3920, pp. 450–454. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  21. Luck, M., McBurney, P., Shehory, O., Willmott, S.: Agent Technology: Computing as Interaction (A Roadmap for Agent Based Computing), AgentLink (2005)

    Google Scholar 

  22. Moreno, M., Pavón, J., Rosete, A.: Testing in agent oriented methodologies. In: Omatu, S., Rocha, M.P., Bravo, J., Fernández, F., Corchado, E., Bustillo, A., Corchado, J.M. (eds.) IWANN 2009. LNCS, vol. 5518, pp. 138–145. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  23. Nguyen, C.D.: Testing Techniques for Software Agents, PhD thesis, International Doctorate School in Information and Communication Technologies - University of Trento (2008)

    Google Scholar 

  24. Nguyen, C.D., Miles, S., Perini, A., Tonella, P., Harman, M., Luck, M.: Evolutionary testing of autonomous software agents. In: The Eighth International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2009), pp. 521–528. IFAAMAS (2009)

    Google Scholar 

  25. Nguyen, C.D., Perini, A., Tonella, P.: Ontology-based Test Generation for Multi Agent Systems. In: Proc. of the International Conference on Autonomous Agents and Multiagent Systems (2008)

    Google Scholar 

  26. Nguyen, C.D., Perini, A., Tonella, P.: Goal-oriented testing for MASs. Int. J. Agent-Oriented Software Engineering 4(1), 79–109 (2010)

    Article  Google Scholar 

  27. Núñez, M., Rodríguez, I., Rubio, F.: Specification and testing of autonomous agents in e-commerce systems. Software Testing, Verification and Reliability 15(4), 211–233 (2005)

    Article  Google Scholar 

  28. Padgham, L., Winikoff, M.: Developing Intelligent Agent Systems: A Practical Guide. John Wiley and Sons, Chichester (2004)

    Book  MATH  Google Scholar 

  29. Perepletchikov, M., Ryan, C., Frampton, K.: Cohesion metrics for predicting maintainability of service-oriented software. In: QSIC 2007: Proceedings of the Seventh International Conference on Quality Software, pp. 328–335. IEEE Computer Society, Washington (2007)

    Google Scholar 

  30. Perepletchikov, M., Ryan, C., Frampton, K., Tari, Z.: Coupling metrics for predicting maintainability in service-oriented designs. In: Australian Software Engineering Conference, pp. 329–340 (2007)

    Google Scholar 

  31. Poutakidis, D., Winikoff†, M., Padgham, L., Zhang, Z.: Debugging and Testing of Multi-Agent Systems using Design Artefacts. In: Multi-Agent Programming, pp. 215–258 (2009)

    Google Scholar 

  32. Raibulet, C., Masciadri, L.: Towards evaluation mechanisms for runtime adaptivity: from case studies to metrics. In: Dini, P., Gentzsch, W., Geraci, P., Lorenz, P., Singh, K. (eds.) Computation World: Future Computing, Service Computation, Cognitive, Adaptive, Content, Patterns (Adaptive 2009), pp. 146–152. IEEE Computer Society, Los Alamitos (2009)

    Google Scholar 

  33. Serrano, E., Botia, J.A.: Infrastructure for forensic analysis of multi-agent systems. In: Hindriks, K.V., Pokahr, A., Sardina, S. (eds.) ProMAS 2008. LNCS, vol. 5442, pp. 168–183. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  34. Serrano, E., Gómez-Sanz, J.J., Botía, J.A., Pavón, J.: Intelligent data analysis applied to debug complex software systems. Neurocomputing 72(13-15), 2785–2795 (2009)

    Article  Google Scholar 

  35. Sierra, C., Aguilar, J.A.R., Noriega, P., Esteva, M., Arcos, J.L.: Engineering Multi-Agent Systems as Electronic Institutions. Novatica 170, 33–39 (2004)

    Google Scholar 

  36. Sudeikat, J., Renz, W.: A systemic approach to the validation of self–organizing dynamics within MAS. In: Luck, M., Gomez-Sanz, J.J. (eds.) AOSE 2008. LNCS, vol. 5386, pp. 31–45. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  37. Tiryaki, A.M., Öztuna, S., Dikenelli, O., Erdur, R.C.: SUNIT: A unit testing framework for test driven development of multi-agent systems. In: Padgham, L., Zambonelli, F. (eds.) AOSE VII / AOSE 2006. LNCS, vol. 4405, pp. 156–173. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  38. Zhang, Z., Thangarajah, J., Padgham, L.: Automated unit testing for agent systems. In: 2nd International Working Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2007), Barcelona, Spain, pp. 10–18 (2007)

    Google Scholar 

  39. Zhang, Z., Thangarajah, J., Padgham, L.: Automated unit testing intelligent agents in pdt. In: AAMAS (Demos), pp. 1673–1674 (2008)

    Google Scholar 

  40. Zhang, Z., Thangarajah, J., Padgham, L.: Model based testing for agent systems. In: AAMAS, vol. (2), pp. 1333–1334 (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. FBK-IRST, Center for Information Technology, Trento, Italy

    Cu D. Nguyen & Anna Perini

  2. IRIT, University of Toulouse, Toulouse, France

    Carole Bernon

  3. GRASIA, Universidad Complutense de Madrid, Madrid, Spain

    Juan Pavón

  4. RMIT University, Melbourne, Australia

    John Thangarajah

Authors
  1. Cu D. Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anna Perini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Carole Bernon
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Juan Pavón
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. John Thangarajah
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute de Recherche en Infomatique de Toulouse, Paul Sabatier University, IRIT, 118, Route de Narbonne, 31062, Toulouse, Cedex 9, France

    Marie-Pierre Gleizes

  2. Facultad de Informatica, Universidad Complutense de Madrid, Avda. Complutense s/n, 28040, Madrid, Spain

    Jorge J. Gomez-Sanz

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Nguyen, C.D., Perini, A., Bernon, C., Pavón, J., Thangarajah, J. (2011). Testing in Multi-Agent Systems. In: Gleizes, MP., Gomez-Sanz, J.J. (eds) Agent-Oriented Software Engineering X. AOSE 2009. Lecture Notes in Computer Science, vol 6038. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19208-1_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-19208-1_13

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-19207-4

  • Online ISBN: 978-3-642-19208-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation