Abstract
This paper introduces semantic mutation testing (SMT) into multi-agent systems. SMT is a test assessment technique that makes changes to the interpretation of a program and then examines whether a given test set has the ability to detect each change to the original interpretation. These changes represent possible misunderstandings of how the program is interpreted. SMT can also be used to assess robustness to and reliability of semantic changes. This paper applies SMT to three rule-based agent programming languages, namely Jason, GOAL and 2APL, provides several contexts in which SMT for these languages is useful, and proposes three sets of semantic mutation operators (i.e., rules to make semantic changes) for these languages respectively, and a systematic approach to derivation of semantic mutation operators for rule-based agent languages. This paper then shows, through preliminary evaluation of our semantic mutation operators for Jason, that SMT has some potential to assess tests, robustness to and reliability of semantic changes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This rule can be relaxed, namely mutating the semantics of only parts of the program instead of mutating the semantics of the whole program. This is useful, e.g., when the program is developed by several people.
- 2.
A trusted test set is the one that is considered to be “good enough” for the requirement. It doesn’t need to be the full test set that is usually impractical; instead it can choose not to cover some aspects or to tolerate some errors.
- 3.
Here the term “equivalent” is different to the one used in the context of test assessment, in which a mutant is equivalent only if there exist no tests that can kill the mutant. In the context of robustness/reliability assessment, a mutant is equivalent if only the trusted test set cannot kill it.
- 4.
Unlike Jason and 2APL, a GOAL agent has no intention set or similar structure, so a GOAL rule is immediately attempted to execute to completion once selected.
- 5.
GOAL supports four available rule evaluation orders: linear, linearall, random and randomall.
- 6.
We ever considered two more kinds of the interpreter behaviour, namely transit (between deliberation steps) and execute (a rule or action). However, we find that these two kinds can be classified as select, namely select between deliberation steps and select a rule or action to execute. This simplifies our classification.
- 7.
These changeable aspects may have overlaps, e.g., the change “select one rule \( \to \) select all rules” can be a change to the order or the quantity.
- 8.
The tolerance value is added because the exact time taken by the Jason project varies over a limited range in different runs. It is generous because the execution efficiency is not considered as part of the test pass criteria.
References
Adra, S.F., McMinn, P.: Mutation operators for agent-based models. In: Proceedings of 5th International Workshop on Mutation Analysis. IEEE Computer Society (2010)
Ammann, P., Offutt, J.: Introduction to Software Testing. Cambridge University Press, New York (2008)
Bordini, R.H., Hübner, J.F.: Semantics for the Jason variant of AgentSpeak (plan failure and some internal actions). In: Proceedings of ECAI 2010, pp. 635–640 (2010)
Bordini, R.H., Hübner, J.F., Wooldridge, M.: Programming Multi-Agent Systems in AgentSpeak using Jason. Wiley, Hoboken (2007)
Clark, J.A., Dan, H., Hierons, R.M.: Semantic Mutation Testing. Science of Computer Programming (2011)
Dastani, M.: 2APL: a practical agent programming language. Auton. Agent. Multi-Agent Syst. 16(3), 214–248 (2008)
Dastani, M., van Riemsdijk, M.B., Meyer, J.J.C.: Programming multi-agent systems in 3APL. In: Bordini, R.H., Dastani, M., Dix, J., El Fallah Seghrouchni, A. (eds.) Multi-Agent Programming. Languages, Platforms and Applications, pp. 39–67. Springer, Heidelberg (2005)
Hindriks, K.V.: Programming rational agents in GOAL. In: Bordini, R.H., Dastani, M., Dix, J., El Fallah Seghrouchni, A. (eds.) Multi-agent programming. Languages, platforms and applications, vol. 2, pp. 3–37. Springer, Heidelberg (2009)
Houhamdi, Z.: Multi-agent system testing: a survey. Int. J. Adv. Comput. Sci. Appl. (IJACSA) 2(6), 135–141 (2011)
Huang, Z., Alexander, R., Clark, J.: Mutation testing for Jason agents. In: Dalpiaz, F., Dix, J., van Riemsdijk, M. (eds.) EMAS 2014. LNCS, vol. 8758, pp. 309–327. Springer, Heidelberg (2014)
Jason changelog. http://sourceforge.net/p/jason/svn/HEAD/tree/trunk/release-notes.txt
JGit documentation. https://eclipse.org/jgit/documentation/
JRebel documentation. http://zeroturnaround.com/software/jrebel/learn/
Mathur, A.P.: Foundations of Software Testing. Pearson, New Delhi (2008)
Nguyen, C.D., Perini, A., Bernon, C., Pavón, J., Thangarajah, J.: Testing in multi-agent systems. In: Gomez-Sanz, J.J. (ed.) AOSE 2009. LNCS, vol. 6038, pp. 180–190. Springer, Heidelberg (2011)
Saifan, A.A., Wahsheh, H.A.: Mutation operators for JADE mobile agent systems. In: Proceedings of the 3rd International Conference on Information and Communication Systems, ICICS (2012)
Savarimuthu, S., Winikoff, M.: Mutation operators for cognitive agent programs. In: Proceedings of the 2013 International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2013), pp. 1137–1138 (2013)
Savarimuthu, S., Winikoff, M.: Mutation operators for the GOAL agent language. In: Winikoff, M. (ed.) EMAS 2013. LNCS, vol. 8245, pp. 255–273. Springer, Heidelberg (2013)
Tiryaki, A.M., Öztuna, S., Dikenelli, O., Erdur, R.C.: SUNIT: a unit testing framework for test driven development of multi-agent systems. In: Padgham, L., Zambonelli, F. (eds.) AOSE VII/AOSE 2006. LNCS, vol. 4405, pp. 156–173. Springer, Heidelberg (2007)
Winikoff, M.: Novice programmers’ faults & failures in GOAL programs. In: Proceedings of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2014), pp. 301–308 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Huang, Z., Alexander, R. (2015). Semantic Mutation Testing for Multi-agent Systems. In: Baldoni, M., Baresi, L., Dastani, M. (eds) Engineering Multi-Agent Systems. EMAS 2015. Lecture Notes in Computer Science(), vol 9318. Springer, Cham. https://doi.org/10.1007/978-3-319-26184-3_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-26184-3_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26183-6
Online ISBN: 978-3-319-26184-3
eBook Packages: Computer ScienceComputer Science (R0)