Analyzing Stigmergetic Algorithms Through Automata Games

Abstract

Stigmergy describes a class of mechanisms that mediate animal to animal interaction through the environment. Recently this concept has proved interesting for use in multi-agent systems, as it provides a simple framework for agent interaction and coordination. However, determining the global system behavior that will arise from local stigmergetic interactions is a complex problem. In this paper stigmergetic mechanisms are modeled using simple reinforcement learners, called learning automata. We show that using automata to model stigmergy, the learning problem can be asymptotically approximated by an automata game. Existing convergence results for automata games enables us to understand these stigmergetic methods and predict their global behavior. A simple multi-pheromone example is described and analyzed through its corresponding automata game.