ROS Hybrid Behaviour Planner: Behaviour Hierarchies and Self-organisation in the Multi-Agent Programming Contest

Abstract

While the decision-making and planning framework ROS Hybrid Behaviour Planner (RHBP) has been used in a wide variety of projects, newer features have not yet been tested in complex scenarios. One of those features allows creating multiple independent levels of decision-making by encapsulating a separate behaviour network into behaviours. Another one is an extension for implicit coordination through self-organisation. This paper discusses our system that was developed for the multi-agent contest 2018 using RHBP, while especially making use of newer features wherever possible. Our team TUBDAI achieved the shared top spot in the contest, showing that RHBP and in particular the new features can be used successfully in a complex scenario and measures up to the multi-agent frameworks, other teams have used. Especially, when a last-minute change to the contest environment required us to integrate substantial strategy changes in last-minute, it turned out that RHBP fostered adaptiveness during our development.

1 Team Overview: Short Answers

1.1 1.1 Participants and Their Background

• What was your motivation to participate in the contest?

  The motivation was to further evaluate the decision-making and planning framework ROS Hybrid Behaviour Planner (RHBP). While the framework has been used in a wide variety of projects (also in MAPC 2017), newer features have not been tested in complex scenarios. One of those features allows to create multiple independent levels of decision making by encapsulating a separate behaviour network into a behaviour. Another one is an extension for implicit coordination on the foundation of self-organisation.

• What is the history of your group? (course project, thesis, ...)

  Researchers of the DAI-Labor started to participate in the contest in 2007. Since then they have contributed to every edition of the contest and have won four of them using successive generations of the JIAC multi-agent framework. The TUBDAI 2018 team originates from a Master’s Thesis student and its supervising PhD student. The applied framework RHBP is developed in one Ph.D. thesis and several independent Bachelor and Master’s theses.

• What is your field of research? Which work therein is related?

  Our field of research is multi-agent systems applied in the robotics domain.

1.2 1.2 Development

• How much time did you invest in the contest for programming vs. other tasks (for example organization)? creating vs. optimizing your agents?

  We invested approximately 600 h without time for framework development and the communication proxy (mac_ros_bridge). The mac_ros_bridge maps the xml-based socket communication of the MASSim simulation server to ROS communication means (which was also partly reused from MAPC 2017). Furthermore, 400 h of the invested time budget are spend on programming tasks while 200 h are used for optimising our approach.

• How many lines of code did you produce for your final agent team?

  The scenario specific code contains approximately 7000 LOC.

• How many people were involved and to which degree?

  Christopher-Eyk Hrabia (Ph.D. Student at Technische Universität Berlin) provided the general supervision, was especially responsible for the consultation about scientific approaches as well as giving technical support for the RHBP framework and its application.

  Michael Franz Ettlinger (M.Sc. Student at Technische Universität Berlin) was responsible for the scenario specific implementation and execution of the contest.

  Axel Hessler (Post-Doc at Technische Universität Berlin) was responsible for the infrastructure and overall administration.

• When did you start working on your agents?

  The major work started mid May 2018, communication infrastructure (e.g. mac_ros_bridge) was already done mid of April 2018.

1.3 1.3 System Details

• How do your agents work together? (coordination, information sharing, ...)

  Information sharing for implicit coordination (exploration, opponent well states) as well as explicit coordination (jobs) through a contract-net protocol implementation.

• What are critical components of your team?

  The most critical component is the job planning component which coordinates the job tasks amongst the agents.

• Can your agents change their behaviour during runtime? If so, what triggers the changes?

  Yes. The agents select the most appropriate behaviour based on the current perception and the results of the hybrid planning decision-making component of RHBP.

• Did you have to make changes to the team (e.g. fix critical bugs) during the contest?

  No.

• How do you organize your agents? Do you use e.g. hierarchies? Is your organization implicit or explicit?

  No hierarchy. We partly use implicit (self-organised) and partly explicit (contract-net protocol) coordination, see above.

• Is most of your agents’ behaviour emergent on an individual or team level?

  All behaviour if it was possible emerge from individual level, which results from the autonomously taken decision by each individual agent.

• If your agents perform some planning, how many steps do they plan ahead?

  They plan one task ahead. A task can technically have unlimited amount of steps but practically has no more than 40 simulation steps.

• If you have a perceive-think-act cycle, how is it synchronized with the server?

  Our perceive-think-act cycle is performed as quick as possible as soon as the server delivers the percept of the current simulation step. Through enough calculation power it was made sure that the actions are always delivered in time.

• How did you go about debugging your system?

  We applied three different debugging techniques. First, RHBP offers extensions visualisation and monitoring of behaviours and their internal states. Secondly, agents can be started in development environment and analysed with a normal Python debugger. Thirdly, we used custom log messages to analyse the runtime behaviour without interference.

• Which operating system did you use, and is your team portable to other operating systems?

  We used Ubuntu 16.04, our solution is portable to all other Linux distributions that have ROS support. Execution on Windows is also possible through the Windows Subsystem for Linux (WSL) using a Ubuntu-binding.

• What hardware did you use to run your agent team? (RAM, CPU, disk space, multiple computers, any other notable requirements)

  Intel(R) Core(TM) i7-4930K @ 3.40 GHz CPU (6 cores with hyper-threading), 32 GB RAM and a Samsung SSD 840. It is the same machine that was used already by our team in MAPC 2017.

1.4 1.4 Scenario and Strategy

• What is the main strategy of your agent team?

  Our strategy has three main tiers. Stockpiling items as well as assembled items. Building wells at positions that are only accessible for special agents (drones). Attacking opponent wells aggressively.

• How do your agents decide which jobs to complete?

  If all required items are on stock the jobs are completed.

• Do you have different strategies for the different roles?

  Yes, only drones build wells and one drone is responsible for well exploration at the map border.

• Do your agents form ad-hoc teams to complete a task?

  Yes, the sub-tasks of a job are coordinated ad-hoc with a contract-net protocol implementation. Here, all agents participate in the auctions that are used for the task assignment.

• How do your agents decide when and where to build wells?

  Random places that can’t be reached by other agents (off-road). We use the Graphhopper back-end to determine which map locations are not accessible by road agents.

• If your agents accumulated a lot of currency, why did they not spend it immediately?

  Our strategy requires drones to execute the well building, due to the fact that drones are comparable inefficient in building it is possible that we accumulate currency if the drones are not able to build fast enough.

1.5 1.5 And the Moral of It Is ...

• What did you learn from participating in the contest?

• What are the strong and weak points of your team?

  The strategy decisions proved to be a viable solution for the contest. The stockpile with feedback strategy worked good enough to produce the required money (massium) for wells, while providing enough idle agent time for other tasks. If the contest would have been about massium alone (like last year) the strategy in its current implementation would likely preform worse. In such a scenario the use of storages could massively improve the massium output but would reduce the availability of agents for well building tasks. The off-road well locations strategy was not expected by any opponent and therefore performed well. The main drawback of the strategy was that if it is expected, it is easy to counter. This was observable in the match against SMART JaCaMo, who were able to adapt their strategy in a few hours to beat our team. The only assumption that turned out wrong was that we expected opponents to try to defend their wells once they were built. RHBP proved to be a great framework to use for the project. After an initial learning period, RHBP bore out to be robust and allowed agents to adapt well to changes at run time. The implementation was robust and performed well during the contest. The assembly coordination strategy worked well but resulted in many empty coordination cycles. If this would have been implemented using a client initiating contract net protocol, its performance as well as simplicity would likely have increased.

• How viable were your chosen programming language, methodology, tools, and algorithms?

  One goal of this project was to use RHBP and its new features and extensions, evaluate them and offer improvement suggestions. RHBP was used quite successfully and allowed to create a fast, adaptive and flexible solution for the contest. It also allowed quick and robust changes to the strategy as discussed in the evaluation. The run-time adaptiveness of RHBP could be observed at the match against Akuanduba-UDESC. While many resources were usually used for dismantling opponent wells, there was no dismantling required against Akuanduba-UDESC, which freed up resources for other tasks.

• Did you encounter new problems during the contest?

  We have been able to find several bugs and performance bottle necks in our SoBuffer library, which is used for communicating and handling messages for self-organisation.

• Did playing against other agent teams bring about new insights on your own agents?

  We did not gain major insights, we could only prove the runtime adaptation capabilities in situations we have not especially considered before.

• What would you improve if you wanted to participate in the same contest a week from now (or next year)?

  We would less emphasis on job completion and would add a well defence strategy.

• Which aspect of your team cost you the most time?

  Implementing the job coordination and execution.

• Why did your team perform as it did? Why did the other teams perform better/worse than you did?

  Because other teams didn’t expect off-road well locations and our solution adapted robustly to different situations in the games.

1.6 1.6 The Future of the MAPC

• What can be improved regarding the contest for next year?

  Due to incidents in this years contest, we propose to make handing in code before contest obligatory. Furthermore, we think code changes should not be allowed, which also solves the problem of fair schedules. Maybe it could also be a good approach to run everything on the same virtual machines or docker containers, which are running in the organisers department to avoid problems with connection performance or too much deviating hardware requirements. Furthermore, we encourage to focus more on decentralisation and autonomous agent development. while avoiding the focus on optimisation problems.

• What kind of scenario would you like to play next? What kind of features should the new scenario have? We would suggest to have a scenario that requires less optimisation of a scenario specific problem, highlighting more features of intelligent agents such as being adaptive, able to learn, robust, ...

• Should the teams be allowed to make changes to their agents during the contest (even based on opponent agent behaviour observed in earlier matches)? If yes, should only some changes be legal and which ones (e.g. bugfixes), and how to decide a change’s classification? If no, should we ensure that no changes are made and how?

  Changes should not be allowed because having modifications during the contest defeats the purpose of finding a great strategy as well as autonomous decision making when developers make decisions based on their observations.

  Even more, we propose to enforce a code submission before the contest starts. Bugfixes could potentially be allowed but would have to go through a peer-reviewed pull request. For organisational reasons the review of the pull request could also be done after the contest.

• Do you have ideas to reduce the impact of unforeseen strategies (e.g., playing a second leg after some time)?

  As long as the strategies are done on the foundation of the provided API not exploiting bugs everything should be allowed. Even more, unforeseen strategies should be encouraged.

  If the organisers want to prevent this (which we don’t think they should), they could request a detailed strategy description to make sure they agree that the strategy is “expected” and not “unforeseen”.