Systemic design of distributed multi-UAV cooperative decision-making for multi-target tracking

  • Yunyun Zhao
  • Xiangke WangEmail author
  • Chang Wang
  • Yirui Cong
  • Lincheng Shen


In this paper, we consider the cooperative decision-making problem for multi-target tracking in multi-unmanned aerial vehicle (UAV) systems. The multi-UAV decision-making problem is modeled in the framework of distributed multi-agent partially observable Markov decision processes (MPOMDPs). Specifically, the state of the targets is represented by the joint multi-target probability distribution (JMTPD), which is estimated by a distributed information fusion strategy. In the information fusion process, the most accurate estimation is selected to propagate through the whole network in finite time. We propose a max-consensus protocol to guarantee the consistency of the JMTPD. It is proven that the max-consensus can be achieved in the connected communication graph after a limited number of iterations. Based on the consistent JMTPD, the distributed partially observable Markov decision algorithm is used to make tracking decisions. The proposed method uses the Fisher information to bid for targets in a distributed auction. The bid is based upon the reward value of the individual UAV’s POMDPs, thereby removing the need to optimize the global reward in the MPOMDPs. Finally, the cooperative decision-making approach is deployed in a simulation of a multi-target tracking problem. We compare our proposed algorithm with the centralized method and the greedy approach. The simulation results show that the proposed distributed method has a similar performance to the centralized method, and outperforms the greedy approach.


Multi-UAV Decision-making Multi-target tracking Distributed information fusion Max-consensus 


Supplementary material

Supplementary material 1 (mp4 13621 KB)


  1. 1.
    Kassas, Z. M., & Özgüner, Ü. (2010). A nonlinear filter coupled with hospitability and synthetic inclination maps for in-surveillance and out-of-surveillance tracking. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(1), 87–97.CrossRefGoogle Scholar
  2. 2.
    Cesare, K., Skeele, R., Yoo, S. H., Zhang, Y., & Hollinger, G. (2015). Multi-UAV exploration with limited communication and battery. In 2015 IEEE international conference on robotics and automation (ICRA) (pp. 2230–2235). IEEE.Google Scholar
  3. 3.
    Darrah, M., Wilhelm, J., Munasinghe, T., Duling, K., Yokum, S., Sorton, E., et al. (2015). A flexible genetic algorithm system for multi-UAV surveillance: Algorithm and flight testing. Unmanned Systems, 3(1), 49–62.CrossRefGoogle Scholar
  4. 4.
    Capitán, J., Merino, L., & Ollero, A. (2016). Cooperative decision-making under uncertainties for multi-target surveillance with multiples UAVs. Journal of Intelligent & Robotic Systems, 84(1–4), 371–386. Springer.CrossRefGoogle Scholar
  5. 5.
    Caraballo, L., Acevedo, J., Díaz-Báñez, J., Arrue, B., Maza, I., & Ollero, A. (2014). The block-sharing strategy for area monitoring missions using a decentralized multi-UAV system. In 2014 International conference on unmanned aircraft systems (ICUAS) (pp. 602–610). IEEE.Google Scholar
  6. 6.
    Mersheeva, V., & Friedrich, G. (2015). Multi-UAV monitoring with priorities and limited energy resources. In Proceedings of the 25th international conference on automated planning and scheduling (pp. 347–356).Google Scholar
  7. 7.
    Qi, J., Song, D., Shang, H., Wang, N., Hua, C., Wu, C., et al. (2016). Search and rescue rotary-wing UAV and its application to the lushan ms 7.0 earthquake. Journal of Field Robotics, 33(3), 290–321. Wiley Online Library.CrossRefGoogle Scholar
  8. 8.
    Adamey, E., & Ozguner, U. (2011). Cooperative multitarget tracking and surveillance with mobile sensing agents: A decentralized approach. In 2011 14th International IEEE conference on intelligent transportation systems (ITSC) (pp. 1916–1922). IEEE.Google Scholar
  9. 9.
    Dames, P., Tokekar, P., & Kumar, V. (2017). Detecting, localizing, and tracking an unknown number of moving targets using a team of mobile robots. The International Journal of Robotics Research, 36(13–14), 1540–1553. SAGE.CrossRefGoogle Scholar
  10. 10.
    Tokekar, P., Isler, V., & Franchi, A. (2014). Multi-target visual tracking with aerial robots. In 2014 IEEE/RSJ international conference on intelligent robots and systems (IROS 2014) (pp. 3067–3072). IEEE.Google Scholar
  11. 11.
    Bayram, H., Stefas, N., Engin, K. S., & Isler, V. (2017). Tracking wildlife with multiple UAVs: System design, safety and field experiments. In 2017 International symposium on multi-robot and multi-agent systems (MRS) (pp. 97–103). IEEE.Google Scholar
  12. 12.
    Hausman, K., Müller, J., Hariharan, A., Ayanian, N., & Sukhatme, G. S. (2016). Cooperative control for target tracking with onboard sensing. In M. A. Hsieh, O. Khatib, & V. Kumar (Eds.), Experimental robotics (pp. 879–892). Berlin: Springer.CrossRefGoogle Scholar
  13. 13.
    Schlotfeldt, B., Thakur, D., Atanasov, N., Kumar, V., & Pappas, G. J. (2018). Anytime planning for decentralized multirobot active information gathering. IEEE Robotics and Automation Letters, 3(2), 1025–1032. IEEE.CrossRefGoogle Scholar
  14. 14.
    Nestmeyer, T., Giordano, P. R., Bülthoff, H. H., & Franchi, A. (2017). Decentralized simultaneous multi-target exploration using a connected network of multiple robots. Autonomous Robots, 41(4), 989–1011. Springer.CrossRefGoogle Scholar
  15. 15.
    Mahmoud, M. S., & Khalid, H. M. (2013). Distributed Kalman filtering: A bibliographic review. IET Control Theory & Applications, 7(4), 483–501. IET.MathSciNetCrossRefGoogle Scholar
  16. 16.
    Capitan, J., Spaan, M. T., Merino, L., & Ollero, A. (2013). Decentralized multi-robot cooperation with auctioned POMDPs. The International Journal of Robotics Research, 32(6), 650–671. SAGE.CrossRefGoogle Scholar
  17. 17.
    Capitán, J., Merino, L., Caballero, F., & Ollero, A. (2011). Decentralized delayed-state information filter (DDSIF): A new approach for cooperative decentralized tracking. Robotics and Autonomous Systems, 59(6), 376–388. Elsevier.CrossRefGoogle Scholar
  18. 18.
    Adamey, E., & Ozguner, U. (2012). A decentralized approach for multi-UAV multitarget tracking and surveillance. In SPIE Defense, Security, and Sensing (Vol. 8389, pp. 838915-1–838915-6). International Society for Optics and Photonics.Google Scholar
  19. 19.
    Chagas, R. A. J., & Waldmann, J. (2015). A novel linear, unbiased estimator to fuse delayed measurements in distributed sensor networks with application to UAV fleet. In D. Choukroun, Y. Oshman, J. Thienel, & M. Idan (Eds.), Advances in estimation, navigation, and spacecraft control (pp. 135–157). Berlin: Springer.Google Scholar
  20. 20.
    Fanti, M. P., Mangini, A. M., & Ukovich, W. (2012). A quantized consensus algorithm for distributed task assignment. In 2012 IEEE 51st annual conference on decision and control (CDC) (pp. 2040–2045). IEEE.Google Scholar
  21. 21.
    Luo, L., Chakraborty, N., & Sycara, K. (2015). Distributed algorithms for multirobot task assignment with task deadline constraints. IEEE Transactions on Automation Science and Engineering, 12(3), 876–888. IEEE.CrossRefGoogle Scholar
  22. 22.
    Peng, Z., Yang, S., Wen, G., & Rahmani, A. (2014). Distributed consensus-based robust adaptive formation control for nonholonomic mobile robots with partial known dynamics. In Mathematical Problems in Engineering.
  23. 23.
    Seo, J., Kim, Y., Kim, S., & Tsourdos, A. (2012). Consensus-based reconfigurable controller design for unmanned aerial vehicle formation flight. Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, 226(7), 817–829. SAGE.CrossRefGoogle Scholar
  24. 24.
    Maggs, M. K., O’Keefe, S. G., & Thiel, D. V. (2012). Consensus clock synchronization for wireless sensor networks. IEEE Sensors Journal, 12(6), 2269–2277.CrossRefGoogle Scholar
  25. 25.
    Battistelli, G., & Chisci, L. (2014). Kullback–Leibler average, consensus on probability densities, and distributed state estimation with guaranteed stability. Automatica, 50(3), 707–718. Elsevier.MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Palacios-Gasós, J. M., Montijano, E., Sagüés, C., & Llorente, S. (2016). Distributed coverage estimation and control for multirobot persistent tasks. IEEE Transactions on Robotics, 32(6), 1444–1460. IEEE.CrossRefGoogle Scholar
  27. 27.
    Di Paola, D., Petitti, A., & Rizzo, A. (2015). Distributed Kalman filtering via node selection in heterogeneous sensor networks. International Journal of Systems Science, 46(14), 2572–2583. Taylor & Francis.MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Smallwood, R. D., & Sondik, E. J. (1973). The optimal control of partially observable markov processes over a finite horizon. Operations Research, 21(5), 1071–1088. INFORMS.CrossRefzbMATHGoogle Scholar
  29. 29.
    Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1), 99–134. Elsevier.MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Ponda, S. S., Johnson, L. B., Geramifard, A., & How, J. P. (2015). Cooperative mission planning for multi-UAV teams. In K. P. Valavanis & G. J. Vachtsevanos (Eds.), Handbook of unmanned aerial vehicles (pp. 1447–1490). Berlin: Springer.CrossRefGoogle Scholar
  31. 31.
    Oliehoek, F. A. (2012). Decentralized POMDPs. In Reinforcement Learning (Vol. 12, pp. 471–503). Springer.Google Scholar
  32. 32.
    Wu, F., Zilberstein, S., & Chen, X. (2011). Online planning for multi-agent systems with bounded communication. Artificial Intelligence, 175(2), 487–511. Elsevier.MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Vaisenberg, R., Della Motta, A., Mehrotra, S., & Ramanan, D. (2014). Scheduling sensors for monitoring sentient spaces using an approximate POMDP policy. Pervasive and Mobile Computing, 10, 83–103. Elsevier.CrossRefGoogle Scholar
  34. 34.
    Panella, A., & Gmytrasiewicz, P. (2017). Interactive POMDPs with finite-state models of other agents. Autonomous Agents and Multi-Agent Systems, 31(4), 861–904.CrossRefGoogle Scholar
  35. 35.
    Yu, H., Meier, K., Argyle, M., & Beard, R. W. (2015). Cooperative path planning for target tracking in urban environments using unmanned air and ground vehicles. IEEE/ASME Transactions on Mechatronics, 20(2), 541–552. IEEE.CrossRefGoogle Scholar
  36. 36.
    Farmani, N., Sun, L., & Pack, D. (2015). Tracking multiple mobile targets using cooperative unmanned aerial vehicles. In 2015 International conference on unmanned aircraft systems (ICUAS) (pp. 395–400). IEEE.Google Scholar
  37. 37.
    Zhang, K., Collins, E. G, Jr., & Shi, D. (2012). Centralized and distributed task allocation in multi-robot teams via a stochastic clustering auction. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 7(2), 21.Google Scholar
  38. 38.
    Edalat, N., Tham, C. K., & Xiao, W. (2012). An auction-based strategy for distributed task allocation in wireless sensor networks. Computer Communications, 35(8), 916–928. Elsevier.CrossRefGoogle Scholar
  39. 39.
    Spaan, M. T., Veiga, T. S., & Lima, P. U. (2015). Decision-theoretic planning under uncertainty with information rewards for active cooperative perception. Autonomous Agents and Multi-Agent Systems, 29(6), 1157–1185. Springer.CrossRefGoogle Scholar
  40. 40.
    Zhao, Y., Wang, X., Cong, Y., & Shen, L. (2018). Information geometry based action decision-making for target tracking by fixed-wing UAV: From algorithm design to theory analysis. International Journal of Advanced Robotic Systems, 15(4), 1729881418787061.CrossRefGoogle Scholar
  41. 41.
    Ragi, S., & Chong, E. K. (2013). Uav path planning in a dynamic environment via partially observable Markov decision process. IEEE Transactions on Aerospace and Electronic Systems, 49(4), 2397–2412. IEEE.CrossRefGoogle Scholar
  42. 42.
    Burguera, A., González, Y., & Oliver, G. (2009). Sonar sensor models and their application to mobile robot localization. Sensors, 9(12), 10217–10243. Molecular Diversity Preservation International.CrossRefGoogle Scholar
  43. 43.
    Nejad, B. M., Attia, S. A., & Raisch, J. (2009). Max-consensus in a max-plus algebraic setting: The case of fixed communication topologies. In XXII international symposium on information, communication and automation technologies, 2009 (ICAT 2009) (pp. 1–7). IEEE.Google Scholar
  44. 44.
    Petitti, A., Di Paola, D., Rizzo, A., & Cicirelli, G. (2011). Consensus-based distributed estimation for target tracking in heterogeneous sensor networks. In 2011 50th IEEE conference on decision and control and European control conference (CDC-ECC) (pp. 6648–6653). IEEE.Google Scholar
  45. 45.
    Bui, M., Butelle, F., & Lavault, C. (2004). A distributed algorithm for constructing a minimum diameter spanning tree. Journal of Parallel and Distributed Computing, 64(5), 571–577. Elsevier.CrossRefzbMATHGoogle Scholar
  46. 46.
    Burkard, R. E. (2002). Selected topics on assignment problems. Discrete Applied Mathematics, 123(1), 257–302. Elsevier.MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Rao, C. R. (1992). Information and the accuracy attainable in the estimation of statistical parameters. In N. L. Johnson, A. W. Kemp, & S. Kotz (Eds.), Breakthroughs in statistics (pp. 235–247). Berlin: Springer.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Yunyun Zhao
    • 1
  • Xiangke Wang
    • 1
    Email author
  • Chang Wang
    • 1
  • Yirui Cong
    • 1
  • Lincheng Shen
    • 1
  1. 1.College of Intelligence Science and TechnologyNational University of Defense TechnologyChangshaChina

Personalised recommendations