Hybrid Formation Control for Multi-Robot Hunters Based on Multi-Agent  Deep Deterministic Policy Gradient

Oussama Hamed; Mohamed Hamlich

doi:10.13164/mendel.2021.2.023

Oussama Hamed Complex Cyber Physical Systems Laboratory,ENSAM, Casablanca, University of Hassan II, Morocco
Mohamed Hamlich Complex Cyber Physical Systems Laboratory,ENSAM, Casablanca, University of Hassan II, Morocco

DOI: https://doi.org/10.13164/mendel.2021.2.023

Keywords: Reinforcement learning, Multi-Robot System, Cooperative hunting, Path Planning, Mobile robot, Collaborative robots

Abstract

The cooperation between mobile robots is one of the most important topics of interest to researchers, especially in the many areas in which it can be applied. Hunting a moving target with random behavior is an application that requires robust cooperation between several robots in the multi-robot system. This paper proposed a hybrid formation control for hunting a dynamic target which is based on wolves’ hunting behavior in order to search and capture the prey quickly and avoid its escape and Multi Agent Deep Deterministic Policy Gradient (MADDPG) to plan an optimal accessible path to the desired position. The validity and the effectiveness of the proposed formation control are demonstrated with simulation results.

References

R. Konda, H. M. La and J. Zhang, "Decentralized Function Approximated Q-Learning in Multi-Robot Systems for Predator Avoidance," in IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 6342-6349, Oct. 2020, doi: 10.1109/LRA.2020.3013920.

Oussama Hamed, Mohamed Hamlich, Year: 2021, Improvised multi-robot cooperation strategy for hunting a dynamic target, IOT, EAI, DOI: 10.4108/eai.8-2-2021.168691

Cai L, Sun Q. Multiautonomous underwater vehicle consistent collaborative hunting method based on generative adversarial network. International Journal of Advanced Robotic Systems. May 2020. doi:10.1177/1729881420925233

X. Cao and X. Xu, "Hunting Algorithm for Multi-AUV Based on Dynamic Prediction of Target Trajectory in 3D Underwater Environment," in IEEE Access, vol. 8, pp. 138529-138538, 2020, doi: 10.1109/ACCESS.2020.3013032.

Ma,J , Lu, H., Xiao, J. et al. Multi-robot Target Encirclement Control with Collision Avoidance via Deep Reinforcement Learning. J Intell Robot Syst 99, 371–386 (2020). https://doi.org/10.1007/s10846-019-01106-x

Won Joon Yun, Soyi Jung, Joongheon Kim, Jae-Hyun Kim, Distributed deep reinforcement learning for autonomous aerial eVTOL mobility in drone taxi applications, ICT Express, Volume 7, Issue 1,2021,Pages 1-4,ISSN 2405-9595, https://doi.org/10.1016/j.icte.2021.01.005.

R. K. Dewangan, A. Shukla, W. W. Godfrey, A solution for priority-based multi-robot path planning problem with obstacles using ant lion optimization, Mod. Phys. Lett. B, 34 (2020), 2050137.

Lowe, R.; Wu, Y.; Tamar, A.; Harb, J.; Abbeel, P.; and Mordatch, I. 2017. Multi-agent actor-critic for mixed cooperative-competitive environments. arXiv preprint arXiv:1706.02275.

O. Hamed and M. Hamlich, "Improvised multi-robot cooperation strategy for hunting a dynamic target," 2020 International Symposium on Advanced Electrical and Communication Technologies (ISAECT), 2020, pp. 1-4, doi: 10.1109/ISAECT50560.2020.9523684.

Zheng, Yanbin, Fan, Wenxin, and Han, Mengyun. ‘Research on Multi-agent Collaborative Hunting Algorithm Based on Game Theory and Q-learning for a Single Escaper’. 1 Jan. 2021 : 205 – 219.

Xiang Cao & Fen Zuo (2021) A fuzzy-based potential field hierarchical reinforcement learning approach for target hunting by multi-AUV in 3-D underwater environments, International Journal of Control, 94:5, 1334-1343, DOI: 10.1080/00207179.2019.1648875

K. Zhu, Q. Zong and R. Zhang, "Real-time Virtual Simulation Platform for Multi-UVA hunting target using Deep Reinforcement Learning," 2021 40th Chinese Control Conference (CCC), 2021, pp. 4978-4983, doi: 10.23919/CCC52363.2021.9550261.

Parak, R. and Matousek, R. 2021. Comparison of Multiple Reinforcement Learning and Deep Reinforcement Learning Methods for the Task Aimed at Achieving the Goal. MENDEL. 27, 1 (Jun. 2021), 1-8. DOI:https://doi.org/10.13164/mendel.2021.1.001.

Woojun Kim, Jongeui Park, and Youngchul Sung. 2021. Communication in MultiAgent Reinforcement Learning: Intention Sharing. In International Conference on Learning Representations. https://openreview.net/forum?id=qpsl2dR9twy

Kim, W., Cho, M., & Sung, Y. (2019). Message-Dropout: An Efficient Training Method for Multi-Agent Deep Reinforcement Learning. https://doi.org/10.1609/aaai.v33i01.33016079 Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 6079-6086.