A Literature Analysis-Based Study on Advances in Underwater Multi-Robot Pursuit-Evasion Problems

LEI Zhenkun; CHEN Mingzhi; ZHU Daqi

doi:10.11993/j.issn.2096-3920.2025-0032

Volume 33 Issue 3

Jun 2025

Turn off MathJax

Article Contents

Article Navigation > Journal of Unmanned Undersea Systems > 2025 > 33(3): 484-494

LEI Zhenkun, CHEN Mingzhi, ZHU Daqi. A Literature Analysis-Based Study on Advances in Underwater Multi-Robot Pursuit-Evasion Problems[J]. Journal of Unmanned Undersea Systems, 2025, 33(3): 484-494. doi: 10.11993/j.issn.2096-3920.2025-0032

Citation:

LEI Zhenkun, CHEN Mingzhi, ZHU Daqi. A Literature Analysis-Based Study on Advances in Underwater Multi-Robot Pursuit-Evasion Problems[J]. Journal of Unmanned Undersea Systems, 2025, 33(3): 484-494. doi: 10.11993/j.issn.2096-3920.2025-0032

Citation:

PDF( 1347 KB)

A Literature Analysis-Based Study on Advances in Underwater Multi-Robot Pursuit-Evasion Problems

doi: 10.11993/j.issn.2096-3920.2025-0032

School of Mechanical Engineering, University of Shanghai for Science and Technology, Shanghai 200093, China

Received Date: 2025-02-26
Accepted Date: 2025-05-08
Rev Recd Date: 2025-04-21

Available Online: 2025-06-05

Abstract

Abstract

Investigating the applications and challenges of multi-robot pursuit-evasion problems in underwater environments holds significant importance for enhancing the autonomous decision-making and collaborative capabilities of underwater robot systems. By searching the Web of Science Core Collection database, over 2 200 relevant literatures published between 2004 and 2024 were screened, and a comprehensive analysis was conducted on the definition of pursuit-evasion problems, research status, intelligent pursuit-evasion methods, and their applications in underwater environments. The principles, advantages, disadvantages, and applicability of four intelligent pursuit-evasion methods, including reinforcement learning, model predictive control, Apollonius circle, and artificial potential field, were analyzed in depth. The study reveals that reinforcement learning optimizes strategies through training to adapt to complex environments but suffers from a long training cycle; model predictive control formulates strategies based on future state predictions, boasting high accuracy but facing real-time challenges; the Apollonius circle optimizes paths using geometric relationships; and the artificial potential field method guides robots with virtual force fields. In underwater environments, robot pursuit-evasion games confront multiple challenges, such as ocean current disturbances and limited communication. This paper summarizes the application potential and existing issues of current methods in underwater environments and proposes future research directions, including the development of more efficient and adaptive intelligent pursuit-evasion algorithms, so as to address the technical requirements of complex underwater environments and provide theoretical references for designing pursuit-evasion strategies for underwater multi-robot systems.
- multi-robot systems,
- pursuit-evasion,
- reinforcement learning,
- model predictive control,
- underwater robot

FullText(HTML)

References(46)

References

[1]	MU Z, PAN J, ZHOU Z, et al. A survey of the pursuit–evasion problem in swarm intelligence[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(8): 1093-1116.
[2]	KANE S A, ZAMANI M. Falcons pursue prey using visual motion cues: New perspectives from animal-borne cameras[J]. Journal of Experimental Biology, 2014, 217(2): 225-234. doi: 10.1242/jeb.092403
[3]	CHAKRABORTY D, BHUNIA S, DE R. Survival chances of a prey swarm: How the cooperative interaction range affects the outcome[J]. Scientific Reports, 2020, 10(1): 8362. doi: 10.1038/s41598-020-64084-3
[4]	ISAACS R. Differential games: A mathematical theory with applications to warfare and pursuit, control and optimization[M]. Gloucester, MA, USA: Courier Corporation, 1999.
[5]	MURRIETA-CID R, TOVAR B, HUTCHINSON S. A sampling-based motion planning approach to maintain visibility of unpredictable targets[J]. Autonomous Robots, 2005, 19: 285-300. doi: 10.1007/s10514-005-4052-0
[6]	BHATTACHARYA S, HUTCHINSON S. On the existence of Nash equilibrium for a two-player pursuit—evasion game with visibility constraints[J]. The International Journal of Robotics Research, 2010, 29(7): 831-839. doi: 10.1177/0278364909354628
[7]	TANG X, YE D, XIAO Y, et al. Epsilon Nash equilibrium differential game strategy for spacecraft terminal pursuit-evasion under incomplete information[J]. Journal of Astronautics, 2024, 45(1): 63-73. doi: 10.3873/j.issn.1000-1328.2024.01.007
[8]	PACHTER M, D’AZZO J J, PROUD A W. Tight formation flight control[J]. Journal of Guidance, Control, and Dynamics, 2001, 24(2): 246-254. doi: 10.2514/2.4735
[9]	GARCIA E, CASBEER D W, PACHTER M. Design and analysis of state-feedback optimal strategies for the differential game of active defense[J]. IEEE Transactions on Automatic Control, 2018, 64(2): 553-568.
[10]	JIA J, WANG W. Review of reinforcement learning research[C]//2020 35th Youth Academic Annual Conference of Chinese Association of Automation(YAC). Zhanjiang, China: IEEE, 2020: 186-191.
[11]	KAELBLING L P, LITTMAN M L, MOORE A W. Reinforcement learning: A survey[J]. Journal of Artificial Intelligence Research, 1996, 4: 237-285. doi: 10.1613/jair.301
[12]	SUTTON R S, BARTO A G. Reinforcement learning: An introduction[M]. Cambridge, MA, USA: MIT Press, 2018.
[13]	WANG Y K, WANG Y, CUI R X, et al. Game of marine robots: USV pursuit evasion game using online reinforcement learning[C]//2023 IEEE International Conference on Development and Learning(ICDL). Macau, China: IEEE, 2023: 121-126.
[14]	LI Z, CHEN S, ZHOU C, et al. Orbital multi-player pursuit-evasion game with deep reinforcement learning[J]. Journal of the Astronautical Sciences, 2025, 72(1): 1-29.
[15]	QI Q, ZHANG X, GUO X. A deep reinforcement learning approach for the pursuit evasion game in the presence of obstacles[C]//2020 IEEE International Conference on Real-time Computing and Robotics(RCAR). Asahikawa, Japan: IEEE, 2020: 68-73.
[16]	ZHANG R, ZONG Q, ZHANG X, et al. Game of drones: Multi-UAV pursuit-evasion game with online motion planning by deep reinforcement learning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 34(10): 7900-7909.
[17]	ZHUANG H, GAO P, WU X, et al. Coordination and control in multiagent systems for enhanced pursuit-evasion game performance[J]. Dynamic Games and Applications, 2024: 1-22.
[18]	MORARI M, LEE J H. Model predictive control: Past, present and future[J]. Computers & Chemical Engineering, 1999, 23(4-5): 667-682.
[19]	GARCIA C E, PRETT D M, MORARI M. Model predictive control: Theory and practice—a survey[J]. Automatica, 1989, 25(3): 335-348. doi: 10.1016/0005-1098(89)90002-2
[20]	HUANG D, ZHANG M, ZHAN T, et al. Insensitive mechanism-based nonlinear model predictive guidance for UAVs intercepting maneuvering targets with input constraints[J]. Drones, 2024, 8(11): 608. doi: 10.3390/drones8110608
[21]	QIU T, ZHANG H, WANG J. Nash pursuit strategy for nonzero-sum MPC game via inverse optimal control[C]//2022 13th Asian Control Conference(ASCC). Jeju, Korea: IEEE, 2022: 2354-2359.
[22]	HU P, ZHAO C, PAN Q. A novel method for a pursuit-evasion game based on fuzzy Q-learning and model-predictive control[J]. Drones, 2024, 8(9): 509. doi: 10.3390/drones8090509
[23]	CUI J, LI D, LIU P, et al. Game-model prediction hybrid path planning algorithm for multiple mobile robots in pursuit evasion game[C]//2021 IEEE International Conference on Unmanned Systems(ICUS). Beijing, China: IEEE, 2021: 925-930.
[24]	GISCH D, RIBANDO J M. Apollonius’ problem: A study of solutions and their connections[J]. American Journal of Undergraduate Research, 2004, 3(1): 15-25.
[25]	COXETER H S M. The problem of Apollonius[J]. The American Mathematical Monthly, 1968, 75(1): 5-15. doi: 10.1080/00029890.1968.11970941
[26]	RAMANA M V, KOTHARI M. Pursuit-evasion games of high speed evader[J]. Journal of Intelligent & Robotic Systems, 2017, 85: 293-306.
[27]	DOROTHY M, MAITY D, SHISHIKA D, et al. One Apollonius circle is enough for many pursuit-evasion games[J]. Automatic, 2024, 163: 111587.
[28]	WANG Q, WU K Q, YE J F, et al. Apollonius partitions based pursuit-evasion game strategies by Q-learning approach[C]//2022 41st Chinese Control Conference(CCC). Hefei, China: IEEE, 2022: 4843-4848.
[29]	AL-TALABI A A. Fuzzy reinforcement learning algorithm for the pursuit-evasion differential games with superior evader[C]//2017 International Automatic Control Conference(CACS). Taiwan, China: IEEE, 2017: 1-6.
[30]	KHATIB O. Real-time obstacle avoidance for manipulators and mobile robots[J]. The International Journal of Robotics Research, 1986, 5(1): 90-98. doi: 10.1177/027836498600500106
[31]	KUMAR P B, RAWAT H, PARHI D R. Path planning of humanoids based on artificial potential field method in unknown environments[J]. Expert Systems, 2019, 36(2): e12360. doi: 10.1111/exsy.12360
[32]	CHEN M, YANG C, ZHANG X, et al. Pursuit-evasion game of multiple pursuers and evaders with intelligent cooperation and obstacle avoidance in a complex environment[C]//Chinese Conference on Swarm Intelligence and Cooperative Control. Singapore: Springer Nature Singapore, 2023: 222-234.
[33]	DONG J, ZHANG X, JIA X. Strategies of pursuit-evasion game based on improved potential field and differential game theory for mobile robots[C]//2012 Second International Conference on Instrumentation, Measurement, Computer, Communication and Control. Harbin, China: IEEE, 2012: 1452-1456.
[34]	ZHAO F, HUA Y, ZHENG H, et al. Cooperative target pursuit by multiple fixed-wing UAVs based on deep reinforcement learning and artificial potential field[C]//2023 42nd Chinese Control Conference(CCC). Tianjin, China: IEEE, 2023: 5693-5698.
[35]	ZHOU Z, LIU J, YU J. A survey of underwater multi-robot systems[J]. IEEE/CAA Journal of Automatica Sinica, 2021, 9(1): 1-18.
[36]	SUN Z, SUN H, LI P, et al. Pursuit-evasion problem of unmanned surface vehicles in a complex marine environment[J]. Applied Sciences, 2022, 12(18): 9120. doi: 10.3390/app12189120
[37]	YUH J, WEST M. Underwater robotics[J]. Advanced Robotics, 2001, 15(5): 609-639. doi: 10.1163/156855301317033595
[38]	BOGUE R. Underwater robots: A review of technologies and applications[J]. Industrial Robot: An International Journal, 2015, 42(3): 186-191. doi: 10.1108/IR-01-2015-0010
[39]	BOVIO E, CECCHI D, BARALLI F. Autonomous underwater vehicles for scientific and naval operations[J]. Annual Reviews in Control, 2006, 30(2): 117-130. doi: 10.1016/j.arcontrol.2006.08.003
[40]	SUN X, SUN B, SU Z. Cooperative pursuit-evasion game for multi-AUVs in the ocean current and obstacle environment[C]//International Conference on Intelligent Robotics and Applications. Singapore: Springer Nature Singapore, 2023: 201-213.
[41]	CHU Z, WANG F, LEI T, et al. Path planning based on deep reinforcement learning for autonomous underwater vehicles under ocean current disturbance[J]. IEEE Transactions on Intelligent Vehicles, 2022, 8(1): 108-120.
[42]	YANG Y, XIAO Y, LI T. A survey of autonomous underwater vehicle formation: Performance, formation control, and communication capability[J]. IEEE Communications Surveys & Tutorials, 2021, 23(2): 815-841.
[43]	DANIELIS P, PARZYJEGLA H, ALI M A M, et al. Simulation model for energy consumption and acoustic underwater communication of autonomous underwater vehicles[J]. WMU Journal of Maritime Affairs, 2022, 21(1): 89-107. doi: 10.1007/s13437-021-00253-z
[44]	KELASIDI E, PETTERSEN K Y, GRAVDAHL J T. Energy efficiency of underwater robots[J]. IFAC-PapersOnLine, 2015, 48(16): 152-159. doi: 10.1016/j.ifacol.2015.10.273
[45]	ZEREIK E, BIBULI M, MIŠKOVIĆ N, et al. Challenges and future trends in marine robotics[J]. Annual Reviews in Control, 2018, 46: 350-368. doi: 10.1016/j.arcontrol.2018.10.002
[46]	LEI Z F, LEI X F, WANG N, et al. Present status and challenges of underwater acoustic target recognition technology: A review[J]. Frontiers in Physics, 2022, 10: 1044890. doi: 10.3389/fphy.2022.1044890