Research on Dynamic Obstacle Avoidance Planning for Unmanned Surface Vessels Based on the Introduction of Attention Mechanism
-
摘要: 针对无人水面艇在复杂动态海况下的局部避障与路径规划问题, 提出了一种基于深度强化学习的端到端控制方法。该方法采用近端策略优化(PPO)算法, 并设计了一种融合一维深度可分离卷积与多头注意力机制的特征提取网络。针对稀疏激光雷达数据的特性, 首先利用挤压-激励模块增强通道特征的表达能力; 其次, 构建基于Transformer架构的多头注意力模块, 以艇体自身的运动状态作为查询向量, 动态加权环境观测特征, 实现本体感知与环境感知的有效融合; 最后, 在PyBullet物理仿真环境中构建了包含静态礁石与动态干扰障碍的复杂场景。实验结果表明, 文中方法能够有效处理动态障碍物, 实现平滑自主避障控制, 与普通PPO算法相比, 避障规划成功率提升了4%。Abstract: To address the problem of local obstacle avoidance and path planning for Unmanned Surface Vehicles in complex dynamic marine environments, this paper proposes an end-to-end control method based on deep reinforcement learning. The method employs the Proximal Policy Optimization algorithm and designs a feature extraction network that integrates one-dimensional depthwise separable convolution with a multi-head attention mechanism. Given the characteristics of sparse LiDAR data, the Squeeze-and-Excitation (SE) module is first utilized to enhance the representational capacity of channel features. Subsequently, a multi-head attention module based on the Transformer architecture is constructed, which uses the vehicle's own motion states as query vectors to dynamically weight environmental observation features, thereby achieving effective fusion of proprioception and environmental perception. Finally, complex scenarios comprising static reefs and dynamic interfering obstacles are constructed within the PyBullet physics simulation environment. Experimental results demonstrate that the proposed method effectively handles dynamic obstacles and achieves smooth autonomous obstacle avoidance control . Compared with standard PPO, the proposed method improves the obstacle-avoidance planning success rate by 4%.
-
图 7 不同算法回报值对比图
Figure 7. Comparative Chart of Reward Values Across Different Algorithms
-
[1] 林龙信, 张比升. 水面无人作战系统技术发展与作战应用[J]. 水下无人系统学报, 2018, 26(2): 107-114. doi: 10.11993/j.issn.2096-3920.2018.02.002Lin L X, Zhang B S. Technical development and operational application of unmanned surface combat system[J]. Journal of Unmanned Undersea Systems, 2018, 26(2): 107-114. doi: 10.11993/j.issn.2096-3920.2018.02.002 [2] 李佩娟, 颜庭武, 杨书涛, 等. 基于强化学习的无人水面艇能耗最优路径规划算法[J]. 水下无人系统学报, 2023, 31(2): 237-243. doi: 10.11993/j.issn.2096-3920.202203002Li P J, Yan T W, Yang S T, et al. Energy-optimal path planning algorithm for unmanned surface vessel based on reinforcement learning[J]. Journal of Unmanned Undersea Systems, 2023, 31(2): 237-243. doi: 10.11993/j.issn.2096-3920.202203002 [3] 于长东, 刘新阳, 陈聪, 等. 基于多智能体深度强化学习的无人艇集群博弈对抗研究[J]. 水下无人系统学报, 2024, 32(1): 79-86. doi: 10.11993/j.issn.2096-3920.2023-0159Yu C D, Liu X Y, Chen C, et al. Research on game confrontation of unmanned surface vehicles swarm based on multi-agent deep reinforcement learning[J]. Journal of Unmanned Undersea Systems, 2024, 32(1): 79-86. doi: 10.11993/j.issn.2096-3920.2023-0159 [4] Wang N, Xu H. Dynamics-constrained global-local hybrid path planning of an autonomous surface vehicle[J]. IEEE Transactions on Vehicular Technology, 2020, 69(7): 6928-6942. doi: 10.1109/TVT.2020.2991220 [5] 林韩熙, 向丹, 欧阳剑, 等. 移动机器人路径规划算法的研究综述[J]. 计算机工程与应用, 2021, 57(18): 38-48. doi: 10.3778/j.issn.1002-8331.2103-0519Lin H X, Xiang D, Ouyang J, et al. Review of path planning algorithms for mobile robots[J]. Computer Engineering and Applications, 2021, 57(18): 38-48. doi: 10.3778/j.issn.1002-8331.2103-0519 [6] 王梓强, 胡晓光, 李晓筱, 等. 移动机器人全局路径规划算法综述[J]. 计算机科学, 2021, 48(10): 19-29. doi: 10.11896/jsjkx.200700114Wang Z Q, Hu X G, Li X X, et al. Overview of global path planning algorithms for mobile robots[J]. Computer Science, 2021, 48(10): 19-29. doi: 10.11896/jsjkx.200700114 [7] 李二超, 王玉华. 改进人工势场法的移动机器人避障轨迹研究[J]. 计算机工程与应用, 2022, 58(6): 296-304. doi: 10.3778/j.issn.1002-8331.2108-0122Li E C, Wang Y H. Research on obstacle avoidance trajectory of mobile robot based on improved artificial potential field[J]. Computer Engineering and Applications, 2022, 58(6): 296-304. doi: 10.3778/j.issn.1002-8331.2108-0122 [8] 马杰, 苏钰栋, 熊勇, 等. 基于速度障碍和人工势场的受限水域船舶避碰决策方法[J]. 中国安全科学学报, 2020, 30(11): 60-66. doi: 10.16265/j.cnki.issn1003-3033.2020.11.009Ma J, Su Y D, Xiong Y, et al. Decision-making method for collision avoidance of ships in confined waters based on velocity obstacle and artificial potential field[J]. China Safety Science Journal, 2020, 30(11): 60-66. doi: 10.16265/j.cnki.issn1003-3033.2020.11.009 [9] 侯玉立, 王宁, 邱赤东, 等. 无人艇集群路径规划研究综述: 深度强化学习[J]. 水下无人系统学报, 2025, 33(2): 193-203. doi: 10.11993/j.issn.2096-3920.2025-0034Hou Y L, Wang N, Qiu C D, et al. A review of research on path planning of unmanned surface vessel swarm: deep reinforcement learning[J]. Journal of Unmanned Undersea Systems, 2025, 33(2): 193-203. doi: 10.11993/j.issn.2096-3920.2025-0034 [10] Du Z, Yang S, Wang W. An adaptive difference policy gradient method for cooperative multi-USV pursuit in multi-agent reinforcement learning[J]. Journal of Marine Science and Engineering, 2026, 14(3): 252. doi: 10.3390/jmse14030252 [11] Li X, Sun Z, Cao D, et al. Real-time trajectory planning for autonomous urban driving: framework, algorithms, and verifications[J]. IEEE-ASME Transactions on Mechatronics, 2016, 21(2): 740-753. doi: 10.1109/TMECH.2015.2493980 -
下载:
点击查看大图
计量
- 文章访问数: 2
- HTML全文浏览量: 1
- PDF下载量: 0
- 被引次数: 0
图(9)
