Energy-optimal Path Planning Algorithm for Unmanned Surface Vessel Based on Reinforcement Learning
-
摘要: 针对无人水面艇路径规划时存在洋流、障碍物等外部干扰的问题, 提出一种改进的基于强化学习的无人水面艇能耗最优路径规划算法。首先, 建立多个随机涡流组成的二维洋流模型和无人水面艇平面运动学模型; 其次, 依据洋流和无人水面艇相对速度关系, 计算路径点是否可达; 然后, 利用改进的奖励函数、动作集和状态集求解全局最优路径, 并采用B样条法进行平滑; 最后, 在2种典型环境下进行了数值仿真。仿真结果表明，该算法可规划出一条能耗最优且平滑的路径, 验证了算法的有效性和最优性。Abstract: To address path planning for unmanned surface vessels(USVs) in ocean environments affected by physical disturbances, such as ocean currents and obstacles, an energy-optimal algorithm based on improved reinforcement learning is proposed. First, a two-dimensional ocean-current model comprising multiple random vortices and a plane kinematic model of the USV is established. Then, the study determined whether a waypoint is reachable using the relative velocity relationship between the USV and the ocean current. An improved reward function, an action set, and a state set are used to obtain a global optimal path, and the B-spline method is applied to smooth the plan. Finally, numerical simulations are performed in two typical environments. The simulation results show that a path with optimal energy consumption and smoothness can be planned based on the proposed algorithm.
-
图 5 洋流影响下无人水面艇航向角取值范围
Figure 5. Range of the heading angle of USV under the influence of ocean current
图 8 基于洋流约束的无人水面艇仿真路径
Figure 8. Simulation paths of USV based on ocean current constraints
图 9 基于洋流和障碍物约束的无人水面艇仿真路径
Figure 9. Simulation paths of USV based on obstacles and ocean current constraints
表 1 仿真结果对比
Table 1. Comparison of simulation results
环境 算法 时间/s 大转向角个数 路径长度/m 情况1 直接路径 224 0 378 粒子群算法 207 3 453 传统算法 228 3 583 文中算法 183 0 454 情况2 直接路径 224 0 378 粒子群算法 217 1 428 传统算法 242 7 588 文中算法 191 0 451 
下载: 导出CSV
-
[1] Fan X J, Guo J Y, Liu H, et al. Improved artificial potential field method applied for AUV path planning[J]. Mathematical Problems in Engineering, 2020(1): 1-21. [2] Liu C, Mao Q, Chu X, et al. An improved A-star algorithm considering water current, traffic separation and berthing for vessel path planning[J]. Applied Scineces Basel, 2019, 9(6): 1057. doi: 10.3390/app9061057 [3] Wang L, Pang S. Chemical plume tracing using an AUV based on POMDP source mapping and A-star path planning[C]//2019 MTS/IEEE Oceans Conference. Parc Chanotl, Marseille, France: IEEE, 2019. [4] 高博, 徐德民, 张福斌, 等. 海流建模及其在路径规划中的应用[J]. 系统仿真学报, 2010, 22(4): 957-961. doi: 10.16182/j.cnki.joss.2010.04.016Gao Bo, Xu Deming, Zhang Fubin, et al. Method of designing optimal smooth way for vehicle[J]. Journal of System Simulation, 2010, 22(4): 957-961. doi: 10.16182/j.cnki.joss.2010.04.016 [5] 向金林, 王鸿东, 欧阳子路, 等. 基于改进双向RRT的无人艇局部路径规划算法研究[J]. 中国造船, 2020, 61(1): 157-166.Xiang Jinlin, Wang Hongdong, Ouyang Zilu, et al. Algorithm of local path planning for unmanned surface vehicles based on improved Bi-RRT[J]. Shipbuilding of China, 2020, 61(1): 157-166. [6] Alvarez A, Caiti A. A genetic algorithm for autonomous underwater vehicle route planning in ocean environments with complex space-time variability[J]. IFAC Proceedings Volumes, 2001, 34(7): 237-242. doi: 10.1016/S1474-6670(17)35089-9 [7] Alvarez A, Caiti A, Onken R. Evolutionary path planning for autonomous underwater vehicles in a variable ocean[J]. IEEE Journal of Oceanic Engineering, 2004, 29(2): 418-429. doi: 10.1109/JOE.2004.827837 [8] Zeng Z, Sammut K, Lian L, et al. A comparison of optimization techniques for AUV path planning in environments with ocean currents[J]. Robotics and Autonomous Systems, 2016(82): 61-72. [9] Zeng Z, Lammas A, Sammut K, et al. Shell space decomposition based path planning for AUVs operating in a variable environment[J]. Ocean Engineering, 2014, 91: 181-195. doi: 10.1016/j.oceaneng.2014.09.001 [10] 马焱, 肖玉杰, 陈轶, 等. 基于改进烟花-蚁群算法的海流环境下水下无人水面艇的避障路径规划[J]. 导航与控制, 2019, 18(1): 51-59. doi: 10.3969/j.issn.1674-5558.2019.01.007Ma Yan, Xiao Yujie, Chen Yi, et al. Obstacle avoidance path planning of unmanned underwater vehicle in ocean current environment based on improved fireworks-ant colony algorithm[J]. Navigation and Control, 2019, 18(1): 51-59. doi: 10.3969/j.issn.1674-5558.2019.01.007 [11] Chen C, Chen X Q, Ma F, et al. A knowledge-free path planning approach for smart ships based on reinforcement learning[J]. Ocean Engineering, 2019, 189: 106299. doi: 10.1016/j.oceaneng.2019.106299 [12] Guo S, Zhang X, Zheng Y, et al. An autonomous path planning model for unmanned ships based on deep reinforcement learning[J]. Sensors, 2020, 20(2): 426. doi: 10.3390/s20020426 [13] Woo J, Kim N. Collision avoidance for an unmanned surface vehicle using deep reinforcement learning[J]. Ocean Engineering, 2020, 199(1): 1-16. [14] Garau B, Alvarez A, Oliver G. AUV navigation through turbulent ocean environments supported by onboard H-ADCP[C]//2006 IEEE International conference on robotics and automation. Orlando, FL, USA: IEEE, 2006. [15] Li X, Sun Z, Cao D, et al. Real-time trajectory planning for autonomous urban driving: framework, algorithms, and verifications[J]. IEEE-ASME Transactions on Mechatronics, 2016, 21(2): 740-753. doi: 10.1109/TMECH.2015.2493980 -
点击查看大图
图(9) / 表(1)
计量
- 文章访问数: 388
- HTML全文浏览量: 282
- PDF下载量: 147
- 被引次数: 0
