• 中国科技核心期刊
  • JST收录期刊
Volume 29 Issue 5
 
Turn off MathJax
Article Contents
Article Navigation >  Journal of Unmanned Undersea Systems  > 2021  >  29(5): 565-574
YAN Jing, LI Wen-biao, YANG Xian, LI Xing-long, LUO Xiao-yuan. Tracking Control for AUV by Combining Q Learning and a PID Controller[J]. Journal of Unmanned Undersea Systems, 2021, 29(5): 565-574. doi: 10.11993/j.issn.2096-3920.2021.05.008
Citation: YAN Jing, LI Wen-biao, YANG Xian, LI Xing-long, LUO Xiao-yuan. Tracking Control for AUV by Combining Q Learning and a PID Controller[J]. Journal of Unmanned Undersea Systems, 2021, 29(5): 565-574. doi: 10.11993/j.issn.2096-3920.2021.05.008
shu

Tracking Control for AUV by Combining Q Learning and a PID Controller

doi: 10.11993/j.issn.2096-3920.2021.05.008

  • School of Electrical Engineering, Yanshan University, Qinhuangdao 066004, China

  • Received Date: 2020-10-27
  • Rev Recd Date: 2020-12-16
  • Publish Date: 2021-10-31
    Abstract
  • In this study, a tracking control algorithm for autonomous underwater vehicles(AUVs) is developed by combining Q learning and a proportional-integral-derivative(PID) controller. First, a PID-based tracking control algorithm based on the tracking error of the AUV is presented. To improve the static and dynamic tracking performances, the adaptive adjustment of the parameters of the PID controller is described by utilizing a Q learning problem; then, an action update strategy is employed to iteratively optimize the Q values in different states until the Q value corresponding to each state action is stabilized. Compared with the traditional PID controllers, the proposed control algorithm can preserve the simple and practical characteristics of PID controllers and adaptively adjust the parameters according to the changes in the environment information. Finally, the simulation and experimental results confirm the effectiveness of the proposed algorithm

     

    • FullText(HTML)
  • loading
    • References(20)
  • [1]
    赵振轶, 李亚安, 陈晓, 等. 基于双观测站的水下机动目标被动跟踪[J]. 水下无人系统学报, 2018, 26(1): 40-45.

    Zhao Zhen-yi, Li Ya-an, Chen Xiao, et al. Passive Tracking of Underwater Maneuvering Target Based on Double Observation Station[J]. Journal of Unmanned Undersea Systems, 2018, 26(1): 40-45.
    [2]
    杜度. 基于RBF神经网络参数自整定的AUV深度控制[J]. 水下无人系统学报, 2019, 27(3): 284-289.

    Du Du. Parameters Self-Tuning for Depth Control of AUV Based on RBF Neural Network[J]. Journal of Unmanned Undersea Systems, 2019, 27(3): 284-289.
    [3]
    温志文, 蔡卫军, 杨春武. 基于改进蚁群算法的UUV三维路径规划方法[J]. 鱼雷技术, 2016, 24(2): 120-125.

    Wen Zhi-wen, Cai Wei-jun, Yang Chun-wu. Three- dimensional Path Planning Method Based on Improved Ant Colony Algorithm for UUV[J]. Torpedo Technology, 2016, 24(2): 120-125.
    [4]
    Shen C, Shi Y, Buckham B. Trajectory Tracking Control of an Autonomous Underwater Vehicle Using Lyapunov-based Model Predictive Control[J]. IEEE Transactions on Industrial Electronics, 2017, 65(7): 5796-5805.
    [5]
    Li Y, Wei C, Wu Q, et al. Study of 3 Dimension Trajectory Tracking of Underactuated Autonomous Underwater Vehicle[J]. Ocean Engineering, 2015, 105(1): 270-274.
    [6]
    Carlucho I, Paula M, Villar S, et al. Incremental Q-learning Strategy for Adaptive PID Control of Mobile Robots[J]. Expert Systems with Application, 2017, 80(1): 183-199.
    [7]
    赵健, 白春江, 章文俊. 水下潜器姿态角的分数阶PID 控制研究[J]. 船舶科学技术, 2016, 38(11): 129-132.

    Zhao Jian, Bai Chun-jiang, Zhang Wen-jun. Research on Fractional-order PID Control for Underwater Vehicle Attitude Angle[J]. Ship Science and Technology, 2016, 38(11): 129-132.
    [8]
    Ban H, Yang X, Luo X, et al. Fuzzy-based Tracking Controller Design for Autonomous Underwater Vehicle[C]//Chinese Control Conference, Dalian, China: IEEE, 2017: 4813-4818.
    [9]
    Hernandez-Alvarado R, Garcia-Valdovinos L, Salgado-Jimenez T, et al. Self-tuned PID Control Based on Backpropagation Neural Networks for underwater Vehicles [C]//International Conference on Ocean. Monterey, USA: IEEE, 2016, 1-5.
    [10]
    Wu H, Song S, You K. Depth Control of Model-Free AUVs via Reinforcement Learning[J]. IEEE Transactions on System, Man, and Cybernetics, 2019, 49(12): 2499-2510.
    [11]
    Kim M. Greedy Learning of Sparse Eigenfaces for Face Recognition and Tracking[J]. International Journal of Fuzzy Logic and Intelligent Systems, 2014, 14(3): 162-170.
    [12]
    Luo B, Liu D, Huang T, et al. Model-Free Optimal Tracking Control via Critic-Only Q-Learning[J]. IEEE Transactions on Neural Networks & Learning Systems, 2016, 27(10): 2134-2144.
    [13]
    Yuan C, He H, Wang C. Cooperative Deterministic Learning-based Formation Control for a Group of Nonlinear Uncertain Mechanical Systems[J]. IEEE Transactions on Industrial Informatics, 2019, 15(1): 319-333.
    [14]
    Yan J, Luo X, Li X, et al. Joint Localization and Tracking for Autonomous Underwater Vehicle: A Reinforcement Learning Based Approach[J]. IET Control Theory & Applications, 2019, 13(17): 2856-2865.
    [15]
    Yan J, Gao J, Yang X, et al. Tracking Control of a Remotely Operated Underwater Vehicle with Time Delay and Actuator Saturation[J]. Ocean Engineering, 2019, 184(1): 299-310.
    [16]
    Yan J, Gao J, Yang X, et al. Position Tracking Control of Remotely Operated Underwater Vehicles with Communication Delay[J]. IEEE Transactions on Control Systems Technology, 2019, 28(6): 2506-2514.
    [17]
    Galetzka A, Bontinck Z, Romer U, et al. A Multilevel Monte Carlo Method for High-dimensional Uncertainty Quantification of Low-frequency Electromagnetic Devices[J]. IEEE Transactions on Magnetics, 2019, 55(8): 1-12.
    [18]
    Wan Y, Roy S, Lesieutre B, et al. Uncertainty Evaluation Through Mapping Identification in Intensive Dynamic Simulations[J]. IEEE Transactions on System, Man, and Cybernetics, 2010, 40(5): 1094-1104.
    [19]
    Lewis F L, Vrabie D. Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control[J]. IEEE Circuits and Systems Magazine, 2009, 9(3): 32-50.
    [20]
    Huang S M, Giving S N. A Q-Learning Approach to Flocking With UAVs in a Stochastic Environment[J]. IEEE Transactions on Cybernetics, 2017, 47(1): 186-197.
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索
    Get Citation
    PDF
    XML

    Article Metrics

    Article Views(1429) PDF Downloads(58) Cited by()
    Proportional views
    Related
    Service
    Subscribe
    Copyright：陕ICP备09015163号Address:No. 96, Jinye Road, Xi'an, Shaanxi
    Code：710077Tel: 029-88327279QQ: 767358370
    Email: bianjibu705@sohu.com 767358370@qq.com
    Email Alert RSS
    Taobao
    Micro

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return