A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Volume 1 Issue 4
Oct.  2014

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 6.171, Top 11% (SCI Q1)
    CiteScore: 11.2, Top 5% (Q1)
    Google Scholar h5-index: 51, TOP 8
Turn off MathJax
Article Contents
Rubo Zhang, Pingpeng Tang, Yumin Su, Xueyao Li, Ge Yang and Changting Shi, "An Adaptive Obstacle Avoidance Algorithm for Unmanned Surface Vehicle in Complicated Marine Environments," IEEE/CAA J. of Autom. Sinica, vol. 1, no. 4, pp. 385-396, 2014.
Citation: Rubo Zhang, Pingpeng Tang, Yumin Su, Xueyao Li, Ge Yang and Changting Shi, "An Adaptive Obstacle Avoidance Algorithm for Unmanned Surface Vehicle in Complicated Marine Environments," IEEE/CAA J. of Autom. Sinica, vol. 1, no. 4, pp. 385-396, 2014.

An Adaptive Obstacle Avoidance Algorithm for Unmanned Surface Vehicle in Complicated Marine Environments


This work was supported by National Natural Science Foundation of China (60975071, 61100005, 60975019).

  • Unmanned surface vehicles (USVs) are important autonomous marine robots that have been studied and gradually applied into practice. However, the autonomous navigation of USVs, especially the issue of obstacle avoidance in complicated marine environment, is still a fundamental problem. After studying the characteristics of the complicated marine environment, we propose a novel adaptive obstacle avoidance algorithm for USVs, based on the Sarsa on-policy reinforcement learning algorithm. The proposed algorithm is composed of local avoidance module and adaptive learning module, which are organized by the "divide and conquer" strategy-based architecture. The course angle compensation strategy is proposed to offset the disturbances from sea wind and currents. In the design of payoff value function of the learning strategy, the course deviation angle and its tendency are introduced into action rewards and penalty policies. The validity of the proposed algorithm is verified by comparative experiments of simulations and sea trials in three sea-state marine environments. The results show that the algorithm can enhance the autonomous navigation capacity of USVs in complicated marine environments.


  • loading
  • [1]
    Steimle E T, Hall M L. Unmanned surface vehicles as environmentalmonitoring and assessment tools. In: Proceedings of Oceans 2006.Massachusetts, Boston, USA: IEEE, 2006. 1-5
    Bibuli M, Bruzzone G, Caccia M, Indiveri G, Zizzari A A. Linefollowing guidance control: application to the Charlie unmanned surfacevehicle. In: Proceedings of IEEE/RSJ International Conference onIntelligent Robots and Systems, IROS 2008. Nice, France: IEEE, 2008.3641-3646
    Aditya S G, Christian S, Shu D, Daniel J S, Craig W. Guidance andcontrol of an unmanned surface vehicle exhibiting sternward motion. In:Proceedings of the Oceans. Hampton Roads, VA: IEEE, 2012. 1-9
    Raboin E, Svec P, Nau D, Gupta S K. Model-predictive target defense byteam of unmanned surface vehicles operating in uncertain environments.In: Proceedings of the 2013 IEEE International Conference on Roboticsand Automation (ICRA). Karlsruhe, Germany: IEEE, 2013. 3517-3522
    Wang J H, Chen C F, Huang P P, Gu W, Chu J X. Modeling, simulatingand experiment of an autonomous surface vehicle. Energy Procedia,2011, 11(2): 314-318
    Motwani A. Survey of unmanned surface vehicles, Technical ReportMIDAS.SMSE.2012.TR.001, Plymouth University, UK, 2012.
    Tan Min, Wang Shuo. Research progress on robotics. Acta AutomaticaSinica, 2013, 39(7): 963-972 (in Chinese)
    Naeem W, Irwin G W, Yang A. COLREGS-based collision avoidancestrategies for unmanned surface vehicles. Mechatronics, 2012, 22(6):669-678
    Szymak P, Praczyk T. Using neural-evolutionary-fuzzy algorithm foranti-collision system of unmanned surface vehicle. In: Proceedings of the17th International Conference on Methods and Models in Automationand Robotics (MMAR). Miedzyzdro, Poland: IEEE, 2012. 286-290
    Zhuang Jia-Yuan, Su Yu-Min, Liao Yu-Lei, Sun Han-Bing. Unmannedsurface vehicle local path planning based on marine radar. Journal ofShanghai Jiao Tong University, 2012, 46(9): 1371-1375 (in Chinese)
    Yang A L, Niu Q, ZhaoWQ, Li K, Irwin GW. An efficient algorithm forgrid-based robotic path planning based on priority sorting of directionvectors. In: Proceedings of the 2010 International Conference on LifeSystem Modeling and Intelligent Computing. Berlin, Germany: Springer,2010. 456-466
    Huntsberger T, Aghazarian H, Howard A, Trotz D C. Stereo vision-basednavigation for autonomous surface vessels. Journal of Field Robotics,2011, 28(1): 3-18
    Larson J, Bruch M, Halterman R, Rogers J, Webster R. Advancesin autonomous obstacle avoidance for unmanned surface vehicles. In:Proceedings of the AUVSI Unmanned Systems North America 2007.Washington, DC, USA: AUVSI, 2007. 1-15
    Tang P P, Zhang R B, Liu D L, Zou Q J, Shi C T. Research on nearfieldobstacle avoidance for unmanned surface vehicle based on headingwindow. In: Proceedings of the Control and Decision Conference(CCDC). Taiyuan, China: IEEE, 2012. 1262-1267
    Krishnamurthy P, Khorrami F, Ng T L. Obstacle avoidance for unmannedsea surface vehicles: a hierarchical approach. In: Proceedings of the17th World Congress, the International Federation of Automatic Control.Seoul, Korea: IFAC, 2008. 6798-6803
    Bandyophadyay T, Sarcione L, Hover F S. A simple reactive obstacleavoidance algorithm and its application in Singapore harbor. Field andService Robotics, 2010, 62: 455-465
    Kuwata Y, Wolf M T, Zarzhitsky D, Huntsberger T L. Safe maritimeautonomous navigation with COLREGS, using velocity obstacles. IEEEJournal of Oceanic Engineering, 2014, 39(1): 110-119
    Campbell S, NaeemW, Irwin GW. A review on improving the autonomyof unmanned surface vehicles through intelligent collision avoidancemanoeuvres. Annual Reviews in Control, 2012, 36(2): 267-283
    Thakur A, Svec P, Gupta S K. GPU based generation of state transitionmodels using simulations for unmanned surface vehicle trajectoryplanning. Robotics and Autonomous Systems, 2012, 60(12): 1457-1471
    Stelzer R, Prollb T. Autonomous sailboat navigation for short courseracing. Robotics and Autonomous Systems, 2008, 56(7): 604-614
    Faltinsen O M. Hydrodynamics of High-Speed Marine Vehicles. Cambridge,UK: Cambridge University Press, 2005.
    Xin Bin, Chen Jie, Peng Zhi-Hong. Intelligent optimized control:overview and prospect. Acta Automatica Sinica, 2013, 39(11):1831-1848 (in Chinese)
    Yu Z Y, Bao X P, Nonami K. Course keeping control of an autonomousboat using low cost sensors. Journal of System Design and Dynamics,2008, 2(1): 389-400
    Wu G X, Sun H B, Zou J, Wan L. The basic motion control strategyfor the water-jet-propelled USV. In: Proceedings of the 2009 IEEEInternational Conference on Mechatronics and Automation. Changchun,China: IEEE, 2009. 611-616
    Wu G X, Zou J, Wan L, Qin Z B. Design of the intelligence motioncontrol system for the high-speed USV. In: Proceedings of the 2ndInternational Conference on Intelligent Computation Technology andAutomation. Changsha, China: IEEE, 2009. 50-53
    Sutton R S, Barto A G. Reinforcement Learning: An Introduction.Cambridge, MA: MIT Press, 1998.
    Singh S P, Jaakkola T, Littman M L, Szepesvari C. Convergence resultsfor single-step on-policy reinforcement-learning algorithms. MachineLearning, 2000, 38(3): 287-308
    Watkins C. Learning from Delayed Rewards [Ph. D. dissertation], KingsCollege, University of Cambridge, UK, 1989.
    Rummery G A, Niranjan M. On-line Q-learning Using ConnectionistSystems. Cambridge, UK: Department of Engineering, University ofCambridge, 1994.
    Feinberg E A, Shwartz A. Handbook of Markov Decision Processes:Methods and Applications. Boston, MA: Kluwer Academic Publishers,2002.
    Xu Xin, Shen Dong, Gao Yan-Qing, Wang Kai. Learning controlof dynamical systems based on Markov decision processes: researchfrontiers and outlooks. Acta Automatica Sinica, 2012, 38(5): 673-687(in Chinese)


    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Article Metrics

    Article views (1075) PDF downloads(17) Cited by()


    DownLoad:  Full-Size Img  PowerPoint