A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Volume 6 Issue 3
May  2019

IEEE/CAA Journal of Automatica Sinica

• JCR Impact Factor: 6.171, Top 11% (SCI Q1)
CiteScore: 11.2, Top 5% (Q1)
Google Scholar h5-index: 51， TOP 8
Turn off MathJax
Article Contents
Ding Wang and Xiangnan Zhong, "Advanced Policy Learning Near-Optimal Regulation," IEEE/CAA J. Autom. Sinica, vol. 6, no. 3, pp. 743-749, May 2019. doi: 10.1109/JAS.2019.1911489
 Citation: Ding Wang and Xiangnan Zhong, "Advanced Policy Learning Near-Optimal Regulation," IEEE/CAA J. Autom. Sinica, vol. 6, no. 3, pp. 743-749, May 2019.

# Advanced Policy Learning Near-Optimal Regulation

##### doi: 10.1109/JAS.2019.1911489
Funds:

the National Natural Science Foundation of China 61773373

the National Natural Science Foundation of China U1501251

the National Natural Science Foundation of China 61533017

• Designing advanced design techniques for feedback stabilization and optimization of complex systems is important to the modern control field. In this paper, a near-optimal regulation method for general nonaffine dynamics is developed with the help of policy learning. For addressing the nonaffine nonlinearity, a pre-compensator is constructed, so that the augmented system can be formulated as affine-like form. Different cost functions are defined for original and transformed controlled plants and then their relationship is analyzed in detail. Additionally, an adaptive critic algorithm involving stability guarantee is employed to solve the augmented optimal control problem. At last, several case studies are conducted for verifying the stability, robustness, and optimality of a torsional pendulum plant with suitable cost.

•  [1] P. J. Werbos, "Approximate dynamic programming for real-time control and neural modeling, " in Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, D. A. White and D. A. Sofge, Eds. New York: Van Nostrand Reinhold, 1992. [2] D. Liu, Q. Wei, D. Wang, X. Yang, and H. Li, Adaptive Dynamic Programming with Applications in Optimal Control. Cham, Switzerland: Springer, 2017. [3] D. Liu, Y. Xu, Q. Wei, and X. Liu, "Residential energy scheduling for variable weather solar energy based on adaptive dynamic programming, " IEEE/CAA J. Autom. Sinica, vol. 5, no. 1, pp. 36-46, 2018. [4] Z. Fu, W. Xie, S. Rakheja, and J. Na, "Observer-based adaptive optimal control for unknown singularly perturbed nonlinear systems with input constraints, " IEEE/CAA J. Autom. Sinica, vol. 4, no. 1, pp. 48-57, 2017. [5] X. Zhong, H. He, D. Wang, and Z. Ni, "Model-free adaptive control for unknown nonlinear zero-sum differential game, " IEEE Trans. Cybernetics, vol. 48, no. 5, pp. 1633-1646, 2018. [6] W. Gao and Z. P. Jiang, "Learning-based adaptive optimal tracking control of strict-feedback nonlinear systems, " IEEE Trans. Neural Networks and Learning Systems, vol. 29, no. 6, pp. 2614-2624, 2018. [7] D. Wang, H. He, C. Mu, and D. Liu, "Intelligent critic control with disturbance attenuation for affine dynamics including an application to a microgrid system, " IEEE Trans. Industrial Electronics, vol. 64, no. 6, pp. 4935-4944, 2017. [8] D. Wang, H. He, and D. Liu, "Intelligent optimal control with critic learning for a nonlinear overhead crane system, " IEEE Trans. Industrial Informatics, vol. 14, no. 7, pp. 2932-2940, 2018. [9] J. Li, T. Chai, F. L. Lewis, J. Fan, Z. Ding, and J. Ding, "Off-policy Q-learning: Set-point design for optimizing dual-rate rougher flotation operational processes, " IEEE Trans. Industrial Electronics, vol. 65, no. 5, pp. 4092-4102, 2018. [10] D. Wang, H. He, and D. Liu, "Adaptive critic nonlinear robust control: A survey, " IEEE Trans. Cybernetics, vol. 47, no. 10, pp. 3429-3451, 2017. [11] Q. Y. Fan and G. H. Yang, "Adaptive nearly optimal control for a class of continuous-time nonaffine nonlinear systems with inequality constraints, " ISA Trans., vol. 66, pp. 122-133, 2017. [12] B. Kiumarsi, W. Kang, and F. L. Lewis, "$H_{infty}$ control of nonaffine aerial systems using off-policy reinforcement learning, " Unmanned Systems, vol. 4, no. 1, pp. 51-60, 2016. [13] H. Zhang, L. Cui, X. Zhang, and Y. Luo, "Data-driven robust approximate optimal tracking control for unknown general nonlinear systems using adaptive dynamic programming method, " IEEE Trans. Neural Networks, vol. 22, no. 12, pp. 2226-2236, 2011. [14] X. Yang, D. Liu, and D. Wang, "Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints, " International J. Control, vol. 87, no. 3, pp. 553-566, 2014. [15] T. Bian, Y. Jiang, and Z. P. Jiang, "Adaptive dynamic programming and optimal control of nonlinear nonaffine systems, " Automatica, vol. 50, no. 10, pp. 2624-2632, 2014. [16] C. Cox and R. Saeks, "Adaptive critic control and functional link neural networks, " in Proc. IEEE International Conference on Systems, Man, and Cybernetics, 1998, pp. 1652-1657. [17] D. Zhao, D. Liu, F. L. Lewis, J. C. Principe, and S. Squartini, "Special issue on deep reinforcement learning and adaptive dynamic programming, " IEEE Trans. Neural Networks and Learning Systems, vol. 29, no. 6, pp. 2038-2041, 2018. [18] T. Liu, B. Tian, Y. Ai, L. Li, D. Cao, and F. Y. Wang, "Parallel reinforcement learning: a framework and case study, " IEEE/CAA J. Autom. Sinica, vol. 5, no. 4, pp. 827-835, 2018.

### Catalog

###### 通讯作者: 陈斌, bchen63@163.com
• 1.

沈阳化工大学材料科学与工程学院 沈阳 110142

Figures(9)  / Tables(1)

/