IEEE/CAA Journal of Automatica Sinica
Citation:  Ruizhuo Song and Liao Zhu, "Optimal FixedPoint Tracking Control for DiscreteTime Nonlinear Systems via ADP," IEEE/CAA J. Autom. Sinica, vol. 6, no. 3, pp. 657666, May 2019. doi: 10.1109/JAS.2019.1911453 
[1] 
K. V. Berkel, B. D. Jager, T. Hofman, and M. Steinbuch, "Implementation of dynamic programming for optimal control problems with continuous states, " IEEE Trans. Control Syst. Technol., vol. 23, no. 3, pp. 11721179, May 2015.

[2] 
K. Deng, Y. Sun, S. Li, Y. Lu, J. Brouwer, P. G. Mehta, M. Zhou, and A. Chakraborty, "Model predictive control of central chiller clant cith thermal energy storage via dynamic programming and mixedinteger linear programming, " IEEE Trans. Autom. Sci. Eng., vol. 12, no. 2, pp. 565579, Apr. 2015.

[3] 
B. E. Richard, Dynamic Programming. Princeton, NJ, USA: Princeton Univ. Press, 1957.

[4] 
W. T. Miller, R. S. Sutton, and P. J. Werbos, Eds., Neural Networks for Control, Cambridge, MA, USA: MIT Press, 1990.

[5] 
P. J. Werbos, "Advanced forecasting methods for global crisis warning and models of intelligence, " General Syst. Yearbook, vol. 22, pp. 2538, 1977.

[6] 
D. V. Prokhorov and Wunsch D C, "Adaptive critic designs, " IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 9971007, Sep. 1997.

[7] 
R. Padhi, N. Unnikrishnan, X. Wang, and S. N. Balakrishnan, "A single network adaptive critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems, " Neural Netw., vol. 19, no. 10, pp. 16481660, Dec. 2006.

[8] 
Q. Wei, F. L. Lewis, D. Liu, and R. Song, "Discretetime local value iteration adaptive dynamic programming: convergence analysis, " IEEE Trans., Syst., Man, Cybern., Syst., vol. 48, no. 6, pp. 875891, Jun. 2016.

[9] 
D. P. Bertsekas, "Value and policy iterations in optimal control and adaptive dynamic programming, " IEEE Trans. Neural Netw. Learn. Syst., vol. 28, no. 3, pp. 500509, Mar. 2017.

[10] 
B. Fan, Q. Yang, X. Tang, and Y. Sun, "Robust ADP design for continuoustime nonlinear systems with output constraints, " IEEE Trans. Neural Netw. Learn. Syst., vol. 29, no. 6, pp. 21272138, Jun. 2018.

[11] 
D. Liu, Y. Xu, Q. Wei, and X. Liu, "Residential energy scheduling for variable weather solar energy based on adaptive dynamic programming, " IEEE/CAA J. Autom. Sinica, vol. 5, no. 1, pp. 3646, Jan. 2018.

[12] 
Q. Wei, D. Liu, Y. Liu, and R. Song, "Optimal constrained selflearning battery sequential management in microgrid via adaptive dynamic programming, " IEEE/CAA J. Autom. Sinica, vol. 4, no. 2, pp. 168176, Apr. 2017.

[13] 
Z. Wang, L. Liu, and H. Zhang, "Neural networkbased modelfree adaptive faulttolerant control for discretetime nonlinear systems with sensor fault, " IEEE Trans., Syst., Man, Cybern., Syst., vol. 47, no. 8, pp. 23512362, Aug. 2017.

[14] 
R. Song, F. L. Lewis, and Q. Wei, "Offpolicy integral reinforcement learning method to solve nonlinear continuoustime multiplayer nonzerosum games, " IEEE Trans. Neural Netw. Learn. Syst., vol. 28, no. 3, pp. 704713, Mar. 2017.

[15] 
D. Liu, H. Li, and D. Wang, "Online synchronous approximate optimal learning algorithm for multiplayer nonzerosum games with unknown dynamics, " IEEE Trans., Syst., Man, Cybern., Syst., vol. 44, no. 8, pp. 10151027, Aug. 2014.

[16] 
H. Zhang, L. Cui, and Y. Luo, "Nearoptimal control for nonzerosum differential games of continuoustime nonlinear systems using singlenetwork ADP, " IEEE Trans. Cybern., vol. 43, no. 1, pp. 206216, Feb. 2013.

[17] 
Q. Wei, D. Liu, G. Shi, and Y. Liu, "Multibattery optimal coordination control for home energy management systems via distributed iterative adaptive dynamic programming, " IEEE Trans. Ind. Electron., vol. 62, no. 7, pp. 42034214, Jul. 2015.

[18] 
A. Isidori and W. Kang, "$H_{infty}$ control via measurement feedback for general nonlinear systems, " IEEE Trans. Autom. Control, vol. 40, no. 3, pp. 466472, Mar. 1995.

[19] 
T. Basar and P. Bernhard, $H_{infty}$ Optimal Control and Related Minimax Design Problems. Boston, MA, USA: Birkhuser, 1995.

[20] 
T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory. Philadelphia, PA, USA: SIAM, 1999.

[21] 
A. AlTamimi, M. AbuKhalaf, and F. L. Lewis, "Adaptive critic designs for discretetime zerosum games with application to $H_{infty}$ control, " IEEE Trans. Syst., Man, Cybern., Part B: Cybern., vol. 37, no. 1, pp. 240247, Feb. 2007.

[22] 
Q. Wei, R. Song, and P. Yan, "Datadriven zerosum neurooptimal control for a class of continuoustime unknown nonlinear systems with disturbance using ADP, " IEEE Trans. Neural Netw. Learn. Syst., vol. 27, no. 2, pp. 444458, Feb. 2016.

[23] 
Y. Zhu, D. Zhao, and X. Li, "Iterative adaptive dynamic programming for solving unknown nonlinear zerosum game based on online data, " IEEE Trans. Neural Netw. Learn. Syst., vol. 28, no. 3, pp. 714725, Mar. 2017.

[24] 
A. AlTamimi, F. L. Lewis, and M. AbuKhalaf, "Discretetime nonlinear hjb solution using approximate dynamic programming: convergence proof, " IEEE Trans. Syst., Man, Cybern., Part B: Cybern., vol. 38, no. 4, pp. 943949, Aug. 2008.

[25] 
Q. Wei, D. Liu, Q. Lin, and R. Song, "Adaptive dynamic programming for discretetime zerosum games, " IEEE Trans. Neural Netw. Learn. Syst., vol. 29, no. 4, pp. 957969, Apr. 2018.

[26] 
D. Liu, H. Javaherian, O. Kovalenko, and T. Huang, "Adaptive critic learning techniques for engine torque and airfuel ratio control, " IEEE Trans. Syst., Man, Cybern., Part B: Cybern., vol. 38, no. 4, pp. 988993, Aug. 2008.

[27] 
H. Zhang, R. Song, Q. Wei, and T. Zhang, "Optimal tracking control for a class of nonlinear discretetime systems with time delays based on heuristic dynamic programming, " IEEE Trans. Neural Netw., vol. 22, no. 12, pp. 18511862, Dec. 2011.

[28] 
R. Song, Q. Wei, W. Xiao, and Z. Du, "Nearly optimal tracking control for continuous time nonlinear systems using a policy iteration based HJB approach, " in Proc. 34th IEEE Chinese Control Conference (CCC), Hangzhou, China, 2015, pp. 31693172.

[29] 
Q. Wei, R. Song, and Q. Sun, "Nonlinear neurooptimal tracking control via stable iterative Qlearning algorithm, " Neurocomputing, vol. 168, pp. 520528, Nov. 2015.

[30] 
B. Zhao, D. Liu, Y. Li, Q. Wei, and R. Song, Adaptive dynamic programming based decentralized tracking control for unknown largescale systems, in Proc. 36th IEEE Chinese Control Conference (CCC), Dalian, China, 2017, pp. 35753580.

[31] 
Y. Lv, X. Ren, J. Na, and L. Li, $H_{infty}$ tracking control problem for completely unknown nonlinear system based on augmented matrix, in Proc 9th IEEE International Conference on Modelling, Identification and Control (ICMIC), Kunming, China, 2017, pp. 712.

[32] 
B. Luo, D. Liu, T. Huang, and J. Liu, "Output tracking control based on adaptive dynamic programming with multistep policy evaluation, " IEEE Trans., Syst., Man, Cybern., Syst., 2017, DOI: 10.1109/TSMC.2017. 2771516.

[33] 
Q. Yang and S. Jagannathan, "Reinforcement learning controller design for affine nonlinear discretetime systems using online approximators, " IEEE Trans. Syst., Man, Cybern., Part B: Cybern., vol. 42, no. 2, pp. 377390, Apr. 2012.

[34] 
H. Modares, F. L. Lewis, and Z. P. Jiang, "$H_{infty}$ tracking control of completely unknown continuoustime systems via offpolicy reinforcement learning, " IEEE Trans. Neural Netw. Learn. Syst., vol. 26, no. 10, pp. 25502562, Oct. 2015.

[35] 
H. Zhang, X. Cui, Y. Luo, and H. Jiang, "Finitehorizon $H_{infty}$ tracking control for unknown nonlinear systems with saturating actuators, " IEEE Trans. Neural Netw. Learn. Syst., vol. 29, no. 4, pp. 12001212, Apr. 2018.

[36] 
D. Wang, D. Liu, and Q. Wei, "Finitehorizon neurooptimal tracking control for a class of discretetime nonlinear systems using adaptive dynamic programming approach, " Neurocomputing, vol. 78, no. 1, pp. 1422, Feb. 2012.

[37] 
A. Rantzer, "Relaxed dynamic programming in switching systems, " IEE Proc., Control Theory, vol. 153, no. 5, pp. 567574, Sep. 2006.

[38] 
B. Lincoln and A. Rantzer, "Relaxing dynamic programming, " IEEE Trans. Autom. Control, vol. 51, no. 8, pp. 12491260, Aug. 2006.

[39] 
H. Zhang, Y. Luo, and D. Liu, "Neuralnetworkbased nearoptimal control for a class of discretetime affine nonlinear systems with control constraints, " IEEE Trans. Neural Netw., vol. 20, no. 9, pp. 14901503, Sep. 2009.
