A journal of IEEE and CAA , publishes high-quality papers in English on original theoretical/experimental research and development in all areas of automation
Volume 1 Issue 4
Oct.  2014

IEEE/CAA Journal of Automatica Sinica

  • JCR Impact Factor: 6.171, Top 11% (SCI Q1)
    CiteScore: 11.2, Top 5% (Q1)
    Google Scholar h5-index: 51, TOP 8
Turn off MathJax
Article Contents
Manoj Kumar, Karthikeyan Rajagopal, Sivasubramanya Nadar Balakrishnan and Nhan T. Nguyen, "Reinforcement Learning Based Controller Synthesis for Flexible Aircraft Wings," IEEE/CAA J. of Autom. Sinica, vol. 1, no. 4, pp. 435-448, 2014.
Citation: Manoj Kumar, Karthikeyan Rajagopal, Sivasubramanya Nadar Balakrishnan and Nhan T. Nguyen, "Reinforcement Learning Based Controller Synthesis for Flexible Aircraft Wings," IEEE/CAA J. of Autom. Sinica, vol. 1, no. 4, pp. 435-448, 2014.

Reinforcement Learning Based Controller Synthesis for Flexible Aircraft Wings


This work was supported by National Science Foundation of USA (1002333).

  • Aeroelastic study of flight vehicles has been a subject of great interest and research in the last several years. Aileron reversal and flutter related problems are due in part to the elasticity of a typical airplane. Structural dynamics of an aircraft wing due to its aeroelastic nature are characterized by partial differential equations. Controller design for these systems is very complex as compared to lumped parameter systems defined by ordinary differential equations. In this paper, a stabilizing statefeedback controller design approach is presented for the heave dynamics of a wing-fuselage model. In this study, a continuous actuator in the spatial domain is assumed. A control methodology is developed by combining the technique of "proper orthogonal decomposition" and approximate dynamic programming. The proper orthogonal decomposition technique is used to obtain a low-order nonlinear lumped parameter model of the infinite dimensional system. Then a near optimal controller is designed using the single-network-adaptive-critic technique. Furthermore, to add robustness to the nominal single-network-adaptive-critic controller against matched uncertainties, an identifier based adaptive controller is proposed. Simulation results demonstrate the effectiveness of the single-network-adaptive-critic controller augmented with adaptive controller for infinite dimensional systems.


  • loading
  • [1]
    Lasiecka I. Control of systems governed by partial differential equations:a historical perspective. In: Proceedings of the 34th Conference onDecision and Control. New Orleans, LA: IEEE, 1995. 2792-2796
    Burns J A, King B B. Optimal sensor location for robust control ofdistributed parameter systems. In: Proceedings of the 1994 Conferenceon Decision and Control. Lake Buena Vista, FL, USA: IEEE, 1994.3967-3972
    Curtain R F, Zwart H J. An introduction to Infinite Dimensional LinearSystems Theory. New York: Springer-Verlag, 1995.
    Padhi R, Balakrishnan S N. Optimal dynamic inversion control designfor a class of nonlinear distributed parameter systems with continuousand discrete actuators. The Institute of Engineering and Technology,Control Theory, and Applications, 2007, 1(6): 1662-2671
    Haykin S. Neural Networks. New York: Macmillan College Company,1994.
    Gupta S K. Numerical Methods for Engineers. New Delhi: New AgeInternational Ltd., 1995.
    Christofides P D. Nonlinear and Robust Control of PDE Systems:Methods and Applications to Transport-Reaction Processes. Boston:Birkhauser Boston Inc., 2001.
    Holmes P, Lumley J L, Berkooz G. Turbulence Coherent Structures,Dynamical Systems and Symmetry. Cambridge: Cambridge UniversityPress, 1996. 87-154
    Bryson A E, Ho Y C. Applied Optimal Control: Optimization, Estimation,and Control. Washington: Taylor and Francis, 1975
    Lewis F. Applied Optimal Control and Estimation. New Jersey: Prentice-Hall, 1992.
    Werbos P J. Neurocontrol and supervised learning: an overview andevaluation. Handbook of Intelligent Control: Neural, Fuzzy and AdaptiveApproaches. New York: Van Nostrand Reinhold, 1992
    Balakrishnan S N, Biega V. Adaptive-critic based neural networks foraircraft optimal control. Journal of Guidance, Control and Dynamics,1996, 19(4): 893-898
    Padhi R, Unnikrishnan N, Wang X, Balakrishnan S N. A single networkadaptive critic (SNAC) architecture for optimal control synthesis for aclass of nonlinear systems. Neural Networks, 2006, 19: 1648-1660
    Padhi R, Balakrishnan S N. Optimal beaver population managementusing reduced-order distributed parameter model and single networkadaptive critics. In: Proceedings of the 2004 American Control Conference.Boston, MA, 2004. 1598-1603
    Yadav V, Padhi R, Balakrishnan S N. Robust/optimal temperature profilecontrol of a re-entry vehicle using neural networks. In: Proceedings ofthe 2006 AIAA Atmospheric Flight Mechanics Conference and Exhibit.Keystone, Colorado: AIAA, 2006. 21-24
    Prabhat P, Balakrishnan S N, Look D C Jr. Experimental implementationof adaptive-critic based infinite time optimal neurocontrol for aheat diffusion system. In: Proceedings of the 2002 American ControlConference. Alaska, USA: IEEE, 2002. 2671-2676
    Chakravarthy A, Evans K A, Evers J. Sensitivity & functional gains fora flexible aircraft-inspired model. In: Proceedings of the 2011 AmericanControl Conference. Baltimore, MD, USA: IEEE, 2010. 4893-4898
    Chakravarthy A, Evans K A, Evers J, Kuhn L M. Target tracking strategiesfor a nonlinear, flexible aircraft-inspired model. In: Proceedings ofthe 2011 American Control Conference. San Francisco: IEEE, 2011.1783-1788
    Narendra K S, Annaswamy A M. Stable Adaptive Systems. EnglewoodCliffs, NJ: Prentice-Hall, 1989.
    Ioannou P A, Sun J. Robust Adaptive Control. Englewood Cliffs, NJ:Prentice-Hall, 1995.
    Böhm M, Demetriou M A, Reich S, Rosen I G. Model reference adaptivecontrol of distributed parameter systems. SIAM Journal of Control andOptimization, 1998, 36(1): 33-81
    Hong K S, Bentsman J. Direct adaptive control of parabolic systems:algorithm synthesis and convergence and stability analysis. IEEE Transactionsof Automatic Control, 1994, 39(10): 2018-2033
    Hong K S, Yang K J, Kang D H. Model reference adaptive control of atime-varying parabolic system.KSME International Journal, 2000, 14(2):168-176
    Krstic M, Smyshlyaev A. Adaptive control of PDEs. Annual Reviews inControl, 2008, 32(2): 149-160
    Demetriou A M, Rosen I G. On-line robust parameter identification forparabolic systems. International Journal of Adaptive Control and SignalProcessing, 15(6): 615-631
    Sirovich L. Turbulence and the dynamics of coherent structures. Quarterlyof Applied Mathematics, 1987, 45(3): 561-590
    Ravindran S S. Proper Orthogonal Decomposition in Optimal Controlof Fluids, NASA/TM-1999-209113. USA, 1999.
    Padhi R, Balakrishnan S N. Proper orthogonal decomposition basedoptimal control design of heat equation with discrete actuators usingneural networks. American Control Conference, ACC02-IEEE 1545,2002.
    Padhi R, Balakrishnan S N. Proper orthogonal decomposition basedfeedback optimal control synthesis of distributed parameter systemsusing neural networks. In: Proceedings of the 15th American ControlConference. Anchorage, AK: IEEE, 2002. 4389-4394
    Ravindran S S. Adaptive reduced-order controllers for a thermal flowsystem using proper orthogonal decomposition. SIAM Journal on ScientificComputing, 2002, 23(6): 1924-1942
    Popov V M. Hyperstability of Control Systems. Berlin: Springer, 1973.
    Olver P J. Introduction to Partial Differential Equations. New York:Springer-Verlag, 2013.


    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Article Metrics

    Article views (1047) PDF downloads(7) Cited by()


    DownLoad:  Full-Size Img  PowerPoint