logo

SCIENCE CHINA Information Sciences, Volume 63 , Issue 5 : 150210(2020) https://doi.org/10.1007/s11432-019-2663-y

Event-triggered receding horizon control via actor-critic design

More info
  • ReceivedMay 15, 2019
  • AcceptedSep 16, 2019
  • PublishedMar 30, 2020

Abstract

In this paper, we propose a novel event-triggered near-optimal control for nonlinear continuous-time systems. The receding horizon principle is utilized to improve the system robustness and obtain better dynamic control performance. In the proposed structure, we first decompose the infinite horizon optimal control into a series of finite horizon optimal problems. Then a learning strategy is adopted, in which an actor network is employed to approximate the cost function and an critic network is used to learn the optimal control law in each finite horizon. Furthermore, in order to reduce the computational cost and transmission cost, an event-triggered strategy is applied. We design an adaptive trigger condition, so that the signal transmissions and controller updates are conducted in an aperiodic way. Detailed stability analysis shows that the nonlinear system with the developed event-triggered optimal control policy is asymptotically stable. Simulation results on a single-link robot arm with different noise types have demonstrated the effectiveness of the proposed method.


Acknowledgment

This work was supported in part by National Natural Science Foundation of China (Grant Nos. 61803085, 61921004, 61931020) and National Key RD Program of China (Grant No. 2018AAA0101400).


References

[1] Song R Z, Xiao W D, Sun C Y. A new self-learning optimal control laws for a class of discrete-time nonlinear systems based on ESN architecture. Sci China Inf Sci, 2014, 57: 1-10 CrossRef Google Scholar

[2] Li C, Zhang E, Jiu L. Optimal control on special Euclidean group via natural gradient algorithm. Sci China Inf Sci, 2016, 59: 112203 CrossRef Google Scholar

[3] Wei W N. Maximum principle for optimal control of neutral stochastic functional differential systems. Sci China Math, 2015, 58: 1265-1284 CrossRef ADS Google Scholar

[4] Lee J H. Model predictive control: Review of the three decades of development. Int J Control Autom Syst, 2011, 9: 415-424 CrossRef Google Scholar

[5] Bequette B W. Nonlinear control of chemical processes: a review. Ind Eng Chem Res, 1991, 30: 1391-1413 CrossRef Google Scholar

[6] Mayne D Q, Rawlings J B, Rao C V. Constrained model predictive control: Stability and optimality. Automatica, 2000, 36: 789-814 CrossRef Google Scholar

[7] Hu L S, Huang B, Cao Y Y. Robust Digital Model Predictive Control for Linear Uncertain Systems With Saturations. IEEE Trans Automat Contr, 2004, 49: 792-796 CrossRef Google Scholar

[8] Evans M, Cannon M, Kouvaritakis B. Robust and stochastic linear MPC for systems subject to multiplicative uncertainty. IFAC Proc Volumes, 2012, 45: 335-341 CrossRef Google Scholar

[9] Fang H, Shang C, Chen J. An optimization-based shared control framework with applications in multi-robot systems. Sci China Inf Sci, 2018, 61: 014201 CrossRef Google Scholar

[10] Zhou L, Jia L, Wang Y L. A robust integrated model predictive iterative learning control strategy for batch processes. Sci China Inf Sci, 2019, 62: 219202 CrossRef Google Scholar

[11] Aguilera R P, Lezana P, Quevedo D E. Switched Model Predictive Control for Improved Transient and Steady-State Performance. IEEE Trans Ind Inf, 2015, 11: 968-977 CrossRef Google Scholar

[12] Venkat A N, Hiskens I A, Rawlings J B. Distributed MPC Strategies With Application to Power System Automatic Generation Control. IEEE Trans Contr Syst Technol, 2008, 16: 1192-1206 CrossRef Google Scholar

[13] Bertsekas D P. Dynamic Programming and Suboptimal Control: A Survey from ADP to MPC*. Eur J Control, 2005, 11: 310-334 CrossRef Google Scholar

[14] Wei Q L, Liu D R. A novel policy iteration based deterministic Q-learning for discrete-time nonlinear systems. Sci China Inf Sci, 2015, 58: 1-15 CrossRef Google Scholar

[15] Wang D, Mu C. Developing nonlinear adaptive optimal regulators through an improved neural learning mechanism. Sci China Inf Sci, 2017, 60: 058201 CrossRef Google Scholar

[16] Ding D, Wang Z, Han Q L. Neural-Network-Based Output-Feedback Control Under Round-Robin Scheduling Protocols.. IEEE Trans Cybern, 2019, 49: 2372-2384 CrossRef PubMed Google Scholar

[17] Ernst D, Glavic M, Capitanescu F. Reinforcement learning versus model predictive control: a comparison on a power system problem.. IEEE Trans Syst Man Cybern B, 2009, 39: 517-529 CrossRef PubMed Google Scholar

[18] G?rges D. Relations between Model Predictive Control and Reinforcement Learning. IFAC-PapersOnLine, 2017, 50: 4920-4928 CrossRef Google Scholar

[19] Lian C, Xu X, Chen H. Near-Optimal Tracking Control of Mobile Robots Via Receding-Horizon Dual Heuristic Programming.. IEEE Trans Cybern, 2016, 46: 2484-2496 CrossRef PubMed Google Scholar

[20] Xu X, Chen H, Lian C. Learning-Based Predictive Control for Discrete-Time Nonlinear Systems With Stochastic Disturbances.. IEEE Trans Neural Netw Learning Syst, 2018, 29: 6202-6213 CrossRef PubMed Google Scholar

[21] Dong L, Yan J, Yuan X. Functional Nonlinear Model Predictive Control Based on Adaptive Dynamic Programming.. IEEE Trans Cybern, 2019, 49: 4206-4218 CrossRef PubMed Google Scholar

[22] Anta A, Tabuada P. To Sample or not to Sample: Self-Triggered Control for Nonlinear Systems. IEEE Trans Automat Contr, 2010, 55: 2030-2042 CrossRef Google Scholar

[23] Lemmon M D. Event-triggered feedback in control, estimation, and optimization. In: Networked Control Systems. London: Springer, 2010. 293--358. Google Scholar

[24] Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems. IEEE/CAA J Autom Sin, 2014, 1: 282-293 CrossRef Google Scholar

[25] Duan G, Xiao F, Wang L. Hybrid event- and time-triggered control for double-integrator heterogeneous networks. Sci China Inf Sci, 2019, 62: 022203 CrossRef Google Scholar

[26] Zhang X M, Han Q L. Event-triggered dynamic output feedback control for networked control systems. IET Control Theor Appl, 2014, 8: 226-234 CrossRef Google Scholar

[27] Zhang B L, Han Q L, Zhang X M. Event-triggered H$_{}$ reliable control for offshore structures in network environments. J Sound Vib, 2016, 368: 1-21 CrossRef ADS Google Scholar

[28] Zhang X M, Han Q L. A Decentralized Event-Triggered Dissipative Control Scheme for Systems With Multiple Sensors to Sample the System Outputs.. IEEE Trans Cybern, 2016, 46: 2745-2757 CrossRef PubMed Google Scholar

[29] Liu C, Gao J, Li H. Aperiodic Robust Model Predictive Control for Constrained Continuous-Time Nonlinear Systems: An Event-Triggered Approach.. IEEE Trans Cybern, 2018, 48: 1397-1405 CrossRef PubMed Google Scholar

[30] Li H, Shi Y. Event-triggered robust model predictive control of continuous-time nonlinear systems. Automatica, 2014, 50: 1507-1513 CrossRef Google Scholar

[31] Ding D, Han Q L, Wang Z. A Survey on Model-Based Distributed Control and Filtering for Industrial Cyber-Physical Systems. IEEE Trans Ind Inf, 2019, 15: 2483-2499 CrossRef Google Scholar

[32] Krichman M, Sontag E D, Wang Y. Input-Output-to-State Stability. SIAM J Control Optim, 2001, 39: 1874-1928 CrossRef Google Scholar

[33] Tabuada P. Event-Triggered Real-Time Scheduling of Stabilizing Control Tasks. IEEE Trans Automat Contr, 2007, 52: 1680-1685 CrossRef Google Scholar

[34] Heydari A, Balakrishnan S N. Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics.. IEEE Trans Neural Netw Learning Syst, 2013, 24: 145-157 CrossRef PubMed Google Scholar

[35] Khalil H K. Nonlinear Systems. New Jersey: Prentice Hall, 2002. Google Scholar

[36] Dong L, Zhong X, Sun C. Event-Triggered Adaptive Dynamic Programming for Continuous-Time Systems With Control Constraints.. IEEE Trans Neural Netw Learning Syst, 2017, 28: 1941-1952 CrossRef PubMed Google Scholar

[37] Zhong X, He H. An Event-Triggered ADP Control Approach for Continuous-Time System With Unknown Internal States.. IEEE Trans Cybern, 2017, 47: 683-694 CrossRef PubMed Google Scholar

[38] Chen H. Model Predictive Control. Beijing: Sci Press, 2013. Google Scholar

  • Figure 1

    (Color online) The proposed event-triggered RHAC structure.

  • Figure 2

    (Color online) Comparison of the state trajectories for the single-link robot arm system with Gaussian sensor noise.

  • Figure 3

    (Color online) Event-triggered RHAC algorithm for the single-link robot arm system with Gaussian sensor noise. (a) Evolution of the control input; (b) comparison of trigger error $||e_j||$ and trigger threshold $||e_T||$; (c) cumulative number of events.

  • Figure 4

    (Color online) Comparison of the state trajectories for the single-link robot arm system with uniform actuator noise.

  • Figure 5

    (Color online) Event-triggered RHAC algorithm for the single-link robot arm system with uniform actuator noise. (a) Evolution of the control input; (b) comparison of trigger error $||e_j||$ and trigger threshold $||e_T||$; (c) cumulative number of events.

Copyright 2020  CHINA SCIENCE PUBLISHING & MEDIA LTD.  中国科技出版传媒股份有限公司  版权所有

京ICP备14028887号-23       京公网安备11010102003388号