logo

SCIENCE CHINA Information Sciences, Volume 60, Issue 12: 120204(2017) https://doi.org/10.1007/s11432-017-9167-3

Time-inconsistent stochastic linear quadratic control for discrete-time systems

More info
  • ReceivedApr 16, 2017
  • AcceptedJul 16, 2017
  • PublishedNov 6, 2017

Abstract

This paper is mainly concerned with the time-inconsistent stochastic linear quadratic (LQ) control problem in a more general formulation for discrete-time systems. The time-inconsistency arises from three aspects: the coefficient matrices depending on the initial pair, the terminal of the cost function involving the initial pair together with the nonlinear terms of the conditional expectation. The main contributions are: firstly, the maximum principle is derived by using variational methods, which forms a flow of forward and backward stochastic difference equations (FBSDE); secondly, in the case of the system state being one-dimensional, the equilibrium control is obtained by solving the FBSDE with feedback gain based on several nonsymmetric Riccati equations; finally, the necessary and sufficient solvability condition for the time-inconsistent LQ control problem is presented explicitly. The key techniques adopted are the maximum principle and the solution to the FBSDE developed in this paper.


Acknowledgment

This work was supported by National Natural Science Foundation of China (Grant Nos. 61120106011, 61573221, 61633014). Qingyuan QI was supported by the Program for Outstanding Ph.D. Candidate of Shandong University.


References

[1] Anderson B D O, Moore J B. Linear Optimal Control. Englewood Cliffs: Prentice-Hall, 1971. Google Scholar

[2] Bertsekas D P. Dynamic Programming and Optimal Control. Belmont: Athena Scientific, 1995. Google Scholar

[3] Yong J M, Zhou X Y. Stochastic Controls: Hamiltonian Systems and HJB Equations. New York: Springer Science & Business Media, 1999. Google Scholar

[4] Qiu L, Xu B G, Li S B. H 2/H control of networked control system with random time delays. Sci China Inf Sci, 2011, 54: 2615-2630 CrossRef Google Scholar

[5] Wang L, Guo G, Zhuang Y. Stabilization of NCSs by random allocation of transmission power to sensors. Sci China Inf Sci, 2016, 59: 067201 CrossRef Google Scholar

[6] Goldman S M. Consistent Plans. Rev Economic Studies, 1980, 47: 533-537 CrossRef Google Scholar

[7] Pliska S. Introduction to Mathematical Finance. Oxford: Blackwell Publishers, 1997. Google Scholar

[8] Peleg B, Yaari M E. On the Existence of a Consistent Course of Action when Tastes are Changing. Rev Economic Studies, 1973, 40: 391-401 CrossRef Google Scholar

[9] Phelps E S, Pollak R A. On Second-Best National Saving and Game-Equilibrium Growth. Rev Economic Studies, 1968, 35: 185-199 CrossRef Google Scholar

[10] Vieille N, Weibull J W. Multiple solutions under quasi-exponential discounting. Econ Theor, 2009, 39: 513-526 CrossRef Google Scholar

[11] Zhou X Y. Continuous-Time Mean-Variance Portfolio Selection: A Stochastic LQ Framework. Appl Math Optimization, 2000, 42: 19-33 CrossRef Google Scholar

[12] Laibson D. Golden Eggs and Hyperbolic Discounting. Q J Economics, 1997, 112: 443-478 CrossRef Google Scholar

[13] Krusell P, Smith, Jr. A A. Consumption-Savings Decisions with Quasi-Geometric Discounting. Econometrica, 2003, 71: 365-375 CrossRef Google Scholar

[14] Bjork T, Murgoci A. A general theory of Markovian time inconsistent stochastic control problems. Working Paper, Stockholm School of Economics, 2009. 1--65. Google Scholar

[15] Miller M, Salmon M. Dynamic Games and the Time Inconsistency of Optimal Policy in Open Economies. Economic J, 1985, 95: 124-137 CrossRef Google Scholar

[16] Strotz R H. Myopia and Inconsistency in Dynamic Utility Maximization. Rev Economic Studies, 1955, 23: 165-180 CrossRef Google Scholar

[17] Hu Y, Jin H, Zhou X Y. Time-Inconsistent Stochastic Linear--Quadratic Control. SIAM J Control Optim, 2012, 50: 1548-1572 CrossRef Google Scholar

[18] Ekeland I, Lazrak A. Being serious about non-commitment: subgame perfect equilibrium in continuous time,. arXiv Google Scholar

[19] Ekeland I, Pirvu T A. Investment and consumption without commitment. Math Finan Econ, 2008, 2: 57-86 CrossRef Google Scholar

[20] Yong J. A deterministic linear quadratic time-inconsistent optimal control problem. MCRF, 2011, 1: 83-118 CrossRef Google Scholar

[21] Yong J. Deterministic time-inconsistent optimal control problems-an essentially cooperative approach. Acta Math Appl Sin-E, 2012, 28: 1--30. Google Scholar

[22] Li X, Ni Y H, Zhang J F. Discrete-time stochastic linear-quadratic optimal control with time-inconsistency. IFAC-PapersOnLine, 2015, 48: 691--696. Google Scholar

[23] Huang J, Zhang D. The near-optimal maximum principle of impulse control for stochastic recursive system. Sci China Inf Sci, 2016, 59: 112206 CrossRef Google Scholar

[24] Li X, Ni Y H, Zhang J F. On time-consistent solution to time-inconsistent linear-quadratic optimal control of discrete-time stochastic systems,. arXiv Google Scholar

[25] Ni Y H, Zhang J F. Stochastic linear-quadratic optimal control without time-consistency requirement. Commun Inf Syst, 2015, 15: 521--550. Google Scholar

[26] Ni Y H, Zhang J F, Krstic M. Time-inconsistent mean-field stochastic LQ problem: open-loop time-consistent control,. arXiv Google Scholar

[27] Yong J. Time-inconsistent optimal control problems and the equilibrium HJB equation. MCRF, 2012, 2: 271-329 CrossRef Google Scholar

[28] Hu Y, Jin H Q, Zhou X Y. Time-inconsistent stochastic linear-quadratic control: characterization and uniqueness of equilibrium,. arXiv Google Scholar

[29] Ni Y H. Time-inconsistent mean-field stochastic linear-quadratic optimal control. In: Proceedings of the 35th Chinese Control Conference, Chengdu, 2016. 2577--2582. Google Scholar

[30] Markowitz H. Portfolio selection. J Financ, 1952, 7: 77--91. Google Scholar

[31] Rouge R, El Karoui N. Pricing Via Utility Maximization and Entropy. Math Finance, 2000, 10: 259-276 CrossRef Google Scholar

[32] Zhang H S, Wang H X, Li L. Adapted and casual maximum principle and analytical solution to optimal control for stochastic multiplicative-noise systems with multiple input-delays. In: Proceedings of the 51st IEEE Conference on Decision and Control, Maui, 2012. 2122--2127. Google Scholar

[33] Zhang H S, Qi Q Y. Optimal control for mean-field system: discrete-time case. In: Proceedings of the 55th IEEE Conference on Decision and Control, Las Vegas, 2016. 4474--4480. Google Scholar

[34] Peng S. A General Stochastic Maximum Principle for Optimal Control Problems. SIAM J Control Optim, 1990, 28: 966-979 CrossRef Google Scholar

Copyright 2019 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有

京ICP备18024590号-1