Abstract
We consider duality relations between risk-sensitive stochastic control problems and dynamic games. They are derived from two basic duality results, the first involving free energy and relative entropy and resulting from a Legendre-type transformation, the second involving power functions. Our approach allows us to treat, in essentially the same way, continuous- and discrete-time problems, with complete and partial state observation, and leads to a very natural formal justification of the structure of the cost functional of the dual. It also allows us to obtain the solution of a stochastic game problem by solving a risk-sensitive control problem.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
E. N. Barron and R. Jensen, Total risk aversion, stochastic optimal control, and differential games,Appl. Math. Optim. 19 (1989), 313–327.
A. Bensoussan and J. H. van Schuppen, Optimal control of partially observable stochastic systems with exponential-of-integral performance index,SIAM J. Control Optim. 23 (1985), 599–613.
D. P. Bertsekas,Dynamic Programming and Stochastic Control, Academic Press, London, 1976.
M. Boué and P. Dupuis, A Variational Representation for Certain Functionals of Brownian Motion, LCDS Report #95-7.
C. D. Charalambous, The role of information state and adjoint in relating nonlinear output feedback risk-sensitive control and dynamic games, Preprint.
C. D. Charalambous and R. J. Elliott, Classes of nonlinear partially observable stochastic optimal control problems with explicit optimal control laws, Preprint.
J. D. Deuschel and D. W. Stroock,Large Deviations, Academic Press, New York, 1989.
P. Dupuis and R. S. Ellis, A Weak Convergence Approach to the Theory of Large Deviations, LCDS Report #93-6, Brown University, Providence, RI. Forthcoming book to be published by Wiley.
E. Fernández-Gaucherand and S. I. Marcus, Risk-sensitive optimal control of hidden Markov models: a case study,Proc. 33rd CDC, 1994, pp. 1657–1662.
W. H. Fleming and W. M. McEneany, Risk sensitive optimal control and differential games, inStochastic Theory and Adaptive Control (T. E. Duncan and B. Pasik-Duncan, eds.), Springer-Verlag, New York, 1992, 185–197.
W. H. Fleming and W. M. McEneany, Risk sensitive control on an infinite time horizon,SIAM J. Control Optim. 33 (1995), 1881–1915.
D. H. Jacobson, Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games,IEEE Trans. Automat. Control 18 (1973), 124–131.
M. R. James, Asymptotic analysis of nonlinear stochastic risk-sensitive control and differential games,Math. Control Signals Systems 5 (1992), 401–417.
M. R. James, J. S. Baras, and R. J. Elliott, Risk sensitive control and dynamic games for partially observed discrete-time nonlinear systems,IEEE Trans. Automat. Control 39 (1994), 780–792.
R. S. Lipster and A. N. Shiryaev,Statistics of Random Processes, Vols. 1 and 2, Springer-Verlag, New York, 1988.
L. Meneghini, Modelli risolvibili per problemi di controllo di sistemi dinamici imprecisi multivariati, Thesis, University of Padova, 1994.
T. Runolfsson, The equivalence between infinite-horizon optimal control of stochastic systems with exponential-of-integral performance index and stochastic differential games,IEEE Trans. Automat. Control 39 (1994), 1551–1563.
P. Whittle, Risk-sensitive-linear-quadratic-gaussian control,Adv. in Appl. Probab. 13 (1981), 764–777.
P. Whittle,Risk Sensitive Optimal Control, Wiley, New York, 1990.
P. Whittle, A risk sensitive maximum principle: The case of imperfect state observations,IEEE Trans. Automat. Control 36 (1991), 793–801.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Dai Pra, P., Meneghini, L. & Runggaldier, W.J. Connections between stochastic control and dynamic games. Math. Control Signal Systems 9, 303–326 (1996). https://doi.org/10.1007/BF01211853
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01211853