Zero-Sum Discounted Reward Criterion Games for Piecewise Deterministic Markov Processes
Authors: 
O. L. Costa, F. DufourAuthors Info & Claims
Pages 587 - 611
Abstract
This papers deals with the zero-sum game with a discounted reward criterion for piecewise deterministic Markov process (PDMPs) in general Borel spaces. The two players can act on the jump rate and transition measure of the process, with the decisions being taken just after a jump of the process. The goal of this paper is to derive conditions for the existence of min---max strategies for the infinite horizon total expected discounted reward function, which is composed of running and boundary parts. The basic idea is, by using the special features of the PDMPs, to re-write the problem via an embedded discrete-time Markov chain associated to the PDMP and re-formulate the problem as a discrete-stage zero sum game problem.
References
[1]
Costa, O.L.V., Dufour, F.: Continuous average control of piecewise deterministic Markov processes. Springer, New York (2013)
[2]
Davis, M.H.A.: Piecewise-deterministic Markov processes: a general class of non-diffusion stochastic models. J. R. Stat. Soc. (B) 46(3), 353---388 (1984)
[3]
Davis, M.H.A.: Markov Models and Optimization. Chapman and Hall, London (1993)
[4]
Davis, M.H.A., Dempster, M.A.H., Sethi, S.P., Vermes, D.: Optimal capacity expansion under uncertainty. Adv. Appl. Probab. 19(1), 156---176 (1987)
[5]
Fan, K.: Minimax theorems. Proc. Natl. Acad. Sci. USA 39, 42---47 (1953)
[6]
Filar, J.A., Vrieze, K.: Competitive Markov decision processes. Springer, New York (1997)
[7]
Gonzáles-Trejo, J.I., Hernández-Lerma, O., Hoyos-Reyes, L.F.: Minimax control of discrete-time stochastic systems. SIAM J. Control Optim. 41, 1626---1659 (2003)
[8]
Guo, X., Hernanez-Lerma, O.: Zero-sum games for continuous-time jump Markov processes in polish spaces: discounted payoffs. Adv. Appl. Probab. 39, 646---668 (2007)
[9]
Guo, X.P., Hernández-Lerma, O.: New optimality conditions for average-payoff continuous-time Markov games in Polish spaces. Sci. China Math. 54, 793---816 (2011)
[10]
Hernández-Lerma, O., Lasserre, J.B.: Zero-sum stochastic games in Borel spaces: average payoff criterion. SIAM J. Control Optim. 39, 1520---1539 (2001)
[11]
Hernández-Lerma, O., Lasserre, J.B.: Further Topics on Discrete-Time Markov Control Processes. Applications of Mathematics, vol. 42. Springer, New York (1999)
[12]
Jacod, J.: Calcul stochastique et problèmes de martingales. Lecture Notes in Mathematics, vol. 714. Springer, Berlin (1979)
[13]
Jaśkiewicz, A.: Zero-sum semi-Markov games. SIAM J. Control Optim. 41, 723---739 (2002)
[14]
Jaśkiewicz, A.: Zero-sum ergodic semi-Markov games with weakly continuous transition probabilities. J. Optim. Theory Appl. 141, 321---347 (2009)
[15]
Jaskiewicz, A., Nowak, A.S.: Zero-sum ergodic stochastic games with Feller transition probabilities. SIAM J. Control Optim. 45(3), 773---789 (2006)
[16]
Jaśkiewicz, A., Nowak, A.S.: Stochastic games with unbounded payoffs: applications to robust control in economics. Dyn. Games Appl. 1, 253---279 (2011)
[17]
Kuenle, Heinz-Uwe: On Markov games with average reward criterion and weakly continuous transition probabilities. SIAM J. Control Optim. 45, 2156---2168 (2007)
[18]
Nowak, A.S.: Measurable selection theorems for minimax stochastic optimization problems. SIAM J. Control Optim. 23, 466---476 (1985)
[19]
Rieder, U.: On semi-continuous dynamic games. Technical report, University of Karlsruhe, Karlsruhe, Germany, 1978
[20]
Tweedie, Richard L., Lund, Robert B., Meyn, Sean P.: Computable exponential convergence rates for stochastically ordered markov processes. Ann. Appl. Probab. 6(1), 218---237 (1996)
[21]
Van der Duyn Schouten, F.A.: Markov decision drift processes. In: Janssen, J. (ed.) Semi-Markov Models: Theroy and Applications, Chapter 2, pp. 63---78. Springer, New York (1984)
[22]
Vega-Amaya, O.: Zero-sum average semi-Markov games: fixed-point solutions of the Shapley equation. SIAM J. Control Optim. 42, 1876---1894 (2003)
Recommendations
On Risk-Sensitive Piecewise Deterministic Markov Decision Processes
AbstractWe consider a piecewise deterministic Markov decision process, where the expected exponential utility of total (nonnegative) cost is to be minimized. The cost rate, transition rate and post-jump distributions are under control. The state space is ...
Risk-Sensitivity Vanishing Limit for Controlled Markov Processes
AbstractIn this paper, we prove that the optimal risk-sensitive reward for Markov decision processes with compact state space and action space converges to the optimal average reward as the risk-sensitive factor tends to 0. In doing so, a variational ...
Optimally solving Markov decision processes with total expected discounted reward function
Compared computational performance of linear programming and the policy iteration.Considered only discrete-time infinite-horizon MDPs with discounted reward.Used randomly generated test problems and a real-life health-care problem.Showed that, unlike ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © Copyright © 2018 Springer Science+Business Media, LLC, part of Springer Nature.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 01 December 2018
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 13 Nov 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in