Optimal switching problem for countable Markov chains: average reward criterion

Optimal switching we consider is the following generalization of optimal stopping: (i) there are a reward function and a cost function on the state space of a Markov chain; (ii) a controller selects stopping times sequentially; (iii) at those times the controller receives rewards and pays costs in an alternating order. In this paper we treat the case of a positive recurrent countable Markov chain and the average per unit time criterion. We find an optimal strategy and the maximal average gain in terms of the solution of a variational problem with two obstacles, known also in connection with Dynkin games.

Authors and Affiliations

1. University of North Carolina at Charlotte, Department of Mathematics, Charlotte, NC 28223, USA (e-mail: aayushke@email.uncc.edu), , , , , , US

   Alexander Yushkevich

Manuscript received: May 2000

Reprints and permissions