Abstract
In this paper, it is shown that, if the expected cost-to-go functions generated by a suboptimal design for a partially observed, discrete-time, Markov decision problem with a specific state measurement quality are concave, then the suboptimal design has a desirable adaptivity characteristic relative to that state measurement quality. Optimal strategies are shown to possess this adaptivity characteristic, as does a suboptimal design presented in an example.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bertsekas, D. P.,Dynamic Programming and Stochastic Control, Academic Press, New York, New York, 1976.
Asher, R. B., Andrisani, D., andDorato, P.,Bibliography on Adaptive Control Systems, Proceedings of the IEEE, Vol. 64, pp. 1226–1240, 1976.
Saridis, G.,Self-Organizing Control of Stochastic Systems, Marcel Dekker, New York, New York, 1977.
Shreve, S. E.,Dynamic Programming in Complete Separable Spaces, University of Illinois, PhD Thesis, 1977.
Parthasarathy, K. R.,Probability Measures on Metric Spaces, Academic Press, New York, New York, 1967.
Striebel, C.,Optimal Control of Discrete-Time Stochastic Systems, Springer-Verlag, Berlin, Germany, 1975.
Chung, K. L.,A Course in Probability Theory, Harcourt, Brace and World, New York, New York, 1968.
Åström, K. J.,Optimal Control of Markov Processes with Incomplete State Information, II, Journal of Mathematical Analysis and Applications, Vol. 26, pp. 403–406, 1969.
Åström, K. J.,Optimal Control of Markov Processes with Incomplete State Information, Journal of Mathematical Analysis and Applications, Vol. 10, pp. 174–205, 1965.
White, C. C.,Application of Two Inequality Results for Concave Functions to a Stochastic Optimization Problem, Journal of Mathematical Analysis and Applications, Vol. 55, pp. 347–350, 1976.
Sternby, J.,A Simple Dual Control Problem With an Analytical Solution, IEEE Transactions on Automatic Control, Vol. AC-21, pp. 840–844, 1976.
Bar-Shalom, Y., andTse, E.,Dual Effect, Certainty Equivalence, and Separation in Stochastic Control, IEEE Transactions on Automatic Control, Vol. AC-19, pp. 494–500, 1974.
Author information
Authors and Affiliations
Additional information
Communicated by C. T. Leondes
This research was supported by NSF Grant No. ENG-76-15774 and NSF Grant No. ENG-78-06733.
Rights and permissions
About this article
Cite this article
White, C.C., Harrington, D.P. Application of Jensen's inequality to adaptive suboptimal design. J Optim Theory Appl 32, 89–99 (1980). https://doi.org/10.1007/BF00934845
Issue Date:
DOI: https://doi.org/10.1007/BF00934845