Abstract
Problems of convergence and stability of Bayesian estimates in the identification of stochastic control systems are considered. The informational measure of the mismatch between the estimated distribution and the estimate is the main apparatus for establishing the fact of convergence. The choice of a priori distribution of parameters is not always obvious. The Kullback-Leibler information number is taken as such measure. The convergence of the estimates of the transition function of the process to the non-stationary transition function is established in this paper. The problem of synthesis of optimal strategies for dynamic systems in which there is no part of the main information needed for constructing the optimal control is also considered. It is assumed that the system contains at least one unknown parameter belonging to some parameter space. Therefore, the class of control systems considered in the article is the class of parametric adaptive systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Demyanov, V.F., Karelin, V.V.: On a minimax approach to the problem of identification of dynamic systems in the presence of uncertainty. In: Oettli, W., Pallaschke, D. (eds.) Advances in Optimization Proceedings of 6th French-German Colloquium of Optimization. Lecture Notes in Economics and Mathematical Systems, pp. 515–518. Springer, Heidelberg (1991). https://doi.org/10.1007/978-3-642-51682-5
Lipcer, R.S., Sirjaev, A.N.: Statistics of Random Processes: I General Theory. Springer Science & Business Media, Heidelberg (2013). https://doi.org/10.1007/978-3-662-13043-8
Aoki, M.: Optimization of Stochastic Systems. Academic Press, New York (1967)
Karelin, V.V.: Adaptive optimal strategies in controlled markov processes. In: Oettli, W., Pallaschke, D. (eds.) Advances in Optimization Proceedings of 6th French-German Colloquium of Optimization, FRG. Lecture Notes in Economics and Mathematical Systems, pp. 518–525. Springer, Heidelberg (1991)
Liang, H., Wu, H.: Parameter estimation for differential equation models using a framework of measurement error in regression models. J. Am. Stat. Assoc. 103(484), 1570–1583 (2008). https://doi.org/10.1198/016214508000000797
Mariño, I.P., Zaikin, A., Miguez, J.: A comparison of monte carlo-based bayesian parameter estimation methods for stochastic models of genetic networks. PLoS One 12(8), e0182015 (2017). https://doi.org/10.1371/journal.pone.0182015
Prendes, J., Chabert, M., Pascal, F., Giros, A., Tourneret, J.-Y.: A bayesian nonparametric model coupled with a markov random field for change detection in heterogeneous remote sensing images. SIAM J. Imaging Sci. 9(4), 1889–1921 (2016). https://doi.org/10.1137/15M1047908
Schindler, M.R., Phillips, D.R.: Bayesian methods for parameter estimation in effective field theories. Ann. Phys. 324(3), 682–708 (2009). https://doi.org/10.1016/j.aop.2008.09.003
Shaby, B., Ruppert, D.: Tapered covariance: bayesian estimation and asymptotics. J. Comput. Graph. Statist. 21(2), 433–452 (2012). https://doi.org/10.1080/10618600.2012.680819
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Karelin, V., Fominyh, A., Myshkov, S., Polyakova, L. (2019). Parameter Estimation Problems in Markov Random Processes. In: Misra, S., et al. Computational Science and Its Applications – ICCSA 2019. ICCSA 2019. Lecture Notes in Computer Science(), vol 11622. Springer, Cham. https://doi.org/10.1007/978-3-030-24305-0_51
Download citation
DOI: https://doi.org/10.1007/978-3-030-24305-0_51
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-24304-3
Online ISBN: 978-3-030-24305-0
eBook Packages: Computer ScienceComputer Science (R0)