Abstract
Probabilistic finite automata (PFA) model stochastic languages, i.e. probability distributions over strings. Inferring PFA from stochastic data is an open field of research. We show that PFA are identifiable in the limit with probability one. Multiplicity automata (MA) is another device to represent stochastic languages. We show that a MA may generate a stochastic language that cannot be generated by a PFA, but we show also that it is undecidable whether a MA generates a stochastic language. Finally, we propose a learning algorithm for a subclass of PFA, called PRFA.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Paz, A.: Introduction to probabilistic automata. Academic Press, London (1971)
Abe, N., Warmuth, M.: On the computational complexity of approximating distributions by probabilistic automata. Machine Learning 9, 205–260 (1992)
Dempster, A., Laird, N.M., Rubin, D.B.: Maximum likelyhood from incomplete data via the em algorithm. Journal of the Royal Statistical Society 39, 1–38 (1977)
Baldi, P., Brunak, S.: Bioinformatics: The Machine Learning Approach. MIT Press, Cambridge (1998)
Freitag, D., McCallum, A.: Information extraction with HMM structures learned by stochastic optimization. In: AAAI/IAAI, pp. 584–589 (2000)
Gold, E.: Language identification in the limit. Inform. Control 10, 447–474 (1967)
Angluin, D.: Identifying languages from stochastic examples. Technical Report YALEU/DCS/RR-614, Yale University, New Haven, CT (1988)
Carrasco, R., Oncina, J.: Learning stochastic regular grammars by means of a state merging method. In: ICGI, pp. 139–152. Springer, Heidelberg (1994)
Carrasco, R.C., Oncina, J.: Learning deterministic regular grammars from stochastic samples in polynomial time. RAIRO 33, 1–20 (1999)
de la Higuera, C., Thollard, F.: Identification in the limit with probability one of stochastic deterministic finite automata. In: Oliveira, A.L. (ed.) ICGI 2000. LNCS (LNAI), vol. 1891, pp. 141–156. Springer, Heidelberg (2000)
Bergadano, F., Varricchio, S.: Learning behaviors of automata from multiplicity and equivalence queries. In: Italian Conf. on Algorithms and Complexity (1994)
Beimel, A., Bergadano, F., Bshouty, N.H., Kushilevitz, E., Varricchio, S.: On the applications of multiplicity automata in learning. In: IEEE Symposium on Foundations of Computer Science, pp. 349–358 (1996)
Beimel, A., Bergadano, F., Bshouty, N.H., Kushilevitz, E., Varricchio, S.: Learning functions represented as multiplicity automata. Journal of the ACM 47, 506–530 (2000)
Thollard, F., Dupont, P., de la Higuera, C.: In: Proc. 17th ICML in title
Kearns, M., Mansour, Y., Ron, D., Rubinfeld, R., Schapire, R.E., Sellie, L.: On the learnability of discrete distributions, 273–282 (1994)
Esposito, Y., Lemay, A., Denis, F., Dupont, P.: Learning probabilistic residual finite state automata. In: Adriaans, P.W., Fernau, H., van Zaanen, M. (eds.) ICGI 2002. LNCS (LNAI), vol. 2484, p. 77. Springer, Heidelberg (2002)
Denis, F., Esposito, Y.: Residual languages and probabilistic automata. In: ICALP 2003, Springer, Heidelberg (2003)
Angluin, D.: Queries and concept learning. Machine Learning 2, 319–342 (1988)
Vapnik, V.N.: Statistical Learning Theory. John Wiley, Chichester (1998)
Lugosi, G.: Pattern classification and learning theory. In: Principles of Nonparametric Learning., pp. 1–56. Springer, Heidelberg (2002)
Hardy, G.H., Wright, E.M.: An introduction to the theory of numbers. Oxford University Press, Oxford (1979)
Blondel, V.D., Canterini, V.: Undecidable problems for probabilistic automata of fixed dimension. Theory of Computing Systems 36, 231–245 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Denis, F., Esposito, Y. (2004). Learning Classes of Probabilistic Automata. In: Shawe-Taylor, J., Singer, Y. (eds) Learning Theory. COLT 2004. Lecture Notes in Computer Science(), vol 3120. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-27819-1_9
Download citation
DOI: https://doi.org/10.1007/978-3-540-27819-1_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22282-8
Online ISBN: 978-3-540-27819-1
eBook Packages: Springer Book Archive