Sequence Modeling via Segmentations

Chong Wang, Yining Wang, Po-Sen Huang, Abdelrahman Mohamed, Dengyong Zhou, Li Deng
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:3674-3683, 2017.

Abstract

Segmental structure is a common pattern in many types of sequences such as phrases in human languages. In this paper, we present a probabilistic model for sequences via their segmentations. The probability of a segmented sequence is calculated as the product of the probabilities of all its segments, where each segment is modeled using existing tools such as recurrent neural networks. Since the segmentation of a sequence is usually unknown in advance, we sum over all valid segmentations to obtain the final probability for the sequence. An efficient dynamic programming algorithm is developed for forward and backward computations without resorting to any approximation. We demonstrate our approach on text segmentation and speech recognition tasks. In addition to quantitative results, we also show that our approach can discover meaningful segments in their respective application contexts.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-wang17j, title = {Sequence Modeling via Segmentations}, author = {Chong Wang and Yining Wang and Po-Sen Huang and Abdelrahman Mohamed and Dengyong Zhou and Li Deng}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {3674--3683}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/wang17j/wang17j.pdf}, url = {https://proceedings.mlr.press/v70/wang17j.html}, abstract = {Segmental structure is a common pattern in many types of sequences such as phrases in human languages. In this paper, we present a probabilistic model for sequences via their segmentations. The probability of a segmented sequence is calculated as the product of the probabilities of all its segments, where each segment is modeled using existing tools such as recurrent neural networks. Since the segmentation of a sequence is usually unknown in advance, we sum over all valid segmentations to obtain the final probability for the sequence. An efficient dynamic programming algorithm is developed for forward and backward computations without resorting to any approximation. We demonstrate our approach on text segmentation and speech recognition tasks. In addition to quantitative results, we also show that our approach can discover meaningful segments in their respective application contexts.} }
Endnote
%0 Conference Paper %T Sequence Modeling via Segmentations %A Chong Wang %A Yining Wang %A Po-Sen Huang %A Abdelrahman Mohamed %A Dengyong Zhou %A Li Deng %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-wang17j %I PMLR %P 3674--3683 %U https://proceedings.mlr.press/v70/wang17j.html %V 70 %X Segmental structure is a common pattern in many types of sequences such as phrases in human languages. In this paper, we present a probabilistic model for sequences via their segmentations. The probability of a segmented sequence is calculated as the product of the probabilities of all its segments, where each segment is modeled using existing tools such as recurrent neural networks. Since the segmentation of a sequence is usually unknown in advance, we sum over all valid segmentations to obtain the final probability for the sequence. An efficient dynamic programming algorithm is developed for forward and backward computations without resorting to any approximation. We demonstrate our approach on text segmentation and speech recognition tasks. In addition to quantitative results, we also show that our approach can discover meaningful segments in their respective application contexts.
APA
Wang, C., Wang, Y., Huang, P., Mohamed, A., Zhou, D. & Deng, L.. (2017). Sequence Modeling via Segmentations. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:3674-3683 Available from https://proceedings.mlr.press/v70/wang17j.html.

Related Material