research-article

An HDP-HMM for systems with state persistence

Authors:

Emily B. Fox,

Erik B. Sudderth,

Michael I. Jordan,

Alan S. WillskyAuthors Info & Claims

ICML '08: Proceedings of the 25th international conference on Machine learning

Pages 312 - 319

https://doi.org/10.1145/1390156.1390196

Published: 05 July 2008 Publication History

Get Access

Abstract

The hierarchical Dirichlet process hidden Markov model (HDP-HMM) is a flexible, nonparametric model which allows state spaces of unknown size to be learned from data. We demonstrate some limitations of the original HDP-HMM formulation (Teh et al., 2006), and propose a sticky extension which allows more robust learning of smoothly varying dynamics. Using DP mixtures, this formulation also allows learning of more complex, multimodal emission distributions. We further develop a sampling algorithm that employs a truncated approximation of the DP to jointly resample the full state sequence, greatly improving mixing rates. Via extensive experiments with synthetic data and the NIST speaker diarization database, we demonstrate the advantages of our sticky extension, and the utility of the HDP-HMM in real-world applications.

References

[1]

Beal, M. J., Ghahramani, Z., & Rasmussen, C. E. (2002). The infinite hidden Markov model. NIPS (pp. 577--584).

Google Scholar

[2]

Fox, E., Sudderth, E., Jordan, M., & Willsky, A. (2007). A tempered HDP-HMM for systems with state persistence. MIT LIDS, TR #2777.

Digital Library

Google Scholar

[3]

Ishwaran, H., & Zarepour, M. (2002). Exact and approximate sum-representations for the Dirichlet process. Can. J. Stat., 30, 269--283.

Crossref

Google Scholar

[4]

Kivinen, J. J., Sudderth, E. B., & Jordan, M. I. (2007). Learning multiscale representations of natural scenes using Dirichlet processes. ICCV (pp. 1--8).

Google Scholar

[5]

NIST (2007). Rich transcriptions database. http://www.nist.gov/speech/tests/rt/.

Google Scholar

[6]

Rabiner, L. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE, 77, 257--286.

Crossref

Google Scholar

[7]

Rodriguez, A., Dunson, D., & Gelfand, A. (2006). The nested Dirichlet process. Duke ISDS, TR #06--19.

Google Scholar

[8]

Scott, S. (2002). Bayesian methods for hidden Markov models: Recursive computing in the 21st century. J. Amer. Stat. Assoc., 97, 337--351.

Crossref

Google Scholar

[9]

Sethuraman, J. (1994). A constructive definition of Dirichlet priors. Stat. Sinica, 4, 639--650.

Google Scholar

[10]

Teh, Y. W., Jordan, M. I., Beal, M. J., & Blei, D. M. (2006). Hierarchical Dirichlet processes. J. Amer. Stat. Assoc., 101, 1566--1581.

Crossref

Google Scholar

[11]

Wooters, C., & Huijbregts, M. (2007). The ICSI RT07s speaker diarization system. To appear in LNCS.

Google Scholar

[12]

Xing, E., & Sohn, K.-A. (2007). Hidden Markov Dirichlet process: Modeling genetic inference in open ancestral space. Bayes. Analysis, 2, 501--528.

Crossref

Google Scholar

Cited By

View all

Wang YSun PShuai LZhang D(2024)Double Counterfactual Regret Minimization for Generating Safety-Critical Scenario of Autonomous DrivingElectronics10.3390/electronics1321430313:21(4303)Online publication date: 1-Nov-2024
https://doi.org/10.3390/electronics13214303
Gao LLi MJing Z(2024)Multiple maneuvering target tracking based on hierarchical Dirichlet process and hidden Markov modelSignal Processing10.1016/j.sigpro.2023.109344217(109344)Online publication date: Apr-2024
https://doi.org/10.1016/j.sigpro.2023.109344
Huang JMorsomme RDunson DXu J(2024)Detecting changes in the transmission rate of a stochastic epidemic modelStatistics in Medicine10.1002/sim.1005043:10(1867-1882)Online publication date: 26-Feb-2024
https://doi.org/10.1002/sim.10050
Show More Cited By

Index Terms

An HDP-HMM for systems with state persistence

Recommendations

SOS-HMM: self-organizing structure of hidden Markov model
ICANN'11: Proceedings of the 21st international conference on Artificial neural networks - Volume Part II

We propose in this paper a novel approach which makes self-organizing maps (SOM) and the Hidden Markov Models (HMMs) cooperate. Our approach (SOS-HMM: Self Organizing Structure of HMM) allows to learn the Hidden Markov Models topology. The main ...
Speaker state recognition using an HMM-based feature extraction method

In this article we present an efficient approach to modeling the acoustic features for the tasks of recognizing various paralinguistic phenomena. Instead of the standard scheme of adapting the Universal Background Model (UBM), represented by the ...
Learning HMM State Sequences from Phonemes for Speech Synthesis

This paper presents a technique for learning hidden Markov model (HMM) state sequences from phonemes, that combined with modified discrete cosine transform (MDCT), is useful for speech synthesis. Mel-cepstral spectral parameters, currently adopted in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ICML '08: Proceedings of the 25th international conference on Machine learning

July 2008

1310 pages

ISBN:9781605582054

DOI:10.1145/1390156

General Chair:
William Cohen
Carnegie Mellon University
,
Program Chairs:
Andrew McCallum
University of Massachusetts Amherst
,
Sam Roweis
University of Toronto and Google

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Defense Advanced Research Projects Agency

Conference

ICML '08

Sponsor:

Microsoft Research
Intel
IBM

ICML '08: The 25th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

July 5 - 9, 2008

Helsinki, Finland

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

147
Total Citations
View Citations
1,074
Total Downloads

Downloads (Last 12 months)49
Downloads (Last 6 weeks)6

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Wang YSun PShuai LZhang D(2024)Double Counterfactual Regret Minimization for Generating Safety-Critical Scenario of Autonomous DrivingElectronics10.3390/electronics1321430313:21(4303)Online publication date: 1-Nov-2024
https://doi.org/10.3390/electronics13214303
Gao LLi MJing Z(2024)Multiple maneuvering target tracking based on hierarchical Dirichlet process and hidden Markov modelSignal Processing10.1016/j.sigpro.2023.109344217(109344)Online publication date: Apr-2024
https://doi.org/10.1016/j.sigpro.2023.109344
Huang JMorsomme RDunson DXu J(2024)Detecting changes in the transmission rate of a stochastic epidemic modelStatistics in Medicine10.1002/sim.1005043:10(1867-1882)Online publication date: 26-Feb-2024
https://doi.org/10.1002/sim.10050
Lee JChua PChen LNg PKim YWu QJeon SJung JChang SMoon S(2023)Key Enabling Technologies for Smart Factory in Automotive Industry: Status and ApplicationsInternational Journal of Precision Engineering and Manufacturing-Smart Technology10.57062/ijpem-st.2022.00171:1(93-105)Online publication date: 1-Jan-2023
https://doi.org/10.57062/ijpem-st.2022.0017
Gao LWang CWu G(2023)Hidden Semi-Markov Models-Based Visual Perceptual State Recognition for PilotsSensors10.3390/s2314641823:14(6418)Online publication date: 14-Jul-2023
https://doi.org/10.3390/s23146418
Li TMa J(2023)Hidden Markov Mixture of Gaussian Process Functional Regression: Utilizing Multi-Scale Structure for Time Series ForecastingMathematics10.3390/math1105125911:5(1259)Online publication date: 5-Mar-2023
https://doi.org/10.3390/math11051259
许蕊(2023)Research on the Theory and Application of Bayesian Nonparametric Methods in Big DataStatistics and Application10.12677/SA.2023.12203012:02(283-292)Online publication date: 2023
https://doi.org/10.12677/SA.2023.122030
van den Boom WDe Iorio MBeskos A(2023)Bayesian Learning of Graph SubstructuresBayesian Analysis10.1214/22-BA133818:4Online publication date: 1-Dec-2023
https://doi.org/10.1214/22-BA1338
Khadivar FBillard A(2023)Adaptive Fingers Coordination for Robust Grasp and In-Hand Manipulation Under Disturbances and Unknown DynamicsIEEE Transactions on Robotics10.1109/TRO.2023.328002839:5(3350-3367)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1109/TRO.2023.3280028
Ding WXu CArief MLin HLi BZhao D(2023)A Survey on Safety-Critical Driving Scenario Generation—A Methodological PerspectiveIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.325932224:7(6971-6988)Online publication date: Jul-2023
https://doi.org/10.1109/TITS.2023.3259322
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

SOS-HMM: self-organizing structure of hidden Markov model

Speaker state recognition using an HMM-based feature extraction method

Learning HMM State Sequences from Phonemes for Speech Synthesis