Article

Affective content detection using HMMs

Author:

Hang-Bong KangAuthors Info & Claims

MULTIMEDIA '03: Proceedings of the eleventh ACM international conference on Multimedia

Pages 259 - 262

https://doi.org/10.1145/957013.957066

Published: 02 November 2003 Publication History

Abstract

This paper discusses a new technique for detecting affective events using Hidden Markov Models(HMM). To map low level features of video data to high level emotional events, we perform empirical study on the relationship between emotional events and low-level features. After that, we compute simple low-level features that represent emotional characteristics and construct a token or observation vector by combining low level features. The observation vector sequence is tested to detect emotional events through HMMs. We create two HMM topologies and test both topologies. The affective events are detected from our proposed models with good accuracy.

References

[1]

Hanjalic, A. Video and Image Retrieval beyond the Cognitive Level: The Needs and Possibilities. Proc. SPIE Storage and Retrieval for Media Databases 2001, San Jose, CA, pp.130--140, 2001.

[2]

Hanjalic A. and Xu, L. User-oriented Affective Video Content Analysis, Proc. IEEE Workshop on CBAIBL'01, Kauai, HI, pp.50--57, Dec., 2001.

Digital Library

[3]

Moncrieff, S.,Dorai, C. and Venkatesh, S.: Affect Computing in Film through Sound Energy Dynamics, Proc. ACM MM'01, pp. 525--527, 2001.

Digital Library

[4]

Lang, P.: The emotion probe: Studies of motivation and attention, American Psychologist, 50(5), pp. 372--385, 1995.

[5]

Picard, R. Affective Computing. MIT Press, 1997

Digital Library

[6]

Valdez, P. and Mehrabian, A.: Effects of color on emotions, Journal of Experimental Psychology: General, pp. 394--409, 1994.

[7]

Scheirer, J. and Picard, R.: Affective Objects, MIT Media lab Technical Rep. No 524.

[8]

Goldstein, E. : Sensation and Perception, Brooks/Cole, 1999.

[9]

Lee, S., and Hayes, M.: Real-time camera motion classification for content-based indexing and retrieval using templates, Proc. ICASSP, pp.3664--3667, 2002.

[10]

Rabiner, L. and Juang, B.: Fundamentals of Speech Recognition, Prentice Hall PTR, 1993.

Digital Library

[11]

Boreczky, J. and Wilcox, E.: A Hidden Markov Model Framework for Video Segmentation Using Audio and Image Features, Proc. ICASSP' 98, 1998.

[12]

Eickeler, S. and Muller, S.: Content-based Video Indexing of TV Broadcast News Using Hidden Markov Models, Proc. ICASSP'99, 1999.

Digital Library

[13]

Li, B. and Sezan, M.: Event Detection and Summarization in Sports Video, Proc. IEEE CBAIBL'01, Kauai, HI, 2001.

Digital Library

[14]

Naphade, M., Garg A. and Huang, T.: Audio-Visual Event Detection using Duration dependent input output Markov models, Proc. IEEE CBAIBL'01, Kauai, HI, 2001.

Digital Library

[15]

Zhang, H., Wu, J., Zhong, D., and Smoliar, S.: An integrated system for content-based video retrieval and browsing, " Pattern Recognition, Vol. 30, pp.643--58, 1997.

Cited By

Luo YZhong XZeng MXie JWang SLiu G(2024)CGLF-Net: Image Emotion Recognition Network by Combining Global Self-Attention Features and Local Multiscale FeaturesIEEE Transactions on Multimedia10.1109/TMM.2023.328976226(1894-1908)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3289762
Fu KDu CWang SHe H(2024)Improved Video Emotion Recognition With Alignment of CNN and Human Brain RepresentationsIEEE Transactions on Affective Computing10.1109/TAFFC.2023.331617315:3(1026-1040)Online publication date: Jul-2024
https://doi.org/10.1109/TAFFC.2023.3316173
Shaiok LHoque IHasan MGhosh SGedeon THossain M(2024)Attention-Based Multi-layer Perceptron to Categorize Affective Videos from Viewer’s Physiological SignalsRecent Challenges in Intelligent Information and Database Systems10.1007/978-981-97-5934-7_3(25-34)Online publication date: 13-Aug-2024
https://doi.org/10.1007/978-981-97-5934-7_3
Show More Cited By

Index Terms

Affective content detection using HMMs
1. Information systems
  1. Information retrieval

Recommendations

Hierarchical movie affective content analysis based on arousal and valence features
MM '08: Proceedings of the 16th ACM international conference on Multimedia

Emotional factors directly reflect audiences' attention, evaluation and memory. Affective contents analysis not only create an index for users to access their interested movie segments, but also provide feasible entry for video highlights. Most of the ...
Towards the detection of unusual temporal events during activities using HMMs
UbiComp '12: Proceedings of the 2012 ACM Conference on Ubiquitous Computing

Most of the systems for recognition of activities aim to identify a set of normal human activities. Data is either recorded by computer vision or sensor based networks. These systems may not work properly if an unusual event or abnormal activity occurs, ...
Gender-dependent emotion recognition based on HMMs and SPHMMs

It is well known that emotion recognition performance is not ideal. The work of this research is devoted to improving emotion recognition performance by employing a two-stage recognizer that combines and integrates gender recognizer and emotion ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MULTIMEDIA '03: Proceedings of the eleventh ACM international conference on Multimedia

November 2003

670 pages

ISBN:1581137222

DOI:10.1145/957013

General Chairs:
Lawrence Rowe
University of California, Berkeley
,
Harrick Vin
University of Texas, Austin
,
Program Chairs:
Thomas Plagemann
University of Oslo
,
Prashant Shenoy
University of Massachusetts, Amherst
,
John R. Smith
IBM T.J. Watson Research Center

Copyright © 2003 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MM03

Sponsor:

MM03: 2003 11th Annual ACM International Conference on Multimedia

November 2 - 8, 2003

CA, Berkeley, USA

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

86
Total Citations
View Citations
765
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Luo YZhong XZeng MXie JWang SLiu G(2024)CGLF-Net: Image Emotion Recognition Network by Combining Global Self-Attention Features and Local Multiscale FeaturesIEEE Transactions on Multimedia10.1109/TMM.2023.328976226(1894-1908)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3289762
Fu KDu CWang SHe H(2024)Improved Video Emotion Recognition With Alignment of CNN and Human Brain RepresentationsIEEE Transactions on Affective Computing10.1109/TAFFC.2023.331617315:3(1026-1040)Online publication date: Jul-2024
https://doi.org/10.1109/TAFFC.2023.3316173
Shaiok LHoque IHasan MGhosh SGedeon THossain M(2024)Attention-Based Multi-layer Perceptron to Categorize Affective Videos from Viewer’s Physiological SignalsRecent Challenges in Intelligent Information and Database Systems10.1007/978-981-97-5934-7_3(25-34)Online publication date: 13-Aug-2024
https://doi.org/10.1007/978-981-97-5934-7_3
Kerr WHaines TLi W(2023)Unsupervised Scouting and Layout for Storyboarding in Movie Pre-productionProceedings of the 2023 ACM International Conference on Interactive Media Experiences Workshops10.1145/3604321.3604372(86-93)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3604321.3604372
Zhang HXu M(2023)Recognition of Emotions in User-Generated Videos through Frame-Level Adaptation and Emotion Intensity LearningIEEE Transactions on Multimedia10.1109/TMM.2021.313416725(881-891)Online publication date: 2023
https://doi.org/10.1109/TMM.2021.3134167
Muszynski MMorgenroth EVilaclara LVan De Ville DVuilleumier P(2022)Impact of aesthetic movie highlights on semantics and emotions: a preliminary analysisCompanion Publication of the 2022 International Conference on Multimodal Interaction10.1145/3536220.3558544(52-60)Online publication date: 7-Nov-2022
https://dl.acm.org/doi/10.1145/3536220.3558544
Meyer HWei PJiang X(2021)Intelligent Video Highlights Generation with Front-Camera Emotion SensingSensors10.3390/s2104103521:4(1035)Online publication date: 3-Feb-2021
https://doi.org/10.3390/s21041035
Liang YMaeda KOgawa THaseyama M(2021)Deep Metric Network Via Heterogeneous Semantics for Image Sentiment Analysis2021 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP42928.2021.9506701(1039-1043)Online publication date: 19-Sep-2021
https://doi.org/10.1109/ICIP42928.2021.9506701
Liang YMaeda KOgawa THaseyama M(2021)Cross-Domain Semi-Supervised Deep Metric Learning for Image Sentiment AnalysisICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP39728.2021.9414150(4150-4154)Online publication date: 6-Jun-2021
https://doi.org/10.1109/ICASSP39728.2021.9414150
Mu HYong-en YChao-liang L(2020)Affective Classification Method Based on Movie 5.1 SoundProceedings of the 2020 4th International Conference on Digital Signal Processing10.1145/3408127.3408148(255-258)Online publication date: 19-Jun-2020
https://dl.acm.org/doi/10.1145/3408127.3408148
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents