research-article

MultiFusion: A boosting approach for multimedia fusion

Authors:

Mohan KankanhalliAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 6, Issue 4

Article No.: 25, Pages 1 - 18

https://doi.org/10.1145/1865106.1865109

Published: 26 November 2010 Publication History

Abstract

The multimodal data usually contain complementary, correlated and redundant information. Thus, multimodal fusion is useful for many multisensor applications. Here, a novel multimodal fusion algorithm is proposed, which is referred to as “MultiFusion.” The approach adopts a boosting structure where the atomic event is considered as the fusion unit. The correlation of multimodal data is used to form an overall classifier in each iteration. Moreover, by adopting the Adaboost-like structure, the overall fusion performance is improved. Both the simulation experiment and the real application show the effectiveness of the MultiFusion approach. Our approach can be applied in different multimodal applications to exploit the multimedia data characteristics and improve the performance.

References

[1]

Aly, R. and Hiemstra, D. 2009. Concept detectors: how good is good enough&quest; In Procedings of the ACM International Conference on Multimedia. 233--242.

Digital Library

[2]

Amores, J., Sebe, N., Radeva, P., Gevers, T., and Smeulders, A. W. M. 2004. Boosting contextual information in content-based image retrieval. In Proceedings of the 6th ACM SIGMM International Workshop on Multimedia Information Retrieval. 31--38.

Digital Library

[3]

Atrey, P. K., Kankanhalli, M. S., and Jain, R. 2006. Information assimilation framework for event detection in multimedia surveillance systems. Multimedia Syst. 12, 3, 239--253.

Digital Library

[4]

Barbu, C., Iqbal, R. T., and Peng, J. 2005. Classifier fusion using shared sampling distribution for boosting. In Proceedings of the International Conference on Data Mining. 34--41.

Digital Library

[5]

Contreras, F. M., Urunuela, C. O., and del Rincon, J. M. 2009. Adaboost multiple feature selection and combination for face recognition. In Proceedings of the Iberian Conference on Pattern Recognition and Image Analysis. 338--345.

Digital Library

[6]

Dasarathy, B. V. 1994. Decision Fusion. Computer Society Press.

Digital Library

[7]

Freund and Schapire. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. J. Compu. Syst. Sci. 55.

Digital Library

[8]

Freund, Y. and Schapire, R. E. 1996. Experiments with a new boosting algorithm. In Proceedings of the International Conference on Machine Learning. 148--156.

[9]

Guo, G. and Zhang, H. 2001. Boosting for fast face recognition. Tech. rep. MSR-TR-2001-16, Microsoft Research.

[10]

Guo, W., Qi, Q., Jiang, L., and Zhang, P. 2008. A new method of sar image target recognition based on adaboost algorithm. In Proceedings of the IEEE International Geoscience and Remote Sensing Symposium.

[11]

Iyengar, G., Nock, H., and Neti, C. 2003. Audio-visual synchrony for detection of monologues in video archives. In Proceedings of the IEEE International Conference on Multimedia and Expo 1, 329--332.

Digital Library

[12]

Joshi, D., Naphade, M. R., and Natsev, A. 2007. Semantics reinforcement and fusion learning for multimedia streams. In Proceedings of the International Conference on Image and Video Retrieval. 309--316.

Digital Library

[13]

Maghooli, K. and Moin, M. S. 2004. A new approach on multimodal biometrics based on combining neural networks using adaboost. In Biometric Authentication. Lecture Notes in Computer Science Vol. 3087 332--341.

[14]

Parikh, D., Kim, M. T., Oagaro, J., Mandayam, S., and Polikar, R. 2004. Combining classifiers for multisensor data fusion. In Proceedings of the International Conference on Systems, Man and Cybernetics. 1232--1237.

[15]

Pickering, M. J., Ruger, S. M., and Sinclair, D. 2002. Video retrieval by feature learning in key frames. In Proceedings of the International Conference on Image and Video Retrieval. 309--317.

Digital Library

[16]

Polikar, R. 2006. Ensemble based systems in decision making. IEEE Circu. Syst. Maga. 6, 3, 21--45.

[17]

Polikar, R., Upda, L., Upda, S. S., and Honavar, V. 2001. Learn++: an incremental learning algorithm for supervised neural networks. IEEE Trans. Syst. Man, Cyber. Part C 31, 4, 497--508.

Digital Library

[18]

Schapire, R. E. 1990. The strength of weak learnability. Mach. Learn. 5, 2.

Digital Library

[19]

Tieu, K. and Viola, P. 2004. Boosting image retrieval. Int. J. Comput. Vis. 56, 1--2, 17--36.

Digital Library

[20]

Xue, F. and Ding, X. 2006. 3D+2D face localization using boosting in multi-modal feature space. In Proceedings of the International Conference on Pattern Recognition. III: 499--502.

Digital Library

[21]

Zhang, L., Li, M. J., and Zhang, H. J. 2002. Boosting image orientation detection with indoor vs. outdoor classification. In Proceedings of the IEEE Workshop on Applications of Computer Vision. 95--99.

Digital Library

[22]

Zhang, T. and Yu, B. 2005. Boosting with early stopping: Convergence and consistency. Ann. Stat. 33, 1538--1579.

Cited By

Jebali MDakhli AJemni M(2021)Vision-based continuous sign language recognition using multimodal sensor fusionEvolving Systems10.1007/s12530-020-09365-yOnline publication date: 22-Jan-2021
https://doi.org/10.1007/s12530-020-09365-y
Kumar PRoy PDogra D(2019)Independent Bayesian classifier combination based sign language recognition using facial expressionInformation Sciences: an International Journal10.1016/j.ins.2017.10.046428:C(30-48)Online publication date: 6-Jan-2019
https://dl.acm.org/doi/10.1016/j.ins.2017.10.046
Pushpalatha K Ananthanarayana V S (2016)Feature pattern based representation of multimedia documents for efficient knowledge discoveryMultimedia Tools and Applications10.1007/s11042-016-3434-y75:15(9461-9487)Online publication date: 1-Aug-2016
https://dl.acm.org/doi/10.1007/s11042-016-3434-y
Show More Cited By

Index Terms

MultiFusion: A boosting approach for multimedia fusion
1. Computing methodologies
  1. Machine learning
2. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

On Comparing Early and Late Fusion Methods
Advances in Computational Intelligence
Abstract
This paper presents a theoretical comparison of early and late fusion methods. An initial discussion on the conditions to apply early or late (soft or hard) fusion is introduced. The analysis show that, if large training sets are available, early ...
A multilevel information fusion approach for visual quality inspection

For visual quality inspection systems to be applicable in industrial settings, it is mandatory that they are highly flexible, robust and accurate. In order to improve these characteristics a multilevel information fusion approach is presented. A first ...
Estimation and decision fusion: A survey

Data fusion has been applied to a large number of fields and the corresponding applications utilize numerous mathematical tools. This survey focuses on some aspects of estimation and decision fusion. In estimation fusion, we discuss the development of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 6, Issue 4

November 2010

159 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/1865106

Issue’s Table of Contents

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 November 2010

Accepted: 01 June 2010

Revised: 01 May 2010

Received: 01 January 2010

Published in TOMM Volume 6, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
481
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jebali MDakhli AJemni M(2021)Vision-based continuous sign language recognition using multimodal sensor fusionEvolving Systems10.1007/s12530-020-09365-yOnline publication date: 22-Jan-2021
https://doi.org/10.1007/s12530-020-09365-y
Kumar PRoy PDogra D(2019)Independent Bayesian classifier combination based sign language recognition using facial expressionInformation Sciences: an International Journal10.1016/j.ins.2017.10.046428:C(30-48)Online publication date: 6-Jan-2019
https://dl.acm.org/doi/10.1016/j.ins.2017.10.046
Pushpalatha K Ananthanarayana V S (2016)Feature pattern based representation of multimedia documents for efficient knowledge discoveryMultimedia Tools and Applications10.1007/s11042-016-3434-y75:15(9461-9487)Online publication date: 1-Aug-2016
https://dl.acm.org/doi/10.1007/s11042-016-3434-y
Mou LHuang TTian YJiang MGao W(2013)Content-based copy detection through multimodal feature representation and temporal pyramid matchingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/2542205.254220810:1(1-20)Online publication date: 27-Dec-2013
https://dl.acm.org/doi/10.1145/2542205.2542208

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents