Multimodal visual pattern mining with convolutional neural networks
H Li - Proceedings of the 2016 ACM on International …, 2016 - dl.acm.org
Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval, 2016•dl.acm.org
In this paper we describe a novel framework and algorithms for discovering image patch
patterns from a large corpus of weakly supervised image-caption pairs generated from news
events. Current pattern mining techniques attempt to find patterns that are representative
and discriminative, we stipulate that our discovered patterns must also be recognizable by
humans and preferably with meaningful names. We propose a new multimodal pattern
mining approach that leverages the descriptive captions often accompanying news images …
patterns from a large corpus of weakly supervised image-caption pairs generated from news
events. Current pattern mining techniques attempt to find patterns that are representative
and discriminative, we stipulate that our discovered patterns must also be recognizable by
humans and preferably with meaningful names. We propose a new multimodal pattern
mining approach that leverages the descriptive captions often accompanying news images …
In this paper we describe a novel framework and algorithms for discovering image patch patterns from a large corpus of weakly supervised image-caption pairs generated from news events. Current pattern mining techniques attempt to find patterns that are representative and discriminative, we stipulate that our discovered patterns must also be recognizable by humans and preferably with meaningful names. We propose a new multimodal pattern mining approach that leverages the descriptive captions often accompanying news images to learn semantically meaningful image patch patterns. The mutltimodal patterns are then named using words mined from the associated image captions for each pattern. Our methods also discover named patterns beyond those covered by the existing image datasets like ImageNet. To the best of our knowledge this is the first algorithm developed to automatically mine image patch patterns that have strong semantic meaning specific to high-level news events, and then evaluate these patterns based on that criteria.
ACM Digital Library