Internet Video Search and Repurposing through Face Analysis

With the ever-growing amount of videos on the internet, searching for desired videos in an effective and efficient way remains a challenge. In addition, repurposing videos of interest into new attractive photo/video products has been an open issue. In this paper, we propose a framework for video retrieval and repurposing by leveraging the face information in videos. Since text query cannot express user's intent precisely and it may generate noisy result, an automatic query image generation method is proposed to provide user with visual and intuitive candidate images. Then, based on user-selected query image, videos are re-ranked through content analysis so that videos with higher relevance can emerge to the top. Furthermore, relevant segments and frames are obtained and they can be used to compose customized photo/video products, which provides user with a fresh experience as a creator. In this paper, a prototype of video retrieval and repurposing system is implemented for retrieving videos of celebrities and generating customized products. Experiments have been done on 3416 video clips related to about 103 celebrities downloaded from YouTube. The experimental results show the effectiveness of our proposed method.