tutorial

Attribute-based People Search: Lessons Learnt from a Practical Surveillance System

Authors:

Russel Bobbitt,

Sharath PankantiAuthors Info & Claims

ICMR '14: Proceedings of International Conference on Multimedia Retrieval

Pages 153 - 160

https://doi.org/10.1145/2578726.2578732

Published: 01 April 2014 Publication History

Abstract

We address the problem of attribute-based people search in real surveillance environments. The system we developed is capable of answering user queries such as "show me all people with a beard and sunglasses, wearing a white hat and a patterned blue shirt, from all metro cameras in the downtown area, from 2pm to 4pm last Saturday". In this paper, we describe the lessons we learned from practical deployments of our system, and how we made our algorithms achieve the accuracy and efficiency required by many police departments around the world. In particular, we show that a novel set of multimodal integral filters and proper normalization of attribute scores are critical to obtain good performance. We conduct a comprehensive experimental analysis on video footage captured from a large set of surveillance cameras monitoring metro chokepoints, in both crowded and normal activity periods. Moreover, we show impressive results using images from the recent Boston marathon bombing event, where our system can rapidly retrieve the two suspects based on their attributes from a database containing more than one thousand people present at the event.

References

[1]

R. Benenson, M. Mathias, T. Tuytelaars, and L. V. Gool. Seeking the strongest rigid detector. In CVPR, 2013.

Digital Library

[2]

L. Bourdev, S. Maji, and J. Malik. Recognizing human actions by attributes. In ICCV, 2011.

[3]

H. Chen, A. Gallagher, and B. Girod. Describing clothing by semantic attributes. In ECCV, 2012.

Digital Library

[4]

P. Dollar, Z. Tu, P. Perona, and S. Belongie. Integral channel features. In BMVC, 2009.

[5]

A. Farhadi, I. Endres, D. Hoiem, and D. Forsyth. Describing objects by their attributes. In CVPR, 2009.

[6]

R. S. Feris, T. Campos, and R. Cesar. Detection and tracking of facial features in video sequences. In MICAI, 2001.

Digital Library

[7]

J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: a statistical view of boosting. Annals of Statistics, 38(2):337--374, 2000.

[8]

J. Klontz and A. Jain. A case study of automated face recognition: The boston marathon bombings suspects. In IEEE Computer, 2013.

Digital Library

[9]

A. Kovashka, D. Parikh, and K. Grauman. Whittlesearch: Image search with relative attribute feedback. In CVPR, 2012.

Digital Library

[10]

G. Kulkarni, V. Premraj, S. Dhar, S. Li, Y. Choi, A. Berg, and T. Berg. Baby talk: Understanding and generating simple image descriptions. In CVPR, 2011.

Digital Library

[11]

N. Kumar, P. Belhumeur, and S. Nayar. Facetracer: A search engine for large collections of images with faces. In ECCV, 2008.

Digital Library

[12]

N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar. Attribute and simile classifiers for face verification. In ICCV, 2009.

[13]

C. H. Lampert, H. Nickisch, and S. Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In CVPR, 2009.

[14]

R. Layne, T. Hospedales, and S. Gong. Person re-identification by attributes. In BMVC, 2012.

[15]

R. Lienhart, A. Kuranov, and V. Pisarevsky. Empirical analysis of detection cascades of boosted classifiers for rapid object detection. In DAGM 25th Pattern Recognition Symposium, 2003.

[16]

J. Liu, B. Kuipers, and S. Savarese. Recognizing human actions by attributes. In CVPR, 2011.

Digital Library

[17]

D. Parikh and K. Grauman. Relative attributes. In ICCV, 2011.

Digital Library

[18]

W. Scheirer, N. Kumar, P. Belhumeur, and T. Boult. Multi-attribute spaces: Calibration for attribute fusion and similarity search. In CVPR, 2012.

[19]

B. Siddiquie, R. S. Feris, and L. Davis. Image ranking and retrieval based on multi-attribute queries. In CVPR, 2011.

Digital Library

[20]

J. Sivic and A. Zisserman. Video google: A text retrieval approach to object matching in videos. In ICCV, 2003.

Digital Library

[21]

C. Stauffer and W. Grimson. Adaptive background mixture models for real-time tracking. In CVPR, 1998.

[22]

J. Terrillon and S. Akamatsu. Comparative performance of different chrominance spaces for color segmentation and detection of human faces in complex scene images. In Face and Gesture Recognition, 2000.

[23]

Y. Tian, M. Lu, and A. Hampapur. Robust and efficient foreground analysis for real-time video surveillance. In CVPR, 2005.

Digital Library

[24]

D. A. Vaquero, R. S. Feris, D. Tran, L. Brown, and A. Hampapur. Attribute-based people search in surveillance environments. In WACV, 2009.

[25]

P. Viola and M. Jones. Robust Real-time Object Detection. In International Journal of Computer Vision, 2001.

Digital Library

[26]

C. Wah, S. Branson, P. Perona, and S. Belongie. Multiclass recognition and part localization with humans in the loop. In ICCV, 2011.

Digital Library

[27]

L. Wolf, T. Hassner, and I. Maoz. Face recognition in unconstrained videos with matched background similarity. In CVPR, 2011.

Digital Library

[28]

L. Wolf and N. Levy. The svm-minus similarity score for video face recognition. In CVPR, 2013.

Digital Library

[29]

H. Wu, R. Luk, K. Wong, and K. Kwok. Interpreting tf-idf term weights as making relevance decisions. In ACM Transactions on Information Systems, 2008.

Digital Library

[30]

R. Yan, J. Tesic, and J. Smith. Model-shared subspace boosting for multi-label classification. In KDD, 2007.

Digital Library

[31]

B. Zadrozny and C. Elkan. Transforming classifier scores into accurate multiclass probability estimates. In KDD, 2002.

Digital Library

[32]

L. Zheng, S. Wang, Z. Liu, and Q. Tian. Lp-norm idf for large scale image search. In CVPR, 2013.

Digital Library

Cited By

Lu WHu HYu JZhou YWang HLi B(2024)Orientation-Aware Pedestrian Attribute Recognition Based on Graph Convolution NetworkIEEE Transactions on Multimedia10.1109/TMM.2023.325968626(28-40)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3259686
Xiao HXie WZhou YLuo YZhang RXu X(2024)Research on Multi-scale Pedestrian Attribute Recognition Based on Dual Self-attention MechanismMobile Networks and Management10.1007/978-3-031-55471-1_16(215-226)Online publication date: 17-Mar-2024
https://doi.org/10.1007/978-3-031-55471-1_16
Wu JHuang YGao MGao ZZhao JShi JZhang A(2023)Exponential Information Bottleneck Theory Against Intra-Attribute Variations for Pedestrian Attribute RecognitionIEEE Transactions on Information Forensics and Security10.1109/TIFS.2023.331158418(5623-5635)Online publication date: 2023
https://doi.org/10.1109/TIFS.2023.3311584
Show More Cited By

Index Terms

Attribute-based People Search: Lessons Learnt from a Practical Surveillance System
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Image manipulation
      1. Image processing

Recommendations

RGB-D based multi-attribute people search in intelligent visual surveillance
MMM'12: Proceedings of the 18th international conference on Advances in Multimedia Modeling

Searching people in surveillance videos is a typical task in intelligent visual surveillance (IVS). However, current IVS techniques can hardly handle multi-attribute queries, which is a natural way of finding people in real-world. The challenges arise ...
People search based on attributes description provided by an eyewitness for video surveillance applications

People search based on attributes description presents a paramount task for several forensics and surveillance applications. The aim is to locate a suspect or to find a missing person in public areas. However, semantic attributes provide a natural ...
Multilingual people search
SIGIR '10: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval

People Search is an important search service with multiple applications (eg. looking up a friend on Facebook, finding colleagues in corporate email directories etc). With the proportion of non-English users on a steady rise, people search services are ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICMR '14: Proceedings of International Conference on Multimedia Retrieval

April 2014

564 pages

ISBN:9781450327824

DOI:10.1145/2578726

Conference Chairs:
Mohan Kankanhalli
National University of Singapore
,
Stefan Rueger
The Open University, UK
,
R. Manmatha
A9.com, USA
,
General Chairs:
Joemon Jose
University of Glasgow, UK
,
Keith van Rijsbergen
University of Glasgow, UK

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 April 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Tutorial
Research
Refereed limited

Conference

ICMR '14

ICMR '14: International Conference on Multimedia Retrieval

April 1 - 4, 2014

Glasgow, United Kingdom

Acceptance Rates

ICMR '14 Paper Acceptance Rate 21 of 111 submissions, 19%;

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

54
Total Citations
View Citations
325
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lu WHu HYu JZhou YWang HLi B(2024)Orientation-Aware Pedestrian Attribute Recognition Based on Graph Convolution NetworkIEEE Transactions on Multimedia10.1109/TMM.2023.325968626(28-40)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3259686
Xiao HXie WZhou YLuo YZhang RXu X(2024)Research on Multi-scale Pedestrian Attribute Recognition Based on Dual Self-attention MechanismMobile Networks and Management10.1007/978-3-031-55471-1_16(215-226)Online publication date: 17-Mar-2024
https://doi.org/10.1007/978-3-031-55471-1_16
Wu JHuang YGao MGao ZZhao JShi JZhang A(2023)Exponential Information Bottleneck Theory Against Intra-Attribute Variations for Pedestrian Attribute RecognitionIEEE Transactions on Information Forensics and Security10.1109/TIFS.2023.331158418(5623-5635)Online publication date: 2023
https://doi.org/10.1109/TIFS.2023.3311584
Li DZhang ZShan CWang L(2023)Incremental Pedestrian Attribute Recognition via Dual Uncertainty-Aware Pseudo-LabelingIEEE Transactions on Information Forensics and Security10.1109/TIFS.2023.326888718(2622-2636)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TIFS.2023.3268887
Cardoz FInfenda LCleetus DDeshpande AMore J(2023)Visual Sentinel: Data Analytics for Missing Subject Identification2023 IEEE Pune Section International Conference (PuneCon)10.1109/PuneCon58714.2023.10450061(1-6)Online publication date: 14-Dec-2023
https://doi.org/10.1109/PuneCon58714.2023.10450061
Zhao YJin HShi XSun H(2023)Temporal Related Attention for Video-Based Pedestrian Attribute Recognition2023 18th International Conference on Intelligent Systems and Knowledge Engineering (ISKE)10.1109/ISKE60036.2023.10481446(93-97)Online publication date: 17-Nov-2023
https://doi.org/10.1109/ISKE60036.2023.10481446
Raghavendra SRamyashree Abhilash SNookala VKaliraj S(2023)Efficient Deep Learning Approach to Recognize Person Attributes by Using Hybrid Transformers for Surveillance ScenariosIEEE Access10.1109/ACCESS.2023.324133411(10881-10893)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3241334
Ridzuan SOmar ZSheikh U(2023)Pedestrian attribute recognition: Upper body clothing classification5TH INTERNATIONAL CONFERENCE ON ELECTRICAL, ELECTRONIC, COMMUNICATION AND CONTROL ENGINEERING (ICEECC 2021)10.1063/5.0121371(050003)Online publication date: 2023
https://doi.org/10.1063/5.0121371
Cao YFang YZhang YHou XZhang KHuang W(2023)A novel self-boosting dual-branch model for pedestrian attribute recognitionSignal Processing: Image Communication10.1016/j.image.2023.116961115(116961)Online publication date: Jul-2023
https://doi.org/10.1016/j.image.2023.116961
Zheng AWang HWang JHuang HHe RHussain A(2023)Diverse features discovery transformer for pedestrian attribute recognitionEngineering Applications of Artificial Intelligence10.1016/j.engappai.2022.105708119(105708)Online publication date: Mar-2023
https://doi.org/10.1016/j.engappai.2022.105708
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents