Automated multimedia surveillance systems capture, process and analyze multimedia data coming from heterogeneous sensors. These systems are often designed to support (semi-) automatic decision making, such as generating an alarm in response to a surveillance event, as well as providing useful information to human decision makers to ensure public safety. Various tools and techniques from different fields such as Computer Vision, Pattern Recognition, and Multimedia Computing have contributed to the success of such systems.

Although there has been significant progress in the field of multimedia surveillance research, we still face situations when the system is unable to detect critical events, wrongly identifies individuals or generates false alarms leading to undesired consequences. Hence, the goal of this special issue is to bring forward recent advancements in automated multimedia surveillance for improved public safety. More specifically, it reports the state-of-the-art techniques, methodologies and systems for automatic surveillance aiming to protect people and ensure their safety.

An open call for papers attracted 16 submissions addressing different aspects of multimedia surveillance systems. After a thorough and highly competitive review process, the guest editorial committee recommended the acceptance of 11 top quality papers, though most submissions were of high quality. The accepted papers are divided into four categories based on the aspects they address. These include automatic event detection based on multi-sensor data, surveillance system design and development, quality of surveillance systems and information, and crowd-supported surveillance.

The first category includes five papers that focused on automatic event detection based on multi-sensor data. The first paper in this category, “Distributed wide-area multi-object tracking with non-overlapping camera views” by Youlu Wang, Senem Velipasalar and Mustafa Cenk Gursoy addresses the problem of tracking objects across multiple non-overlapping cameras. In this paper, multiple cues, such as appearance, texture, latency (the time between exiting the field of view of one camera and enter that of the other camera), object size, and the aspect ratio of the object’s bounding boxes are combined to assign consistent labels to objects as they move through the set of cameras. The authors proposed making a decision at each camera via a probabilistic Petri Net and showed the utility of the proposed approach for tracking vehicles and people across multiple non-overlapping cameras. The second paper, “The large-scale crowd analysis based on sparse spatial-temporal local binary pattern” by Hua Yang, Yihua Cao, Hang Su, Yawen Fan and Shibao Zheng presents a framework for crowd density calculation based on a spatial-temporal local binary pattern (SST-LBP) descriptor. A set of locations is selected in the image to derive the LBP descriptor, which is used to estimate the crowd density. A perspective correction method is proposed to reduce the error due to the variation in a person’s height with respect to distance from the camera. The proposed method has the advantage of low computation complexity and high efficiency compared to other methods. The third paper in this group, “Human segmentation by geometrically fusing visible-light and thermal imaginary” by Jian Zhao and Sen-ching S. Cheung describes a fusion algorithm for visible-light and infrared cameras to perform human segmentation. The visible and thermal cameras act as a short baseline stereo pair, so by necessity the two sets of images need to be registered with each other before fusing the information and making the pixel-level decisions. The authors provided experimental results to show that the proposed system offers significant improvements over existing schemes under various conditions. The fourth paper in this group “Person re-identification by fuzzy space color histogram” by Zong Jie Xiang, Qiren Chen and Yuncai Liu introduces a fuzzy space color histogram-based person re-identification algorithm for tracking across multiple cameras. The authors propose a new feature, the Fuzzy Space Color Histogram, and integrate this feature with Fuzzy foreground to solve the person re-identification problem. The application of the feature is demonstrated in two scenarios: with full gallery and without full gallery. The experiments conducted with three different datasets show promising results. The fifth and final paper in this category, “ES-RU: an entropy based rule to select representative templates in face surveillance” by Maria De Marsico, and Michele Nappi and Daniel Riccio presents a system for the identification of individuals in video surveillance data. Specifically, the aim of the system is to identify individuals within different segments of the same video, as well as across different videos. A system architecture is designed to distribute the different processing steps across dedicated modules that interact with each other to accomplish the final task. The proposed system was tested on six video clips and on a subset of the SCFace database to assess its performance.

The second category of papers focused on solving diverse issues related to surveillance system design and development. The first paper in this group, “Optimal placement of multiple visual sensors considering space coverage and cost constraints” by Yunyoung Nam and Sangjin Hong presents a method for optimal camera placement by considering setup cost and task-specific constraints (e.g. viewing distance, angular range, accessible areas). The camera placement method uses an agent which is modeled and implemented using the A* algorithm to estimate the trajectories of moving people. The path-finding algorithm is improved to minimize the difference between a path actually traveled by people and a path selected by an agent. The authors showed that the proposed approach performed better than other methods including the centralized force-based approach and landmark-based approach. The second paper in this category, “An energy-efficient storage for video surveillance” by Sun Zhizhuo, Tan Yu-An and Li Yuanzhang proposes an energy-efficient data layout, called ‘Semi-RAID’, for video surveillance. The proposed data layout is comprehensively described and the performance is measured and compared to some other RAID methods in a simulated video surveillance experiment. The authors showed significant energy saving through the proposed system design.

In the third category, there are three papers which address the quality-related issues and applications in surveillance systems. The first paper in this group, “QoS-aware service composition for ubiquitous video surveillance” by M. Shamim Hossain describes a surveillance video delivery system that allows different services such as transcoding, capturing, and viewing to be composed based on QoS for heterogeneous users. The author used ant-based algorithm to select the best service composition path. The second paper, “Offline quality monitoring for legal evidence images in video-surveillance applications” by Aldo Maalouf, Mohamed-Chaker Larabi and Didier Nicholson focuses on offline quality monitoring in video surveillance when selecting images. It has three innovative parts: robust tracking using a foveal wavelet with mean shift, a no-reference quality metric based on sharpness, and a super-resolution algorithm. The combination of these three algorithms helps improve the efficiency of the extraction of the most suitable legal evidence images. The next paper in this group is “Utility based decision support engine for camera view selection in multimedia surveillance systems” by Dewan Tanvir Ahmed, M. Anwar Hossain, Shervin Shirmohammadi, Abdullah AlGhamdi, Pradeep K. Atrey and Abdulmotaleb El Saddik. This paper presents a decision support algorithm to provide automatic aid to the surveillance operators. The proposed algorithm selects the most appropriate cameras having critical information to be viewed at the current time. The importance of a view is calculated based on a utility model defined with respect to events that occur in the monitored environment.

The fourth group consists of one paper: “A Target-centric surveillance system based on localization and social networking” by Jinyoung Han, Nakjung Choi, Taejoong Chung, Ted Taekyoung Kwon and Yanghee Choi. It aims to create a target-centric surveillance system combining localization information and social networks. The idea of using social networks and geo-spatial information is very challenging and very interesting. This paper describes a video diary-based surveillance system and essentially tries to combine video diary (life-casting to social networks) and surveillance functionalities.

Many people deserve thanks for making this special issue a success. First of all, we thank all of the authors for submitting their quality work. Next, a word of thanks goes to the numerous reviewers for their hard work and expert comments that proved critical in maintaining the journal high standards. Special thanks go to Prof. Borko Furht, Editor-in-Chief, for his invaluable guidance, and to the editorial staff for their support throughout the process of this special issue.