KR101492473B1 - Context-aware cctv intergrated managment system with user-based - Google Patents

Abstract

The present invention relates to a user-based context-aware integrated CCTV control system, which applies an object recognition method for city safety surveillance. The user-based context-aware integrated CCTV control system comprises: a photographing unit (10) to generate image data; a data processing unit (20) to process the image data which is photographed and inputted by the photographing unit (10); an integrated control server (30) to store, analyze and process the image data which is processed and inputted by the data processing unit (20); and a user terminal server which allows a user and a manager to connect to the integrated control server (30). The user-based context-aware integrated CCTV control system applies a method for recognizing an object using a histogram of oriented gradients (HOG).

Description

{CONTEXT-AWARE CCTV INTERGRATED MANAGEMENT SYSTEM WITH USER-BASED}

The present invention relates to a user-based situational awareness type integrated control system, and is a system integration control system applying an object recognition method for monitoring the safety of the city.

In general, CCTV is increasingly used nowadays, where society is becoming frenzied and accidents occur frequently.

First-generation CCTVs remained at the level of simply relaying real-time situations. However, intelligent CCTVs were developed by various societal requirements, and many technologies using object recognition technology were developed for such intelligent CCTVs.

Accordingly, the applicant of the present invention has also proposed a method and apparatus for controlling object recognition and contextual video search apparatuses, such as Korean Patent Registration No. 10-0983777 (object photographing system and control method thereof, hereinafter referred to as "prior art 1") and Korean Patent No. 10-1178878 An image processing system using the same, and a control method thereof, hereinafter referred to as "Prior Art 2").

Various services are provided by applying the prior art 1 and the prior art 2 as described above.

Most of these object recognition techniques recognize objects using HOG (Histogram of Oriented Gradients) algorithm.

However, this technique analyzes the characteristics of the target object by applying the method of analyzing the full body of the target object. Therefore, even though the same outline condition, that is, the same body condition and the same costume for a person, The same thing as recognizing the object was inevitable.

Also, objects such as vehicles are recognized as the same object if they have the same external condition, that is, the same vehicle type and the same color.

Therefore, there has been a demand for a more precise object recognition and an integrated control system utilizing the same, and accordingly, there is a demand for development of a more improved technique.

Korean Registered Patent No. 10-0983777 (object photography system and its control method, registered on September 16, 2010) Korean Patent No. 10-1178878 (Object Recognition and Surveillance Video Retrieval Apparatus, Image Processing System and Control Method Using It, Registered on August 31, 2012)

In order to overcome the above problem, a user-based context-aware CCS integration control system applying a more precise object recognition method is applied.

According to an aspect of the present invention, there is provided an apparatus for capturing an image, the apparatus comprising: an imaging unit (10) for generating image data; and an imaging unit An NVR server 31 capable of storing image information obtained from the information processing unit 20 so as to store, analyze, and process input image data processed by the information processing unit 20 and the information processing unit 20; A map information server 32 for storing information on an installation area of the image acquisition device or a map of a region where a specific event occurs, a DB management server 33 for managing and storing the acquired image information or usage history, A display control server 35 for controlling the display server 34, an administrator and a user to the integrated control server 30, Fold A user management server 36 that allows the user to set information, history and authority so that the user can use the service after the connection is established, the video information obtained by the information processing unit 20 according to the type of the event, A distribution / storage / search server 37 for allowing the administrator or a user to search for storage information, an integrated management server 38 for managing each server of the integrated control server 30, And a user terminal server 40 for allowing an administrator and a user to connect to the integrated control server 30, Based contextual cognition integrated control system.

Here, the method of recognizing an object as described above applies an object recognition method based on a Histogram of Oriented Gradients (HOG) algorithm technology. In order to more precisely recognize an object, And classifies the object type by analyzing each characteristic.

In addition, the method of dividing the object into three is divided into a head, an upper body, and a lower body when the object is a human, and a feature value of the object is extracted by dividing the object into a vehicle bumper, In order to reduce the error of judging a similar object as the same target object, the target object is divided to extract the feature values of the divided parts, thereby obtaining the type classification of the object and the feature information of each part, After analyzing the unique characteristics of the objects, it compares and analyzes the information obtained from the two processes, and stores the unique characteristics of the object as DATA.

The object recognition method may further include a method of recognizing the license plate of the vehicle object. The method of recognizing the license plate of the vehicle object recognizes the horizontal part of the outline of the bumper part of the vehicle, Recognizes the number portion of the license plate, rotates the license plate by an inclined angle, and recognizes the number portion. The method of recognizing the license plate of the vehicle object includes a method of compensating the lost car number information And only the character region of the obtained number information is divided, and then the divided character region is divided and compared and analyzed.

In addition, when the image information obtained through the user mobile device connected through the user terminal server 40 is transmitted to the integrated control server 30 and used as DATA, Or to analyze the specific image information requested by the user and to transmit the event information to the user's mobile device. The present invention provides a user-based situational awareness system You can.

By providing the present invention, there is an effect that reliability can be improved by applying a user-based context-aware CSIS integrated control system applying a more precise object recognition method.

1 is a structural view showing a system of the present invention.
FIG. 2 and FIG. 3 illustrate an object recognition method of the present invention.
4 is a schematic diagram of a method for recognizing a number of a vehicle object according to the present invention.
5 is a schematic diagram of compensation of a damaged image in the method of recognizing a number of vehicle objects according to the present invention.
6 is a schematic diagram showing an information processing method of the present invention.
7 is a schematic diagram of a customized event service that may be implemented through the present invention.
FIGS. 8 and 9 are schematic diagrams of a movement direction detection event of an object according to the present invention.
Figure 10 is a schematic diagram of a mobile-based event in accordance with the present invention.

Hereinafter, a user-based situational awareness type integrated control system of the present invention will be described in detail with reference to the drawings so that those skilled in the art can easily carry out the present invention.

1 is a structural view showing a system of the present invention.

An information processing unit 20 for processing the image data photographed and inputted from the image pickup unit 10 and an information processing unit 20 And a user terminal server 40 for allowing an administrator and a user to connect to the integrated control server 30. The integrated control server 30 includes a server 40,

The image capturing unit 10 is a video input unit installed in a specific or unspecified area. The image capturing unit 10 can be any type of image capturing unit, SDI cameras, analog cameras, IP zoom speed cameras, and means for recognizing various types of images applicable to a CCTV system can be applied.

Also, a personal communication terminal, such as a smart phone of a user, which is located outside and is capable of capturing an image, can be applied.

The information processing unit 20 converts the photographed image acquired from the imaging unit 10 into image information and processes the image information.

Here, the image information refers to a format used for the integrated control server 30, for example, by analyzing images captured by various image capturing apparatuses of the image capturing unit 10, It formalizes information.

More specifically, analog signals are formatted into digital signals, and different protocols are converted into standard protocols. Here, the standard protocol adopts the RTSP format.

This is to ensure that if the case needs to be dealt with in cooperation with other regions, regardless of jurisdiction, it can be mutually compatible.

For example, in the case of an open-inquiry case, when a synchronized investigation is conducted for each region, a target object must be searched and analyzed jointly.

The information processing unit 20 may be any unit capable of recording and signal processing such as a local controller, a DVR, and an NVR.

The image information processed in the information processing unit 20 may be transmitted to the integrated control server 30 and applied to various types of user-based data.

The integrated control server 30 includes an NVR server 31 capable of storing the image information acquired by the information processing unit 20, information on an installation area of each image acquisition device or a map of an area where a specific event occurs, A DB management server 33 for managing and storing acquired image information or usage history, a display server for processing information such as object recognition, displaying the processed information, The display control server 35 for controlling the display server 34 and the display server 34, the administrator and the user to access the integrated control server 30, and the user's information, history and authority A user management server 36 for allowing the user to set the video information, the video information acquired by the information processing unit 20 according to the type of the event, An integrated management server 38 for managing each server of the distribution / storage / search server 37 and the integrated control server 30 for enabling the image display device 39 to display images, .

The user terminal server 40, which allows the user to access the integrated control server 30 in the network or externally, can be connected to the integrated control server 30 through a network And is for controlling the connection of the PC 41 and the personal portable communication terminal 42 that can be connected through the network.

Here, the user terminal server 40 solves a security problem that may occur when a user including an administrator accessing the integrated control server 30 directly accesses the integrated control server 30, This is to reduce the traffic problem that can occur when a number connection is made.

The user-based context-aware CCTV integrated control system of the present invention implemented with the above-described configuration applies user-customized events.

Here, the user-customized event is information that can be selectively applied by a user, such as tracking a movement trajectory of a target object by designating a specific target object, detecting a change in a state of a specific region, and the like.

For this, object recognition method based on Histogram of Oriented Gradients (HOG) algorithm technology is applied.

Hereinafter, this will be described in detail.

FIG. 2 and FIG. 3 illustrate an object recognition method of the present invention.

Referring to FIG. 2, the object recognition method according to the present invention recognizes an object based on a Histogram of Oriented Gradients (HOG) algorithm technology as described above.

Here, the HOG technology is a technology known to any one of ordinary skill in the art, and a variety of open source sources are available, so a detailed description thereof will be omitted.

In this object recognition method, an object is a tangible object such as a person or an automobile.

It recognizes these objects separately, recognizes the characteristics, and uses them as DATA.

The object recognition method as described above is performed in three steps.

The object is firstly classified, secondly, the characteristic of the object is distinguished, and the object characteristic is compared with the third object. Then, the object is stored as DATA and applied to the customized event.

In the primary object classification, the types of the target objects are classified based on the HOG technology, and the characteristics of the target objects are analyzed.

In this case, the applied method is to distinguish and apply characteristic parts of an object. In the case of a human, it is determined whether or not a person is a person by analyzing each characteristic of the head, upper body, and lower body in three minutes, To obtain information.

As described above, the unique characteristics of the object are classified based on the information obtained by extracting the types of the objects and the features of the parts.

In this case, the full body of the object is analyzed by applying HOG technology.

The third step is to compare and analyze the data based on the first and second analyzed data as described above, and to apply the characteristic of the object to the data and store the data.

More specifically, the feature values of the body parts (head, upper body, and lower body) of the object are compared with the intrinsic characteristics of the secondary object in the primary object classification process, and then the result values are mixed and applied.

That is, by analyzing the characteristics of the target object through the full body analysis and combining the characteristic values of each body part, it is possible to analyze the characteristics of the object more precisely.

This is to reduce errors that are judged to be the same object even though they are different objects when similar body conditions and same clothes are worn when the object is a person.

In addition, when the target object is a vehicle, it is divided into three parts (bumpers, bumper abnormalities, and front glass parts), and each characteristic value is extracted and applied.

This is because even if the vehicle is of the same vehicle type and the same color, the specific value of the vehicle varies depending on whether the bumper is damaged, such as peeling of the paint, degree of tanning of the front glass part,

Of course, although the car number recognition technology is applied in parallel, it is difficult to extract the characteristic value unique to the vehicle only by the car number because the car number can forge the license plate, so that the error is reduced.

The target object is identified and analyzed according to the above-described method, and is applied as a DATA of a user-customized event based on a specific unique value of the target object.

The above-described method of recognizing the number of the vehicle object will also be described in detail below.

FIG. 4 is a schematic diagram of a method of recognizing a vehicle object number according to the present invention, and FIG. 5 is a schematic diagram of compensation of a damaged image in the method of recognizing a vehicle object number according to the present invention.

Referring to FIGS. 4 and 5, the method of recognizing the number of objects of the vehicle is required to compensate the loss of the recognized license plate and to supplement the image information because of the characteristics of the vehicle.

This is because distortion may occur due to the positional difference between the moving vehicle and the photographing device for photographing, and it may be difficult to obtain clear image information due to breakage of the license plate.

For example, due to the nature of the photographing device, it must be installed on the side of the roadway and is separated from the trajectory of the vehicle movement by a certain angle or more.

Due to the above-described reasons, the photographed image information is supplemented and compensated for.

Here, the method of recognizing the license plate of the vehicle is to recognize the license plate after classifying the structure of the vehicle by applying the object recognition technology described above.

More specifically, after recognizing the bumper portion of the vehicle and recognizing the horizontal portion of the contour of the bumper, the angle of the bumper (0: the horizontal value of the bumper) and the value After extracting the difference, recognize the number part of the license plate and rotate it by the tilted angle to recognize the number part.

This is for recognizing a more accurate number by supplementing the slope of the car number part that can occur during shooting.

In addition, as a method of compensating the lost part of the recognized car number, a method of supplementing the slope of the car number described above and then supplementing the image information by analyzing the outline component is applied.

In more detail, for the lost part of the recognized number information, the character part is analyzed to remove the imaginary part, and the recognized number is compensated.

For example, only the character area is divided into information of the obtained license plate, and then the characters are classified and analyzed.

Referring to FIG. 5 as an example, in the case of the information located on the right side of the obtained information, the division of 2 and 3 becomes ambiguous due to the lost portion.

The ambiguous information is divided into an upper part, a central part and a lower part. The outline of the upper part of the character is recognized, and the left side is recognized as a curved line, and the letters '2' and '3' After the range is defined, the linear components of the center and the left and right sides are analyzed.

As described above, the straight line component can be analyzed to determine '2' as the same pattern, and the lost part of the lower part is compensated by applying the information of the lost part from the complete standard character.

A method of processing information after recognizing the object information will be described in detail below.

6 is a schematic diagram showing an information processing method of the present invention.

6, information can be processed using the image information obtained by using various devices of the image pickup unit 10. The obtained image information is divided into still images and moving images such as photographs Processing.

In the case of a still image such as a photograph, it can be directly applied by a method such as object information recognition, but in the case of a moving image, a separate processing method is used.

In more detail, a specific frame is extracted based on characteristics of a moving picture, and the information is utilized.

This is known to anyone skilled in the art, but moving pictures continuously reproduce still images continuously photographed for a predetermined time, so that a specific still image is extracted from a plurality of still images and a specific value is extracted .

For such a method, there is a method of dividing a moving image by a certain time, but a boundary of a shot for each image is detected and a specific frame (key-frame) is detected and stored in the DB.

The stored DB retrieves and compares the extracted feature values and the image information stored in the DB through the feature extraction algorithm based on the user's request and displays the result through the image display device.

FIG. 7 is a schematic diagram of a user-customized event service that can be performed through the present invention, FIGS. 8 and 9 are schematic diagrams of a movement direction detection event of an object according to the present invention, It is an overview.

7 to 10, a user-customized event service according to the present invention includes a motion detection event of a target object, a color detection event of a target object, an intrusion detection event of an inhibited zone, a movement direction detection event of an object, Object number recognition event, object movement path tracking event, object type classification event, object color classification event, and mobile based event can be applied.

Here, the event is not limited thereto, and various events that can be utilized and transformed through the object recognition system can be applied.

In the case of the movement direction detection event of the object among the above-mentioned events, the direction in which the object moves can be deduced through the trajectory extraction, so that the continuous motion for a specific object can be analyzed and applied.

For example, in the case of a vehicle object, after recognizing the position of the vehicle at the first search point and the direction of the vehicle, the position and direction of the vehicle are recognized and compared after a certain time, and the direction of the vehicle is predicted.

At this time, an analysis of the moving direction and the moving speed of the vehicle can also be performed. Since a number of known techniques are disclosed as an art to anyone skilled in the art to analyze the moving speed of the vehicle, a detailed description thereof will be omitted .

As described above, the moving direction of the object is tracked, and the information of the photographing apparatus provided at the associated point is tracked and investigated to acquire information such as the movement trajectory and the moving direction of the vehicle.

In addition, such a function can be applied to a mobile device such as a personal mobile communication device in addition to a PC via a network. The mobile terminal can access the integrated control server 30 through a mobile device, It may be transmitted to the control server 30 and used as DATA.

It is also possible to use various event services by transmitting image information about a specific target object.

For example, in the case of a vehicle hit-and-run accident, when a user captures a moving image or the like and transmits the captured moving image to the integrated control server 30 to report an incident, a specific object can be designated based on the detected incident, .

Also, in case of the occurrence of the child, it is necessary for the informant to determine the similarity of the object by self-determination using only the reported information of the characteristic of the child, which is the target object, And clothes may be difficult to judge.

At this time, it is possible to analyze and apply the image information transmitted by the caller to the characteristic of each part of the target object and the characteristic value unique to each body.

This is intended to be more precise and reduce the error range so that it can be applied to investigation or the like.

In addition to these methods, we refer to unusual behaviors that occur in traffic violation, violation of driving regulations, waste disposal, disaster, disaster, suspicious movements in child protection area, vulnerable areas such as crime areas. Storage, and analysis of the data, and enforcement of the data.

The integrated control system can be implemented more efficiently by the above-described configuration and method.

10: image pickup section 20: information processing section
30: Integrated control server 40: User terminal server

Claims (8)

1. An apparatus capable of capturing an image, comprising: an imaging unit (10) for generating image data;
An information processing unit 20 for processing the image data photographed and input from the image sensing unit 10;
An NVR server 31 that can store image information obtained by the information processing unit 20 so that the image data processed and input from the information processing unit 20 can be stored, analyzed, and processed; A map information server 32 in which information about a map of a region where a specific event occurs is stored, a DB management server 33 for managing and storing the acquired image information or use history, A display control server 35 for controlling the display server 34, an administrator and a user to connect to the integrated control server 30, A user terminal server 36 for setting the information, history and authority of the user so that the user can use the post-service, the image information obtained by the information processing unit 20 can be distributed or stored according to the type of the event A distribution / storage / search server 37 for allowing storage information to be retrieved by an administrator or a user, an integrated management server 38 for managing each server of the integrated control server 30, An integrated control server 30 composed of a video display unit 39 capable of displaying the video; And
And a user terminal server 40 that allows an administrator and a user to access the integrated control server 30. The method of recognizing the object is based on Histogram of Oriented Gradients (HOG) algorithm technology, Head, upper body, and lower body, and when the object is a vehicle, it is divided into three parts by a vehicle bumper, a vehicle bumper abnormality, and a front glass part. In order to reduce the error, we divide the target object, extract the feature values of the divided parts, acquire the classification of the object and the feature information of each part, analyze the full body of the target object, And the characteristic of the target object is applied to the data as a result of comparison and analysis based on the information acquired in the two processes, Aware ssissi TV integrated control system.
delete delete delete delete The method according to claim 1,
The object recognition method may further include a method of recognizing a license plate of a vehicle object. A method of recognizing a license plate of a vehicle object may include recognizing a horizontal portion of an outline of a bumper portion of the vehicle, And recognizes the number portion of the license plate and rotates it by an inclined angle, thereby recognizing and complementing the numeric portion, thereby recognizing the user-based context-aware CCTV control system.
The method according to claim 6,
The method of recognizing the license plate of the vehicle object may further include a method of compensating lost car number information. After dividing only the character area of the obtained number information, the divided character area is divided, Based on the user-based situational awareness type CCS integrated control system.
The method according to any one of claims 1, 6, and 7,
In transmitting the image information acquired through the user mobile device connected through the user terminal server 40 to the integrated control server 30 and using the image information as DATA, Or analyze the specific image information requested by the user and transmit the event information to the user's mobile device.

Images