KR20200136583A - System for searching numbers of vehicle and pedestrian based on artificial intelligence - Google Patents

Abstract

The present invention discloses a method for recognizing a vehicle number and a face. More specifically, provided is a system for searching a vehicle number and a pedestrian based on artificial intelligence which has a limit on the number of a plurality of vehicles and pedestrians from images at a short distance or a remote distance and performs recognition and search in a state where visual identification is possible without any size limitation. According to an embodiment of the present invention, image signals are collected from a plurality of CCTV cameras installed in each area, the ROI is set for vehicles and pedestrians which appear in the image, and the accuracy of searching for vehicles and pedestrians can be improved by determining whether the vehicle or the pedestrian exists using an artificial intelligence technique.

Description

AI-based vehicle number and pedestrian search system and method {SYSTEM FOR SEARCHING NUMBERS OF VEHICLE AND PEDESTRIAN BASED ON ARTIFICIAL INTELLIGENCE}

The present invention relates to a method for recognizing vehicle numbers and faces, and in particular, there is no limit on the number of faces of a plurality of vehicles and pedestrians from an image at a short distance or a long distance, and recognition in a state in which visual identification is possible without limiting the size. And an artificial intelligence-based vehicle number and pedestrian search system that performs a search.

In order to cope with increasing crime problems due to population increase, population density due to urbanization, alienation due to industrialization, etc., and to establish a social safety net, the introduction of CCTV by government agencies such as local governments is spreading nationwide. The share of used crime solutions is increasing year by year.

While CCTV is increasingly important as evidence that can objectively prove the fact of a crime or criminal, information overload (image information collected too much) has caused excessive reading work, which has also led to a large number of people in charge. Is giving you difficulties.

In general, there are reports that the probability of the situation when monitoring CCTV continuously for more than 22 minutes in the general public drops to less than 5%.

In order to solve such a problem, there is an increasing demand for the introduction of intelligent video surveillance technology that analyzes various behaviors such as vehicle number recognition, pedestrian face recognition, and intrusion.

However, according to the "Intelligent CCTV behavior-based DB construction, test, and authentication method study (Korea Internet & Security Agency, 2012)", the existing CCTV system was surveyed as 21.5 out of 100 points of satisfaction due to frequent false positives. Systems are found to be rarely used.

The present invention intends to present an artificial intelligence-based CCTV integrated control solution in which a computer (image analysis solution) automatically recognizes and searches vehicle numbers and faces using images through CCTV cameras, but maximizes the accuracy.

The present invention has been conceived to solve the problems of the prior art, and has a problem to provide a method for recognizing a vehicle number and a face through deep learning-based data learning.

In particular, an object of the present invention is to provide a method for improving false detection by using a hierarchical structure of objects in a deep learning-based data learning method.

In order to solve the above problems, the artificial intelligence-based vehicle number and pedestrian search system according to an embodiment of the present invention is a search system that collects video signals in real time from a plurality of CCTV cameras installed at remote locations through an information communication network. A database server that stores the vehicle number and face information of pedestrians, and a storage distribution server that receives the image signal from a transmission terminal and extracts one or more objects appearing in the image, and classifies and stores the extracted object as a vehicle or a pedestrian And an image analysis server that performs image processing on vehicles or pedestrians classified through an image processing technique, and identifies vehicles or pedestrians of each object appearing in the image of the image signal by referring to the database. .

The image analysis server includes an image signal input unit for sequentially loading one or more objects stored in a storage distribution server, a classification unit for setting one or more ROIs in an object area according to characteristics included in the loaded object, the ROI and the data In accordance with the verification result and a verification unit that verifies vehicles or pedestrians belonging to the same or similar category by collating multiple vehicle numbers or face information stored in the base using artificial intelligence techniques, the vehicle information or face information having a similarity greater than or equal to the reference value It may include an identification unit that identifies the object of the corresponding ROI as a specific vehicle or pedestrian.

In addition, the artificial intelligence-based vehicle number and pedestrian search method according to another aspect of the present invention for solving the above-described problem, acquires an image signal from a plurality of CCTV cameras in real time, and one or more objects appearing in the image. Extracting and classifying and storing the extracted object as a vehicle or a pedestrian, determining whether it is a vehicle or a pedestrian by setting one or more ROIs on the extracted object, and recognizing the vehicle number by detecting a license plate in the case of a vehicle, Determining whether an object other than a vehicle or a pedestrian is a two-wheeled vehicle, verifying the rules of the object, excluding the two-wheeled vehicle, determining whether overlapping with the neighboring object area for the ROI currently being analyzed has occurred, and when overlapping occurs It may include summing areas of neighboring objects, converting each frame of the target image into a pixel size of VGA resolution, and storing the converted image in a file format.

According to an embodiment of the present invention, by collecting image signals from a plurality of CCTV cameras installed in each area, setting an ROI for vehicles and pedestrians appearing in the image, and determining whether a vehicle or a pedestrian is using an artificial intelligence technique. There is an effect of improving the accuracy of vehicle and pedestrian search.

1 is a diagram illustrating the overall structure of an artificial intelligence-based vehicle number and pedestrian search system according to an embodiment of the present invention.
FIG. 2 is a diagram of an image analysis device of an artificial intelligence-based vehicle number and pedestrian search system according to an embodiment of the present invention.
3 is a diagram of a vehicle number and a pedestrian search method based on artificial intelligence according to an embodiment of the present invention.
4 is a diagram schematically illustrating a vehicle number analysis method of an artificial intelligence-based vehicle number and pedestrian search system according to an embodiment of the present invention.
5 is a diagram illustrating an electronic display screen provided by an artificial intelligence-based vehicle number and pedestrian search system according to an embodiment of the present invention.

When a part of the specification says that a certain component is "equipped" or "included", it means that other components may be further included instead of excluding other components unless otherwise stated. In addition, terms such as "...unit", "...server" and "...system" described in the specification refer to units that process one or more functions or operations. It means, and this may be implemented in hardware, software, or a combination of hardware and software.

In addition, in the present specification, the term “an embodiment” means serving as an illustration, example, or illustration, but the subject of the invention is not limited by such an example. In addition, "comprising", "having", "having" and other similar terms are used, but when used in the claims, "included" as an open transition word that does not exclude any additional or other elements. It may be used generically in a manner similar to the term "comprising".

The various techniques described herein may be implemented with hardware or software, or, where appropriate, with a combination of both. As used herein, terms such as "...unit", "...server" and "...system" are likewise computer-related entities, i.e. hardware , It can be treated as a combination of hardware and software, software or equivalent to software at the time of execution. In addition, each function executed in the program of the present invention may be configured in a module unit, and may be recorded in one physical memory, or distributed between two or more memories and a recording medium.

1 is a diagram of an overall structure of an artificial intelligence-based vehicle number and pedestrian search system according to an embodiment of the present invention.

Referring to FIG. 1, an artificial intelligence-based vehicle number and pedestrian search system according to an embodiment of the present invention is a search system that collects video signals in real time from a plurality of CCTV cameras 10 installed at remote locations through an information communication network, A database server 110 storing a plurality of vehicle numbers and face information of pedestrians, receives an image signal from the transmission terminal 50, extracts one or more objects appearing in the image, and classifies the extracted objects as vehicles or pedestrians And a storage distribution server 120 to store, an electronic signage device 130 that displays an image signal corresponding to each CCTV camera 10 on the screen in real time, and image processing for vehicles or pedestrians classified through an image processing technique. And, it may include an image analysis server 200 for identifying vehicles or pedestrians of each object appearing in the image of the image signal with reference to the database server 110.

In detail, the artificial intelligence (AI) based image analysis solution applied to the search system according to the embodiment of the present invention recognizes the vehicle number and the face of a pedestrian in real time from the video data transmitted from the vehicle number recognition camera and the network camera. It features.

In one search system, the vehicle number and the face of a pedestrian can be recognized from images of the CCTV camera 10 of up to 24 or 32 channels.

The database server 110 stores the recognized vehicle number and face information, which can be inquired through the recognized time, vehicle number, and search query terms for a specific face.

The storage distribution server 120 may classify and store the transmitted image signal in a designated storage space. A plurality of CCTV cameras 10 may be installed on roads in each region, and the storage distribution server 120 may receive and store video signals transmitted from the CCTV cameras 10 in each region in real time.

The image analysis server 200 may analyze the transmitted image signal in real time and display the transmitted image signal on the screen through the connected electronic display device 130. The video displayed on the screen can be monitored by control personnel.

For this purpose, the electronic display device 130 may be implemented as a large flat panel display device having a large area and high definition.

In addition, the image analysis server 200 can query the image signal stored in the storage distribution server 120 by combining the recognized time of vehicles and pedestrians appearing in the image, and a search query word for a specific face, and the search result is a storage distribution server You can write to (120).

The image analysis solution applied to the image analysis server 200 can be implemented in the form of a single control device for image analysis, video storage, data storage, and data processing, and can process image analysis tasks through one device. I can.

FIG. 2 is a diagram of an image analysis device of an artificial intelligence-based vehicle number and pedestrian search system according to an embodiment of the present invention.

Referring to FIG. 2, the image analysis apparatus 200 includes an image signal input unit 210 that sequentially loads one or more objects stored in a storage distribution server, and one or more ROIs in an object area according to characteristics included in the loaded object. Classification unit 220 to be set, verification unit 230 for verifying vehicles or pedestrians belonging to the same or similar category by comparing ROI and a plurality of vehicle numbers or face information stored in the database using artificial intelligence techniques, and verification results Accordingly, it may include an identification unit 240 for identifying the object of the ROI corresponding to the vehicle information or face information having a similarity greater than or equal to the reference value as a specific vehicle or pedestrian.

The image signal input unit 210 sequentially loads the image signals collected and classified and stored from the CCTV camera into the memory, and prepares for image processing. In this case, a pre-processing process for the image may be further included.

The classification unit 220 may determine whether an object appears in the image, and when the object appears, classify the type of the object into vehicles, motorcycles, and pedestrians according to the shape and set the ROI. In particular, in the case of a vehicle, a license plate is further included, and an ROI for the license plate can be further set.

The verification unit 230 may verify whether each object is a vehicle or a pedestrian through an image analysis procedure for the ROI. In the case of vehicles, elements corresponding to the windshield, license plates, and tires are included, and in the case of pedestrians, elements corresponding to the face, body, arms, and legs are included. The verification unit 230 may classify the type of each object through the presence or absence of the above-described elements.

The identification unit 240 may perform an identification procedure on an object whose type is specified. In the case of a vehicle, the target can be identified through the vehicle number written on the license plate, and in the case of a pedestrian, the person can be identified through image control on the face part. To this end, a number of images for deep learning are stored in the database server, and the identification unit 240 identifies vehicles and pedestrians using the hierarchical structure of objects in the deep learning-based data learning method and detects false positives. It can be improved.

3 is a diagram illustrating an artificial intelligence-based vehicle number and a pedestrian search method according to an embodiment of the present invention. In the following description, the execution subject of each step becomes the search system of the present invention and each component thereof, and each step is produced by a known programming language, and can be executed by a microprocessor mounted on an image analysis device, and read It can be recorded on a writable recording medium and mounted on an image analysis device.

3, the vehicle number and pedestrian search method of the present invention acquires an image signal from a plurality of CCTV cameras in real time, extracts one or more objects appearing in the image, and classifies and stores the extracted object as a vehicle or a pedestrian. The step of setting one or more ROIs on the extracted object to determine whether it is a vehicle or a pedestrian, and in the case of a vehicle, detecting a license plate to recognize the vehicle number, determining whether an object other than the vehicle and pedestrian is a two-wheeled vehicle, Excluding the motorcycle by verifying the rules of the object, determining whether overlap with the neighboring object region has occurred with respect to the currently analyzed ROI, summing the regions of neighboring objects when overlapping occurs, each of the target images It may include converting the frame into a pixel size of VGA resolution and storing the converted image in a file format.

In detail, first, a search system receives and acquires an image signal photographing a road in real time from a plurality of CCTV cameras installed in a remote location (S100), and extracts one or more objects appearing in the image and extracts the extracted object from the vehicle or Classify and store as pedestrians.

Thereafter, it is determined whether each object appearing in the image signal is a vehicle through image analysis (S110). If it is determined that it is not a vehicle, it is checked whether the object is a pedestrian (S120), and if it is not a pedestrian, it is checked whether it is a two-wheeled vehicle (S130).

In addition, in step S110, if the object is a vehicle, a license plate is detected (S140), and the vehicle number is recognized (S150).

Next, the rule of the object is verified with respect to the object determined to be a vehicle, a pedestrian, or a two-wheeled vehicle (S160), and when it is finally determined as an object, it is determined whether or not they overlap with other neighboring objects (S170).

If it is determined that an overlapping area between two or more neighboring objects has occurred in step S170, the areas of the objects are summed (S180).

Thereafter, the resolution of the image is converted into VGA, that is, the size of 640 horizontally and 480 vertically (S190), and the process is terminated by storing the file (S190).

4 is a diagram schematically illustrating a vehicle number analysis method of an artificial intelligence-based vehicle number and pedestrian search system according to an embodiment of the present invention.

Referring to FIG. 4, in the AI-based vehicle number and pedestrian search system according to an embodiment of the present invention, number extraction may be performed on license plates of two standards.

First, it is determined whether the license plate satisfies the standard conditions of the license plate from the recognized result. The standard conditions of the license plate are as follows.

As a first condition, there must be four digits of the serial number (D1 to D4).

As a second condition, the use character Hangul (H1) must exist.

As a third condition, the Hangul + number combination (H3, H2, D6, D5) excluding the first and second conditions must exist.

Here, the first and second conditions are essential and sufficient conditions, and the third condition may be a necessary condition.

5 is a diagram illustrating an electronic display screen provided by an artificial intelligence-based vehicle number and pedestrian search system according to an embodiment of the present invention.

Referring to FIG. 5, the artificial intelligence-based vehicle number and pedestrian search system according to an embodiment of the present invention may display an image signal collected by a CCTV camera through an electronic sign. The image screen W100 displayed through such an electronic signage device may be an image W110 transmitted in real time from one CCTV camera in a form in which the screen is divided.

A plurality of objects such as vehicles, pedestrians, and motorcycles may appear in one image W110, and the search system may set each object and identify the object through analysis by ROI.

In particular, in the case of vehicles (ROI3) other than pedestrians and motorcycles (ROI1), the license plate (ROI2) is included. It will be used to identify the vehicle through the processing procedure.

Although many items are specifically described in the above description, this should be construed as an example of a preferred embodiment rather than limiting the scope of the invention. Therefore, the invention should not be determined by the described embodiments, but should be determined by the claims and equivalents to the claims.

10: CCTV camera 50: transmission terminal
100: artificial intelligence-based vehicle number and pedestrian search system
110: database server 120: storage distribution server
130: electronic display device 200: image distribution server
210: video signal input unit 220: classification unit
230: verification unit 240: identification unit

Claims (3)

As a search system that collects video signals in real time from a plurality of CCTV cameras installed in remote locations through an information and communication network,
The search system,
A database server storing a plurality of vehicle numbers and face information of pedestrians;
A storage distribution server that receives the image signal from a transmission terminal, extracts one or more objects appearing in the image, and classifies and stores the extracted object as a vehicle or a pedestrian; And
An image analysis server that performs image processing on vehicles or pedestrians classified through an image processing technique, and identifies vehicles or pedestrians of each object appearing in the image of the image signal by referring to the database
Artificial intelligence-based vehicle number and pedestrian search system comprising a. The method of claim 1,
The video analysis server,
An image signal input unit for sequentially loading one or more objects stored in the storage distribution server;
A classification unit configured to set one or more ROIs in an object region according to characteristics included in the loaded object;
A verification unit for verifying vehicles or pedestrians belonging to the same or similar category by comparing the ROI with a plurality of vehicle numbers or face information stored in the database using an artificial intelligence technique; And
According to the verification result, an identification unit that identifies vehicle information having a similarity greater than or equal to the reference value or an object of ROI corresponding to face information as a specific vehicle or pedestrian
Artificial intelligence-based vehicle number and pedestrian search system comprising a. Obtaining an image signal from a plurality of CCTV cameras in real time, extracting one or more objects appearing in the image, and classifying and storing the extracted object as a vehicle or a pedestrian;
Determining whether a vehicle is a vehicle or a pedestrian by setting one or more ROIs on the extracted object, and recognizing a vehicle number by detecting a license plate in the case of a vehicle;
Determining whether an object other than a vehicle and a pedestrian is a two-wheeled vehicle, verifying the rule of the object, and excluding the two-wheeled vehicle;
Determining whether or not overlapping with neighboring object regions has occurred with respect to the ROI currently being analyzed;
Summing areas of neighboring objects when overlapping occurs;
Converting each frame of the target image into a pixel size of VGA resolution; And
Saving the converted video in file format
Artificial intelligence-based vehicle number and pedestrian search method comprising a.

Images