KR20200084947A - Smart cctv system for analysis of parking - Google Patents

Abstract

The present invention relates to an intelligent CCTV system for analyzing a parking surface of a vehicle. According to one embodiment of the present invention, the intelligent CCTV system comprises: one or more CCTV devices for photographing a parking lot in which a plurality of parking surfaces in which vehicles can park are arranged; and a server training a detection model for the parking of the vehicle on the plurality of parking surfaces based on an artificial neural network by using a database for extracting a video ad an image photographed in the one or more CCTV devices and characteristics of the parking surfaces and the vehicle, and operating and outputting a result for the parking of the vehicle on the plurality of surfaces of the parking lot. Even without large-scale construction, whether the parking space is used can be checked using the video photographed by the CCTV device, so as to simply check the use of the parking surface.

Description

Intelligent CCTV system for vehicle parking surface analysis {SMART CCTV SYSTEM FOR ANALYSIS OF PARKING}

The present invention relates to an intelligent CCTV system for analyzing the parking surface of a vehicle, and more specifically, an intelligent CCTV system for analyzing the parking surface of a vehicle that performs real-time parking area analysis based on an image captured through CCTV. It is about.

Parking is a place for vehicles to park. When parking a vehicle in such a parking lot, it is not easy for the driver to find a location in the vehicle where the vehicle is not parked on the parking surface of the parking lot. Therefore, in order to guide the driver to park quickly in the parking lot, a service is provided to guide the location where the parking surface is empty.

As a service for guiding the empty position of the parking surface, a technology for detecting presence of a vehicle on the parking surface using a detection sensor such as a geomagnetic sensor or an ultrasonic sensor is used. However, there may be a case where a person is standing on the parking surface or water is standing, or when there are other objects, an error such as a vehicle being displayed on the parking surface may occur.

In addition, since it is necessary to install a detection sensor for each parking surface, in the case of a large parking lot, there is a problem that requires continuous management, such as installing a detection sensor on a large scale and also replacing a battery for continuously supplying power to the detection sensor. .

Republic of Korea Registered Patent No. 10-1846063 (2018.03.30.) Republic of Korea Registered Patent No. 10-1556076 (2015.09.21)

The problem to be solved by the present invention is to provide an intelligent CCTV system for analyzing the parking surface of a vehicle that can determine whether parking is possible in real time by analyzing the presence or absence of a vehicle on the parking surface by analyzing the image taken by shooting the parking lot. will be.

An intelligent CCTV system according to an embodiment of the present invention includes: one or more CCTV devices for photographing a parking lot on which a plurality of parking surfaces on which a vehicle can be parked are disposed; And a database for extracting features of the parking surface and the vehicle or images captured by the one or more CCTV devices to learn a detection model for parking of the vehicle on a plurality of parking surfaces based on an artificial neural network. , It may include a server that calculates and outputs the results of whether the vehicle is parked on the plurality of parking surfaces according to the learned results.

The server extracts a category feature of whether a vehicle is parked on each of the plurality of parking surfaces using a database, learns a detection model for each parking surface using the extracted category feature, and learns the learned detection model. It is possible to obtain an image or image of whether a vehicle is parked on each of the parking surfaces by using to calculate a result of whether the vehicle is parked on the plurality of parking surfaces.

The server may calculate whether a vehicle is parked on the plurality of parking surfaces using a restricted Boltzmann machine model.

The CCTV device includes: a photographing unit for photographing a parking lot on which a plurality of parking surfaces on which a vehicle can be parked are disposed; And using a database for extracting features of the parking surface and the vehicle and the images or images photographed on the photographing unit to learn a detection model for parking of the vehicle on the plurality of parking surfaces based on an artificial neural network, It may include a calculation unit for outputting a result of whether the vehicle is parked on the plurality of parking surfaces according to the learned results.

The server may transmit the calculated result to the computing unit of the CCTV device.

The server learns a detection model for the type of the vehicle based on the artificial neural network by using a database for extracting features of the image or image captured by the CCTV device and the birds of the vehicle, and the learned result Depending on the result of the vehicle type can be calculated.

According to the present invention, the use of the parking surface by using an image captured by a CCTV device without large-scale construction, such as installing a detection sensor on each parking surface to check whether the vehicle is parked on the parking surface of the parking lot Since it is possible to check whether or not the parking surface is used, it is effective.

In addition, through deep learning, it is possible to check the type of vehicle parked on the parking surface, so it is possible to guide and check the use of the parking surface according to the type of vehicle, so that the corresponding parking surface can be guided according to the type of vehicle. It is possible to guide the parking of the vehicle more accurately.

Moreover, it is possible to check the type of the vehicle and check the parking fee according to the type of vehicle.

1 is a block diagram showing an intelligent CCTV system according to an embodiment of the present invention.
2 is a block diagram showing an intelligent CCTV device according to an embodiment of the present invention.
3 is a view for explaining a concept for performing deep learning in a CCTV system according to an embodiment of the present invention.
4 is a view for explaining an example of performing deep learning in a CCTV system according to an embodiment of the present invention.
5 is a view for explaining an example of photographing a vehicle parked in a parking lot in a CCTV system according to an embodiment of the present invention.
6 is a view showing that each parking surface is divided to check the type of vehicle parked in the parking lot in the CCTV system according to an embodiment of the present invention.
7 is a view for explaining the parking of the vehicle according to the type of parking lot in the CCTV system according to an embodiment of the present invention.

With reference to the accompanying drawings, a preferred embodiment of the present invention will be described in more detail.

1 is a block diagram showing a CCTV system including an intelligent CCTV device according to an embodiment of the present invention, Figure 2 is a block diagram showing an intelligent CCTV device according to an embodiment of the present invention. 3 is a diagram for explaining a concept for performing deep learning in a CCTV system according to an embodiment of the present invention, and FIG. 4 is a diagram illustrating an example of performing deep learning in a CCTV system according to an embodiment of the present invention It is a drawing for. And Figure 5 is a view for explaining an example of a vehicle parked in a parking lot in a CCTV system according to an embodiment of the present invention, Figure 6 is a vehicle parked in a parking lot in a CCTV system according to an embodiment of the present invention It is a diagram showing that each parking surface is divided to confirm the type of the vehicle. 7 is a view for explaining the parking of the vehicle according to the type of parking lot in the CCTV system according to an embodiment of the present invention.

1 and 2, the CCTV system 100 including the intelligent CCTV device 110 according to an embodiment of the present invention, the CCTV device 110, the administrator terminal 120 and the server 130 Includes.

A plurality of CCTV devices 110 may be provided. In addition, the plurality of CCTV devices 110 respectively photograph a plurality of parking surfaces of the parking lot 200 and analyze whether the vehicle is parked on the parking surface by analyzing the captured images. In addition, in this embodiment, the CCTV device 110 may analyze the type of vehicle parked on the parking surface, and may also perform analysis on the number of vehicles. To this end, the CCTV device 110 includes a photographing unit 111, a calculation unit 113, and a storage unit 115.

The photographing unit 111 may perform photographing during the day or at night. That is, the photographing unit 111 may include a camera for daytime shooting and an infrared camera for nighttime shooting.

In the present embodiment, the photographing unit 111 describes a fixed location of the CCTV device 110 fixed to one side of the parking lot 200 so as to photograph the parking surface of the parking lot 200 while fixing a specific location. , It is not limited thereto, and the CCTV device 110 may be configured to rotate or move, and the photographing unit 111 may continuously photograph while the CCTV device 110 rotates or moves.

The imaging unit 111 may include one or more camera modules to capture an image.

The calculation unit 113 may perform analysis on the image captured by the imaging unit 111, and may also receive an analysis result of the image captured by the imaging unit 111 from the server 130.

To this end, the calculation unit 113 is based on deep learning to recognize the parking surface of the parking lot 200, a parking surface learning model, vehicle information extracted from collected video images, and whether to use the parking surface in connection with the parking surface learning model. The analysis can be performed through a program capable of analyzing information that can discriminate.

Therefore, the operation unit 113 may be configured with hardware including computing resources (CPU, GPU, RAM, storage, etc.) to perform analysis through the above-described program. In addition, information on the type of the vehicle, a parking surface learning model, and an analysis program may be stored in the storage device included in the calculation unit 113.

In the first step, the calculation unit 113 is constructed with a database for extracting features of the parking surface and the vehicle of the parking lot 200. Then, as a second step, for the learning based on the artificial neural network, a category feature of whether a vehicle is parked on a parking surface is extracted from a database. Then, in the third step, the detection model of the parking surface is trained using the category feature of whether the vehicle is parked. In the fourth step using the learned detection model, it is possible to extract whether the vehicle is parked on the parking surface. At this time, the learned detection model can secure an image or an image for a case where a vehicle is parked on a parking surface and a case where the vehicle is not parked.

At this time, in the third step, the process of learning the detection model of the parking surface includes various weather changes (for example, snow or rain) or time changes that may occur in the parking lot 200 (day and day as the time changes). You can learn the detection model for the parking surface through various images or videos of changes in the night.

And in the fourth step, it is detected whether the actual parking surface is used by using the video image of the actual parking lot 200, and if an error occurs, it is possible to reduce the error by repeating again from the second step. Accordingly, the calculation unit 113 can secure information on whether the vehicle is finally parked on the parking surface, and can also check information on the parked vehicle.

As described above, the calculation unit 113 may transmit the analyzed result to the server 130.

The storage unit 115 stores the results calculated by the calculation unit 113. The storage unit 115 may be provided separately from the storage device of the calculation unit 113, and in some cases, may be a storage device of the calculation unit 113.

The manager terminal 120 is a terminal for managing the parking lot, and management of the parking lot may be performed based on information transmitted from the CCTV device 110 and the server 130. That is, the manager terminal 120 may manage information about a vehicle parked in a parking lot, information about a vehicle entering or exiting, and information about charging a parking fee for the vehicle.

The server 130 receives the image captured by the imaging unit 111 of the CCTV device 110 and analyzes the received image. To this end, the server 130 uses a parking surface in connection with a parking surface learning model capable of recognizing the parking surface of the parking lot 200 based on deep learning, vehicle information extracted from the collected video image, and a parking surface learning model. The analysis may be performed through a program capable of analyzing whether information can be determined and information on a moving path of a vehicle collected from a plurality of CCTV devices 110.

That is, the server 130 may analyze information about each parking surface of the parking lot in real time and transmit the analyzed result and the result transmitted from the calculation unit 113 to the manager terminal 120. The server 130 may perform management of a vehicle entering or leaving a parking lot, management of a parking fee of a vehicle, management of a vehicle type, and management of a discount target according to a vehicle type.

Accordingly, the server 130 uses the parking surface detection model of the parking lot based on the neural network to continuously detect the detection model for the parking surface, the model for vehicle type and vehicle number plate recognition, and the vehicle tracking model through deep learning. It is possible to update and maintain the most accurate model according to each updated model.

In detail, the server 130 receives the video or image photographed in real time through the photographing unit 111 of the CCTV device 110 as a first step. Then, as a second step, information on whether the vehicle is parked is received on the parking surface of the parking lot analyzed by the operation unit 113 of the CCTV device 110. In the third step, it is determined whether the vehicle is parked on the corresponding parking surface by using information on whether the vehicle is parked or not on the parking surface received from the operation unit 113 of the CCTV device 110. In a fourth step, a parking surface learning model for whether parking is performed using a video or image received from the imaging unit 111 of the CCTV device 110 and information received from the operation unit 113 of the CCTV device 110, a vehicle Kind and update the vehicle license plate recognition model and vehicle tracking model. Then, in the fifth step, the updated parking surface learning model, vehicle type and vehicle license plate recognition model, and vehicle tracking model are transmitted to the CCTV device 110 and the manager terminal 120. In the sixth step, management of the parking lot can be managed in the manager terminal 120 based on the information transmitted to the manager terminal.

At this time, the server 130 may be performed through the same process as the first to fourth steps made in the operation unit 113 described above to perform the parking surface learning model. Alternatively, the server 130 generates the parking surface learning model through image analysis based on the artificial neural network as the first step. As a second step, as shown in FIG. 5, images or images of a parking lot are collected in real time from each CCTV device 110. In a third step, as shown in FIGS. 6A to 6F, images corresponding to each parking surface are divided from the collected images or images. In the fourth step, it is determined whether the vehicle is parked on the parking surface using the image of the divided parking surface. For example, FIGS. 6(a), 6(b), 6(d), and 6(f) are determined to be parked, and FIGS. 6(c) and 6( e) may be determined to be a parking surface capable of parking.

At this time, the artificial neural network (ANN) used in the operation unit 113 or the server 130 of the CCTV device 110 refers to a statistical learning algorithm inspired by a neural network of biology in machine learning and cognitive science. do. An artificial neural network may mean an entire model having a problem-solving ability by changing the intensity of synaptic binding through learning of artificial neurons (nodes) that form a network through synaptic coupling.

In this embodiment, a restricted Boltzmann machine model among artificial neural network models may be used. The limited Boltzmann machine model is a model proposed by Geoff Hinton as an algorithm that can be used for dimension reduction, classification, linear regression analysis, collaborative filtering, feature learning, and topic modeling.

The limited Boltzmann machine model consists of a visible layer and a hidden layer, the visible layer includes a number of visible nodes, and the hidden layer includes a number of hidden nodes. And, hidden nodes located in all hidden layers are connected to visible nodes located in the visible layer, and visible nodes in all visible layers are connected to hidden nodes in the hidden layer. That is, in the limited Boltzmann machine model, nodes on the same layer are not connected to each other, but to nodes on other layers.

The visible node of the visible layer makes a stochastic decision on how much the hidden data will be transmitted when the data is input. That is, it is displayed as 1 when it is determined that the input data is to be transmitted according to probability, and 0 when it is determined that the data is not to be transmitted.

Referring to FIG. 3, when input data v is input to the visible node i of the visible layer, a multiplication value of input data of the visible node and a weight wij is added to each other. Then, it is sampled as a value of 0 or 1 through an active function (eg, a sigmoid function), and an output value h is output to the hidden node j.

The limited Boltzmann machine model learns important features of the input data by adjusting weights through unsupervised learning. Here, the weight is input data (v), the output value (h) of the hidden node, the corrected input data (v') calculated through the reconstruction process, and the corrected output value (h) calculated through the regeneration process. It can be adjusted by calculating the error value (v'h'-vh) from').

Referring to FIG. 4, an example of adjusting the weight in the limited Boltzmann machine model will be described. As shown in (a) of FIG. 4, in the reconstruction process, the multiplication values of the output value (h) and the weight (w _ji ) of each hidden node (j) of the hidden layer are summed, and then pass through the active function to 0 or It is sampled with a value of 1 and output as input data v'corrected at the input node i.

As shown in (b) of FIG. 4, the multiplication value of the corrected input data (v') of the visible node (i) and the weight (w _ij ) in the reproducing process is summed with each other and then passed through the active function to be 0 or 1 It is sampled as a value and output as the corrected output value (h') from the hidden node (j) again.

Accordingly, calculation of the output value h, the corrected input data v', and the corrected output value h'can be expressed as Equation 1, Equation 2, and Equation 3, respectively.

[Equation 1]

[Equation 2]

[Equation 3]

Here, P is the sampling function, h _cj is the output value of the hidden node (j) for the input case (c), N _v is the number of visible nodes, v _ci is the input case (c) input as the visible node (i) The input data of, w _ij is the weight between the visible node (i) and the hidden node (j), b ^h _j is the input bias value of the hidden node (j), σ is the active function (e.g., a logistic function) to be.

And N _h is the number of hidden nodes, h _cj is the output value of the input case (c) output from the hidden node (j), w _ji is the weight between the hidden node (j) and the visible node (i), b ^h _i Is the output bias value of the visible node (i), _h'cj is the corrected output value of the hidden node (j) for the input case (c).

The weight (w _ij ), output bias (b _h ), and input bias (b _v ) are input using the calculated output value (h), corrected input data (v'), and corrected output value (h'). It can be corrected to learn the characteristics of the data, and the correction of the weight, output bias and input bias can be corrected by Equation 4, Equation 5 and Equation 6.

[Equation 4]

[Equation 5]

[Equation 6]

Here, N _c is the number of input cases constituting the input arrangement, ε is the learning rate (learning rate). In order to increase computation time and learning efficiency in an artificial neural network, weight correction is performed based on an input batch, which is a set of input cases (c) composed of a plurality of input data.

In the present embodiment using the limited Boltzmann machine model as described above, as input data, an image or an image of an image photographed by the photographing unit 111 may be input. In addition, information about whether the vehicle is parked may be output by deep learning whether the vehicle is parked on the parking surface using the image or image input from the operation unit 113.

In other words, as an example, an image or image photographed by the photographing unit 111 is input to input data v input to the visible node i of the visible layer, and a vehicle is parked on the parking surface from the input image or image. When the value is 1, a value of 1 is output as the output value h of the hidden node j, and when the vehicle is not parked on the parking surface, a value of 0 can be output as the output value h of the hidden node j. And it can be learned while going through the process shown in Figure 4 (a) and 4 (b) to output the result of whether the vehicle is parked on the parking surface.

Through the above analysis, as shown in FIG. 5, in the CCTV system 100 according to the present embodiment, the server 130 is parked when the vehicle 300 is parked on the parking surface in the parking lot 200 and parking If not, the server 130 can confirm that the vehicle 300 is parked on four of the six parking surfaces, and that the vehicle 300 is not parked on the two parking surfaces. . In addition, the detected information may be transmitted to the manager terminal 120, and the manager terminal 120 may display information so that the driver of the vehicle 300 can check the information.

In addition, as shown in FIG. 7, when various vehicles 300 are separated in the parking lot 200, the server 130 checks information on the type of the vehicle 300 parked on the parking surface, Information about the type of the vehicle 300 to be parked can be checked. That is, when the vehicle 300 entering the parking lot 200 is a general vehicle, it is possible to guide the general parking surface 210 where the general vehicle can park. In addition, when the vehicle 300 entering the parking lot 200 is an electric vehicle for electric charging, it is possible to guide the electric vehicle parking surface 220 for the electric vehicle.

In addition, when the vehicle 300 entering the parking lot 200 is a disabled vehicle, it is possible to guide the disabled parking surface 230 where the disabled vehicle can park.

That is, in this embodiment, the server 130 is input to the input data (v) input to the gas node (i) of the visible layer through the deep learning process described in Figure 4 (a) and 4 (b) An image or video captured by the photographing unit 111 of the CCTV device 110 is input. In this case, the input image or image may be information about the vehicle 300. Therefore, the server 130 may input an image or image of the vehicle 300 entering the parking lot 200 as input data v and output a result of the type of the vehicle 300. Accordingly, the type of the corresponding vehicle 300 can be checked. In this embodiment, the type of the vehicle 300 that can be confirmed can confirm the types of light vehicles, small sedans, medium sedans, large sedans, small SUVs, medium SUVs, large SUVs, etc. have.

In addition, in this embodiment, the server 130 may check whether the disabled registration certificate is exposed on the front side of the vehicle 300 in the corresponding vehicle 300. Accordingly, when it is detected that the disabled registration certificate is disposed on the front of the vehicle 300, it is determined that the vehicle is disabled and the corresponding information may be transmitted to the manager terminal 120 to guide the disabled parking surface 230.

Moreover, when the vehicle enters the parking lot, the server 130 may check a moving path of the corresponding vehicle based on images captured by a plurality of CCTV devices 110 for the corresponding vehicle. Through this vehicle tracking model, the server 130 can check information on the parking surface of the vehicle where the vehicle is located.

As described above, the detailed description of the present invention has been made by the embodiments with reference to the accompanying drawings, but the above-described embodiments are only described as preferred examples of the present invention, and the present invention is limited to the above-described embodiments only. It should not be understood, and the scope of the present invention should be understood as the claims and the equivalent concept described below.

100: CCTV system
110: CCTV device
111: filming unit
113: operation unit
115: storage
120: administrator terminal
130: server
200: parking lot
210: normal parking
220: Electric vehicle parking surface
230: handicapped parking
300: vehicle

Claims (6)

At least one CCTV device for photographing a parking lot in which a plurality of parking surfaces on which a vehicle can park is disposed; And
Learning a detection model of whether a vehicle is parked on a plurality of parking surfaces based on an artificial neural network by using an image taken from the one or more CCTV devices or a database for extracting features of the parking surface and the vehicle, and Intelligent CCTV system including a server that calculates and outputs the results of whether or not the vehicle is parked on the plurality of parking surfaces according to the learned results. The method according to claim 1,
The server extracts a category feature of whether a vehicle is parked on each of the plurality of parking surfaces using a database, learns a detection model for each parking surface using the extracted category feature, and the learned detection model An intelligent CCTV system that calculates the result of whether a vehicle is parked on the plurality of parking surfaces by securing an image or a video of whether the vehicle is parked on each of the parking surfaces by using. The method according to claim 1,
The server is an intelligent CCTV system that calculates whether a vehicle is parked on the plurality of parking surfaces using a restricted Boltzmann machine model. The method according to claim 1, wherein the CCTV device,
A photographing unit for photographing a parking lot on which a plurality of parking surfaces capable of parking a vehicle are disposed; And
Learning and detecting a detection model for whether a vehicle is parked on the plurality of parking surfaces based on an artificial neural network by using an image or image photographed on the photographing unit and a database for extracting features of the parking surface and the vehicle. Intelligent CCTV system including a calculation unit to output a result of whether the vehicle is parked on the plurality of parking surfaces according to the result. The method according to claim 4,
The server is an intelligent CCTV system that transmits the calculated result to the computing unit of the CCTV device. The method according to claim 1,
The server learns a detection model for the type of the vehicle based on the artificial neural network by using a database for extracting features of the image or image captured by the CCTV device and the birds of the vehicle, and the learned result Intelligent CCTV system to calculate the results for the type of vehicle according to.

Images