CN113569844A - License plate detection method and device - Google Patents
License plate detection method and device Download PDFInfo
- Publication number
- CN113569844A CN113569844A CN202110732946.7A CN202110732946A CN113569844A CN 113569844 A CN113569844 A CN 113569844A CN 202110732946 A CN202110732946 A CN 202110732946A CN 113569844 A CN113569844 A CN 113569844A
- Authority
- CN
- China
- Prior art keywords
- license plate
- target
- detection model
- image
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 150
- 238000012549 training Methods 0.000 claims abstract description 54
- 238000000034 method Methods 0.000 claims abstract description 21
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 13
- 238000012545 processing Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 9
- 238000002372 labelling Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 5
- 230000000717 retained effect Effects 0.000 claims description 3
- 230000001629 suppression Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application discloses a license plate detection method and device, which are used for detecting and classifying all license plates in a vehicle image. The method in the embodiment of the application comprises the following steps: training an initial detection model to obtain a target detection model, wherein the target detection model is a convolutional neural network model and is used for detecting all license plates in an image; acquiring a target vehicle image, wherein the target vehicle image comprises at least one license plate; inputting the target vehicle image into the target detection model, and generating a target license plate image and a category probability of each image in the target license plate image, wherein the number of the target license plate images is consistent with that of the license plates, the category probability comprises the probability of each preset category, and the preset category comprises a single-row license plate and a double-row license plate; and for each image in the target license plate image, determining a preset class with the maximum probability value in the corresponding class probability as a license plate class.
Description
Technical Field
The embodiment of the application relates to the technical field of license plate detection, in particular to a method and a device for license plate detection.
Background
With the continuous discovery of the society, vehicles are also continuously increased, and in order to facilitate the management of the vehicles, a vehicle management system based on deep learning is developed, and the vehicle management system comprises a license plate detection model, a license plate recognition model and the like.
In the prior art, when a vehicle arrives, a vehicle management system acquires a vehicle image corresponding to the vehicle, then acquires a corresponding license plate image through a license plate detection model and the vehicle image, and finally acquires a corresponding license plate number through a license plate recognition model and the license plate image.
However, the license plate detection model in the prior art can only detect one of a single-line license plate or a double-line license plate, and can only detect one type of license plate when two types of license plates are hung on one vehicle at the same time.
Disclosure of Invention
The embodiment of the application provides a license plate detection method and device, which can detect different license plates hung on the same vehicle at the same time.
A first aspect of an embodiment of the present application provides a method for detecting a license plate, including:
training an initial detection model to obtain a target detection model, wherein the target detection model is a convolutional neural network model and is used for detecting all license plates in an image;
acquiring a target vehicle image, wherein the target vehicle image comprises at least one license plate;
inputting the target vehicle image into the target detection model, and generating a target license plate image and a category probability of each image in the target license plate image, wherein the number of the target license plate images is consistent with that of the license plates, the category probability comprises the probability of each preset category, and the preset category comprises a single-row license plate and a double-row license plate;
and for each image in the target license plate image, determining a preset class with the maximum probability value in the corresponding class probability as a license plate class.
Optionally, the training of the initial detection model to obtain the target detection model includes:
acquiring an initial data set, wherein the initial data set comprises a vehicle image only with a single-row license plate, a license plate image only with a double-row license plate and a vehicle image simultaneously with the single-row license plate and the double-row license plate;
labeling license plate frames in the initial data set and license plate types of the license plate frames to obtain a sample set;
performing iterative training on an initial detection model according to the sample set;
judging whether the initial detection model reaches a preset convergence condition or not;
if so, determining the initial detection model obtained by the last iteration as a target detection model; and if not, updating the network parameters of the initial detection model according to the result of the current iterative training, and performing the next iterative training on the initial detection model according to the sample set.
Optionally, the iteratively training the initial detection model according to the sample set includes:
generating a prior box and a class probability of the prior box from the sample set;
determining a positive sample and a negative sample according to the prior frame and the license plate frame, wherein the positive sample is the prior frame meeting a positive sample rule, and the negative sample is the prior frame except the positive sample;
removing part of the negative samples according to a preset selection rule to enable the proportion of the positive samples to the negative samples to be equal to a preset proportion;
and calculating a total loss value according to the reserved positive sample, the reserved negative sample and the license plate frame, wherein the total loss value comprises a positioning loss value and a classification loss value.
Optionally, the calculating a total loss value according to the retained positive and negative samples, the license plate frame and a total loss function includes:
calculating a classification loss value according to the reserved positive sample, the reserved negative sample and the license plate frame;
calculating a positioning loss value according to the positive sample and the license plate frame;
and multiplying the positioning loss value and the classification loss value by preset weights respectively and summing to obtain a total loss value.
Optionally, after the obtaining of the initial data set and before the license plate labeling of the initial data set, the method further includes:
performing data enhancement on the initial data set, wherein the data enhancement comprises at least one of random rotation, random brightness change, random Gaussian noise increase and random Gaussian blur.
Optionally, the inputting the target vehicle image into the target detection model, and the generating the target license plate image and the category probability of each image in the target license plate image includes:
inputting the target vehicle image into the target detection model to generate a plurality of predicted target license plate frames and the category probability of each frame in the predicted target license plate frames;
and performing Non-Maximum Suppression (NMS) processing on the predicted target license plate frame to obtain a target license plate image.
A second aspect of the embodiments of the present application provides a device for detecting a license plate, including:
the training unit is used for training an initial detection model to obtain a target detection model, the target detection model is a convolutional neural network model, and the target detection model is used for detecting a license plate in an image;
the system comprises an acquisition unit, a processing unit and a display unit, wherein the acquisition unit is used for acquiring a target vehicle image, and the target vehicle image comprises at least one license plate;
the generation unit is used for inputting the target vehicle image into the target detection model and generating a target license plate image and a category probability of each image in the target license plate image, wherein the number of the target license plate images is consistent with that of the license plates, the category probability comprises the probability of each preset category, and the preset category comprises a single-row license plate and a double-row license plate;
and the determining unit is used for determining a preset category with the maximum probability value in the corresponding category probability as the license plate category for each image in the target license plate image.
Optionally, the training unit includes:
the system comprises an acquisition module, a storage module and a display module, wherein the acquisition module is used for acquiring an initial data set, and the initial data set comprises a vehicle image only with a single-row license plate, a license plate image only with a double-row license plate and a vehicle image simultaneously with the single-row license plate and the double-row license plate;
the marking module is used for marking the license plate frame in the initial data set and the license plate category of the license plate frame to obtain a sample set;
the iterative training module is used for performing iterative training on the initial detection model according to the sample set;
the judging module is used for judging whether the initial detection model reaches a preset convergence condition or not;
the determining module is used for determining the initial detection model obtained by the last iteration as a target detection model when the judging module determines that the initial detection model reaches the preset convergence condition;
and the updating module is used for updating the network parameters of the initial detection model according to the result of the iterative training when the judging module determines that the initial detection model does not reach the preset convergence condition, and performing the next iterative training on the initial detection model according to the sample set.
Optionally, the iterative training module includes:
a generation submodule for generating a prior frame and a class probability of the prior frame from the sample set;
the determining submodule is used for determining a positive sample and a negative sample according to the prior frame and the license plate frame, wherein the positive sample is the prior frame meeting a positive sample rule, and the negative sample is the prior frame except the positive sample;
the removing submodule is used for removing part of the negative samples according to a preset selection rule so that the proportion of the positive samples to the negative samples is equal to a preset proportion;
and the calculation submodule is used for calculating a total loss value according to the reserved positive sample, the reserved negative sample and the license plate frame, wherein the total loss value comprises a positioning loss value and a classification loss value.
Optionally, the calculation sub-module is specifically configured to:
calculating a classification loss value according to the reserved positive sample, the reserved negative sample and the license plate frame;
calculating a positioning loss value according to the positive sample and the license plate frame;
and multiplying the positioning loss value and the classification loss value by preset weights respectively and summing to obtain a total loss value.
Optionally, the training unit further includes:
and the data enhancement unit is used for performing data enhancement on the initial data set, wherein the data enhancement comprises at least one of random rotation, random brightness change, random Gaussian noise increase and random Gaussian blur.
Optionally, the generating unit is specifically configured to:
inputting the target vehicle image into the target detection model to generate a plurality of predicted target license plate frames and the category probability of each frame in the predicted target license plate frames;
and performing NMS (network management system) processing on the predicted target license plate frame to obtain a target license plate image.
A third aspect of the embodiments of the present application provides a device for detecting a license plate, including:
the device comprises a processor, a memory, an input and output unit and a bus;
the processor is connected with the memory, the input and output unit and the bus;
the processor specifically performs the following operations:
training an initial detection model to obtain a target detection model, wherein the target detection model is a convolutional neural network model and is used for detecting all license plates in an image;
acquiring a target vehicle image, wherein the target vehicle image comprises at least one license plate;
inputting the target vehicle image into the target detection model, and generating a target license plate image and a category probability of each image in the target license plate image, wherein the number of the target license plate images is consistent with that of the license plates, the category probability comprises the probability of each preset category, and the preset category comprises a single-row license plate and a double-row license plate;
and for each image in the target license plate image, determining a preset class with the maximum probability value in the corresponding class probability as a license plate class.
The processor is further configured to perform the method of the first aspect and the alternatives of the first aspect.
A fourth aspect of embodiments of the present application provides a computer-readable storage medium having a program stored thereon, where the program, when executed on a computer, causes the computer to perform the first aspect and the method in the alternative to the first aspect.
According to the technical scheme, the embodiment of the application has the following advantages:
according to the technical scheme provided by the embodiment of the application, an initial detection model is trained to obtain a target detection model, the target detection model is a convolutional neural network model and is used for detecting all license plates in an image, and therefore the target license plate image can be input into a target monitoring model, so that a target vehicle image which is consistent with the number of the license plates in the target vehicle image and the class probability of each target vehicle image are generated. And then selecting a preset type with the maximum probability value from the type probabilities of each target vehicle image as the license plate type, wherein the preset type comprises a single-row license plate and a double-row license plate, and when the target license plate image simultaneously comprises the single-row license plate and the double-row license plate, all license plates can be detected and classified. In addition, each image in the target license plate image already contains the license plate category, so that the license plate classification operation can be omitted for subsequent license plate identification, the license plate identification time is saved, and the license plate identification efficiency is improved.
Drawings
FIG. 1 is a schematic flow chart diagram illustrating an embodiment of a license plate detection method according to an embodiment of the present disclosure;
FIG. 2 is a schematic flow chart of another embodiment of a license plate detection method in an embodiment of the present application;
FIG. 3 is a schematic structural diagram of an embodiment of a license plate detection apparatus according to the present application;
FIG. 4 is a schematic structural diagram of another embodiment of a license plate detection apparatus according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of another embodiment of a license plate detection device in an embodiment of the present application.
Detailed Description
The embodiment of the application provides a license plate detection method and device, which are used for detecting different license plates hung on the same vehicle.
The technical solutions in the present application will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The method of the present application may be applied to a server, a terminal, or other devices with logic processing capability, and the present application is not limited thereto. For convenience of description, the following description will be given taking the execution subject as a server as an example.
Referring to fig. 1, an embodiment of a license plate detection method in an embodiment of the present application includes:
101. training an initial detection model to obtain a target detection model;
in practical application, before the target detection model is used, in order to ensure the reliability of the target detection model, the server needs to train the initial detection model first, and when the initial detection model is trained to be convergent, the target detection model is obtained. The target detection model is a convolutional neural network model and is used for detecting all license plates in the image. It should be noted that the embodiments of the present application do not limit the specific category of the convolutional neural network used in the target detection model.
102. Acquiring a target vehicle image;
after the target detection model is obtained, the server can obtain the target vehicle image, and then the target detection model is used for detecting the target vehicle image. The target vehicle image is an image to be detected, and at least one license plate is included in the target vehicle image.
It should be noted that the license plate in the target vehicle image includes not only an automobile license plate, but also a motorcycle license plate and an electric vehicle license plate, and the specific details are not limited herein.
103. Inputting the target vehicle image into a target detection model, and generating a target license plate image and the class probability of each image in the target license plate image;
the server can input the target vehicle image into the target detection model, detect the target vehicle image through the target detection model, and generate the target license plate image and the category probability of each image in the target license plate image. The number of the generated target license plate images is consistent with the number of license plates in the target vehicle images, the category probability comprises the probability of each preset category, and the preset categories comprise single-row license plates and double-row license plates.
104. And for each image in the target license plate image, determining a preset class with the maximum probability value in the corresponding class probability as the license plate class.
After the server generates the target license plate image and the category probability of each image in the target license plate image, the server can compare the probabilities of all the preset categories in the category probabilities for each image in the target license plate image, and select the preset category with the largest value as the license plate category, so that the classification of the target license plate image is completed, and whether each target license plate image is a single-row license plate or a double-row license plate is determined.
In the embodiment of the application, the server firstly trains the initial detection model to obtain the target detection model, and the target detection model is a convolutional neural network model and is used for detecting all license plates in the image, so that the server can input the target license plate image into the target monitoring model to generate the target vehicle image with the number consistent with the number of the license plates in the target vehicle image and the class probability of each target vehicle image. And then selecting a preset type with the maximum probability value from the type probabilities of each target vehicle image as the license plate type, wherein the preset type comprises a single-row license plate and a double-row license plate, and when the target license plate image simultaneously comprises the single-row license plate and the double-row license plate, all license plates can be detected and classified. In addition, each image in the target license plate image already contains the license plate category, so that the license plate classification operation can be omitted for subsequent license plate identification, the license plate identification time is saved, and the license plate identification efficiency is improved.
Referring to fig. 2, another embodiment of the method for vehicle detection in the embodiment of the present application includes:
201. acquiring an initial data set;
the server can obtain an initial data set, wherein the initial data set comprises vehicle images only with single-row license plates, license plate images only with double-row license plates and vehicle images with single-row license plates and double-row license plates. Also, the initial data set may include images of motorcycle vehicles, electric vehicle vehicles, and the like, in addition to images of automobile vehicles.
202. Performing data enhancement on the initial data set;
after the server acquires the initial data set, data enhancement processing can be performed on the initial data set, and the data enhancement mode comprises at least one of random rotation, random brightness change, random Gaussian noise increase and random Gaussian blur.
203. Marking license plate frames in the initial data set and license plate types of the license plate frames to obtain a sample set;
before training the initial detection model, the server needs to label the initial data set, label the positions of all license plates in each vehicle image in the initial data set by the license plate frame, and label the license plate categories of all license plates, so as to obtain a sample set.
The server performs data enhancement processing on the initial data set to generate the initial data set under various conditions, so that a sample set under various conditions is generated, and the robustness of the target detection model can be improved by training the initial detection model through the sample set.
204. Generating a prior frame and the class probability of the prior frame according to the sample set;
the server can transmit the sample set along each layer of the initial detection model and generate feature maps with different sizes at different layers, wherein the shallow network can extract detailed information of the sample set, and the deep network can extract high-level semantic information from complex features formed by the shallow features; and generating a fixed number of prior frames with fixed size, generating scores of each prior frame belonging to different preset categories, and converting the scores into corresponding probabilities through a softmax layer, namely generating the category probabilities of the prior frames.
It should be noted that before the server propagates the sample set along each layer of the initial inspection model, the server needs to process the size of the vehicle images in the sample set to convert all the vehicle images into fixed-size images, such as 300 × 300, meeting the requirements of the initial inspection model.
205. Determining a positive sample and a negative sample according to the prior frame and the license plate frame;
the server may calculate an Intersection Over Union (IOU) of each prior frame and the license plate frame according to the generated prior frame and the pre-labeled license plate frame, then take the prior frame of which the IOU is greater than or equal to a preset threshold as a positive sample, and take the prior frame of which the IOU is less than the preset threshold as a negative sample.
206. Removing part of negative samples according to a preset selection rule to enable the proportion of the positive samples to the negative samples to be equal to a preset proportion;
in practical application, the number of the negative samples is far greater than that of the positive samples, so that the server can remove part of the negative samples according to a preset selection rule, and the proportion of the positive samples to the negative samples is equal to the preset proportion.
In the embodiment, the server removes part of the negative samples, so that the training speed of the initial detection model can be increased, the time spent on training is reduced, and the training efficiency is improved.
207. Calculating a classification loss value according to the reserved positive sample, the reserved negative sample and the license plate frame;
because each positive sample and each negative sample have corresponding class probabilities, the server can calculate the classification loss value according to the reserved positive samples and negative samples and the pre-labeled license plate frames. The specific calculation formula is as follows:
in the formula (1), LconfIs a classification loss value; i denotes a prior frame number, j denotes a pre-labeled license plate frame number, p denotes a category number, p ═ 0 denotes a background,
taking 1 indicates that the ith prior frame and the jth license plate frame IOU are larger than the threshold value at the moment, the preset category of the license plate frame is p at the moment,
the probability that the ith prior box corresponds to the preset category p is referred to.
208. Calculating a positioning loss value according to the positive sample and the license plate frame;
for the positive example, the server may calculate its location loss value from the labeled license plate frame. And for negative samples, the positioning loss calculation is not involved. The specific calculation formula is as follows:
in the formula (2), LlocIs a positioning loss value; l is a predicted value corresponding to the predicted frame, g is a label value corresponding to the license plate frame, and the predicted frame is obtained by regressing the prior frame through regression parameters; (cx, cy) coordinates of the center point of the prediction box, (w, h) are the width and height of the prediction box, respectively; n is the prior frame number matched to the license plate frame;
is an indication parameter when
When the predicted frame is the ith predicted frame, the jth license plate frame is matched, and the preset category of the license plate frame is k.
209. Respectively multiplying the positioning loss value and the classification loss value by preset weights and summing to obtain a total loss value;
for the positioning loss value and the classification loss value, the server needs to take comprehensive consideration, so that corresponding weights need to be set for the positioning loss value and the classification loss value, and then the positioning loss value and the classification loss value are added to obtain a total loss value. The specific calculation formula is as follows:
in the formula (3), L is a classification loss value, LlocTo locate the loss value, α is the assigned weight factor and N is the number of prior frames matched to the license plate frame.
210. Judging whether the initial detection model reaches a preset convergence condition, if so, executing a step 212; if not, go to step 211;
and after the iterative training of the initial detection model is finished each time, the server needs to preset a convergence condition to judge the iterative training, if the initial detection model is determined to be converged, the step 212 is executed, and if the initial detection model is not converged, the step 211 is executed. In this embodiment, the preset convergence condition of the initial detection model may be that the number of iterative training reaches a preset end number, or that the total loss values obtained by a number of consecutive iterative training of the initial detection model are all within a preset interval, which is not limited herein. Wherein, A is a preset value and is a natural number.
211. Updating the network parameters of the initial detection model according to the result of the iterative training, and then re-executing the steps 204 to 210;
if the server determines that the initial detection model does not reach the preset convergence condition in the iterative training, the server may update the network parameters of the initial detection model by using the result of the iterative training, and then re-execute steps 204 to 210 to perform the next iterative training on the initial detection model.
212. Determining an initial detection model obtained by the last iteration as a target detection model;
and if the server determines that the initial detection model reaches the preset convergence condition, determining the initial detection model obtained by the last iterative training as the target detection model.
213. Acquiring a target vehicle image;
in this embodiment, step 213 is similar to step 102 in the previous embodiment, and is not repeated here.
214. Inputting a target vehicle image into a target license plate detection model, and generating a plurality of predicted target license plate frames and a category probability of each frame in the predicted target license plate frames;
the server may input the target vehicle image into a target detection model, from which several predicted target license plate frames and a class probability for each frame are generated.
215. Performing NMS (network management system) processing on the predicted target license plate frame to obtain a target license plate image;
and the server performs NMS treatment on the predicted target license plate frames generated by the target detection model and merges the predicted target license plate frames with high coincidence degree so as to obtain the target license plate image.
216. And for each image in the target license plate image, determining a preset class with the maximum probability value in the corresponding class probability as the license plate class.
In this embodiment, step 216 is similar to step 104 in the previous embodiment, and is not described herein again.
The above describes a license plate detection method in the embodiment of the present application, and a license plate detection device in the embodiment of the present application is described below.
Referring to fig. 3, an embodiment of a license plate detection apparatus in an embodiment of the present application includes:
the training unit 301 is configured to train an initial detection model to obtain a target detection model, where the target detection model is a convolutional neural network model and is used to detect a license plate in an image;
an obtaining unit 302, configured to obtain a target vehicle image, where the target vehicle image includes at least one license plate;
a generating unit 303, configured to input a target vehicle image into a target detection model, and generate a target license plate image and a category probability of each image in the target license plate image, where the number of the target license plate images is consistent with that of license plates, the category probability includes a probability of each preset category, and the preset category includes a single-row license plate and a double-row license plate;
the determining unit 304 is configured to determine, for each image in the target license plate image, a preset category with a highest probability value among the corresponding category probabilities as the license plate category.
In this embodiment, the functions of the units correspond to the steps in the embodiment shown in fig. 1, and are not described herein again.
Referring to fig. 4, another embodiment of the license plate detection apparatus in the embodiment of the present application includes:
a training unit 401, configured to train an initial detection model to obtain a target detection model, where the target detection model is a convolutional neural network model and is used to detect a license plate in an image;
an obtaining unit 402, configured to obtain a target vehicle image, where the target vehicle image includes at least one license plate;
a generating unit 403, configured to input a target vehicle image into a target detection model, and generate a target license plate image and a category probability of each image in the target license plate image, where the number of the target license plate images is consistent with that of license plates, the category probability includes a probability of each preset category, and the preset category includes a single-row license plate and a double-row license plate;
the determining unit 404 is configured to determine, as to each of the target license plate images, a preset category with a highest probability value among the corresponding category probabilities as a license plate category.
In this embodiment, the training unit 401 may include an obtaining module 4011, a labeling module 4012, an iterative training module 4013, a judging module 4014, a determining module 4015, an updating module 4016, and a data enhancing module 4017.
The obtaining module 4011 is configured to obtain an initial data set, where the initial data set includes a vehicle image with only a single-row license plate, a license plate image with only a double-row license plate, and a vehicle image with both a single-row license plate and a double-row license plate.
And the labeling module 4012 is configured to label the license plate frame in the initial data set and the license plate category of the license plate frame to obtain a sample set.
The iterative training module 4013 comprises a generation sub-module 40131, a determination sub-module 40132, a removal sub-module 40133, and a calculation sub-module 40134.
The generating sub-module 40131 is configured to generate the prior frame and the class probability of the prior frame according to the sample set.
A determination submodule 40132, configured to determine a positive sample and a negative sample according to the prior frame and the license plate frame, where the positive sample is the prior frame satisfying the positive sample rule, and the negative sample is the prior frame other than the positive sample.
The removing sub-module 40133 is configured to remove a part of the negative samples according to a preset selection rule, so that a ratio of the positive samples to the negative samples is equal to a preset ratio.
The calculation sub-module 40134 is configured to calculate a classification loss value according to the retained positive and negative samples and the license plate frame; calculating a positioning loss value according to the positive sample and the license plate frame; and multiplying the positioning loss value and the classification loss value by preset weights respectively and summing to obtain a total loss value.
The determining module 4014 is configured to determine whether the initial detection model reaches a preset convergence condition.
The determining module 4015 is configured to determine, when the determining module 4014 determines that the initial detection model reaches the preset convergence condition, that the initial detection model obtained through the last iteration is the target detection model.
And the updating module 4016 is configured to update the network parameters of the initial detection model according to the result of the iterative training when the judging module 4014 determines that the initial detection model does not reach the preset convergence condition, and perform the next iterative training on the initial detection model according to the sample set.
And the data enhancement module 4017 is configured to perform data enhancement on the initial data set, where the data enhancement includes at least one of random rotation, random brightness change, random gaussian noise increase, and random gaussian blur.
In this embodiment, the generating unit 403 may specifically be configured to:
inputting the target vehicle image into a target detection model, and generating a plurality of predicted target license plate frames and the class probability of each frame in the predicted target license plate frames;
and performing non-maximum value suppression NMS processing on the predicted target license plate frame to obtain a target license plate image.
In this embodiment, the functions of each unit and each module correspond to the steps in the embodiment shown in fig. 2, and are not described herein again.
Referring to fig. 5, another embodiment of the license plate detection apparatus in the embodiment of the present application includes:
a processor 501, a memory 502, an input-output unit 503, and a bus 504;
the processor 501 is connected with the memory 502, the input/output unit 503 and the bus 504;
the processor 501 specifically performs the following operations:
training an initial detection model to obtain a target detection model, wherein the target detection model is a convolutional neural network model and is used for detecting all license plates in an image;
acquiring a target vehicle image, wherein the target vehicle image comprises at least one license plate;
inputting a target vehicle image into a target detection model, and generating a target license plate image and a category probability of each image in the target license plate image, wherein the number of the target license plate images is consistent with that of license plates, the category probability comprises the probability of each preset category, and the preset category comprises a single-row license plate and a double-row license plate;
and for each image in the target license plate image, determining a preset class with the maximum probability value in the corresponding class probability as the license plate class.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like.
Claims (10)
1. A method for detecting a license plate, comprising:
training an initial detection model to obtain a target detection model, wherein the target detection model is a convolutional neural network model and is used for detecting all license plates in an image;
acquiring a target vehicle image, wherein the target vehicle image comprises at least one license plate;
inputting the target vehicle image into the target detection model, and generating a target license plate image and a category probability of each image in the target license plate image, wherein the number of the target license plate images is consistent with that of the license plates, the category probability comprises the probability of each preset category, and the preset category comprises a single-row license plate and a double-row license plate;
and for each image in the target license plate image, determining a preset class with the maximum probability value in the corresponding class probability as a license plate class.
2. The method of claim 1, wherein training the initial detection model to obtain the target detection model comprises:
acquiring an initial data set, wherein the initial data set comprises a vehicle image only with a single-row license plate, a license plate image only with a double-row license plate and a vehicle image simultaneously with the single-row license plate and the double-row license plate;
labeling license plate frames in the initial data set and license plate types of the license plate frames to obtain a sample set;
performing iterative training on an initial detection model according to the sample set;
judging whether the initial detection model reaches a preset convergence condition or not;
if so, determining the initial detection model obtained by the last iteration as a target detection model; and if not, updating the network parameters of the initial detection model according to the result of the current iterative training, and performing the next iterative training on the initial detection model according to the sample set.
3. The method of claim 2, wherein iteratively training an initial detection model from the sample set comprises:
generating a prior box and a class probability of the prior box from the sample set;
determining a positive sample and a negative sample according to the prior frame and the license plate frame, wherein the positive sample is the prior frame meeting a positive sample rule, and the negative sample is the prior frame except the positive sample;
removing part of the negative samples according to a preset selection rule to enable the proportion of the positive samples to the negative samples to be equal to a preset proportion;
and calculating a total loss value according to the reserved positive sample, the reserved negative sample and the license plate frame, wherein the total loss value comprises a positioning loss value and a classification loss value.
4. The method of claim 3, wherein the calculating a total loss value from the retained positive and negative examples, the license plate frame, and a total loss function comprises:
calculating a classification loss value according to the reserved positive sample, the reserved negative sample and the license plate frame;
calculating a positioning loss value according to the positive sample and the license plate frame;
and multiplying the positioning loss value and the classification loss value by preset weights respectively and summing to obtain a total loss value.
5. The method of any one of claims 2 to 4, wherein after the obtaining the initial data set and before the license plate labeling of the initial data set, the method further comprises:
performing data enhancement on the initial data set, wherein the data enhancement comprises at least one of random rotation, random brightness change, random Gaussian noise increase and random Gaussian blur.
6. The method of claim 5, wherein inputting the target vehicle image into the target detection model, and wherein generating a target license plate image and a class probability for each of the target license plate images comprises:
inputting the target vehicle image into the target detection model to generate a plurality of predicted target license plate frames and the category probability of each frame in the predicted target license plate frames;
and performing non-maximum value suppression NMS processing on the predicted target license plate frame to obtain a target license plate image.
7. A device for detecting a license plate, comprising:
the training unit is used for training an initial detection model to obtain a target detection model, the target detection model is a convolutional neural network model, and the target detection model is used for detecting a license plate in an image;
the system comprises an acquisition unit, a processing unit and a display unit, wherein the acquisition unit is used for acquiring a target vehicle image, and the target vehicle image comprises at least one license plate;
the generation unit is used for inputting the target vehicle image into the target detection model and generating a target license plate image and a category probability of each image in the target license plate image, wherein the number of the target license plate images is consistent with that of the license plates, the category probability comprises the probability of each preset category, and the preset category comprises a single-row license plate and a double-row license plate;
and the determining unit is used for determining a preset category with the maximum probability value in the corresponding category probability as the license plate category for each image in the target license plate image.
8. The apparatus of claim 7, wherein the training unit comprises:
the system comprises an acquisition module, a storage module and a display module, wherein the acquisition module is used for acquiring an initial data set, and the initial data set comprises a vehicle image only with a single-row license plate, a license plate image only with a double-row license plate and a vehicle image simultaneously with the single-row license plate and the double-row license plate;
the marking module is used for marking the license plate frame in the initial data set and the license plate category of the license plate frame to obtain a sample set;
the iterative training module is used for performing iterative training on the initial detection model according to the sample set;
the judging module is used for judging whether the initial detection model reaches a preset convergence condition or not;
the determining module is used for determining the initial detection model obtained by the last iteration as a target detection model when the judging module determines that the initial detection model reaches the preset convergence condition;
and the updating module is used for updating the network parameters of the initial detection model according to the result of the iterative training when the judging module determines that the initial detection model does not reach the preset convergence condition, and performing the next iterative training on the initial detection model according to the sample set.
9. The apparatus of claim 8, wherein the iterative training module comprises:
a generation submodule for generating a prior frame and a class probability of the prior frame from the sample set;
the determining submodule is used for determining a positive sample and a negative sample according to the prior frame and the license plate frame, wherein the positive sample is the prior frame meeting a positive sample rule, and the negative sample is the prior frame except the positive sample;
the removing submodule is used for removing part of the negative samples according to a preset selection rule so that the proportion of the positive samples to the negative samples is equal to a preset proportion;
and the calculation submodule is used for calculating a total loss value according to the reserved positive sample, the reserved negative sample and the license plate frame, wherein the total loss value comprises a positioning loss value and a classification loss value.
10. The apparatus of claim 9, wherein the computation submodule is specifically configured to:
calculating a classification loss value according to the reserved positive sample, the reserved negative sample and the license plate frame;
calculating a positioning loss value according to the positive sample and the license plate frame;
and multiplying the positioning loss value and the classification loss value by preset weights respectively and summing to obtain a total loss value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110732946.7A CN113569844A (en) | 2021-06-29 | 2021-06-29 | License plate detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110732946.7A CN113569844A (en) | 2021-06-29 | 2021-06-29 | License plate detection method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113569844A true CN113569844A (en) | 2021-10-29 |
Family
ID=78163144
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110732946.7A Pending CN113569844A (en) | 2021-06-29 | 2021-06-29 | License plate detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113569844A (en) |
Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104809443A (en) * | 2015-05-05 | 2015-07-29 | 上海交通大学 | Convolutional neural network-based license plate detection method and system |
| CN105913058A (en) * | 2016-04-05 | 2016-08-31 | 浙江工业大学 | Method for positioning multiple license plates |
| CN108446696A (en) * | 2018-02-09 | 2018-08-24 | 杭州雄迈集成电路技术有限公司 | A kind of end-to-end licence plate recognition method based on deep learning |
| US20180253618A1 (en) * | 2016-06-24 | 2018-09-06 | Ping An Technology (Shenzhen) Co., Ltd. | Method, system, electronic device, and medium for classifying license plates based on deep learning |
| CN109670458A (en) * | 2018-12-21 | 2019-04-23 | 北京市商汤科技开发有限公司 | A kind of licence plate recognition method and device |
| US20190147304A1 (en) * | 2017-11-14 | 2019-05-16 | Adobe Inc. | Font recognition by dynamically weighting multiple deep learning neural networks |
| US20190251369A1 (en) * | 2018-02-11 | 2019-08-15 | Ilya Popov | License plate detection and recognition system |
| CN110163199A (en) * | 2018-09-30 | 2019-08-23 | 腾讯科技（深圳）有限公司 | Licence plate recognition method, license plate recognition device, car license recognition equipment and medium |
| KR102089298B1 (en) * | 2019-10-21 | 2020-03-16 | 가천대학교 산학협력단 | System and method for recognizing multinational license plate through generalized character sequence detection |
| CN110889323A (en) * | 2019-10-10 | 2020-03-17 | 平安科技（深圳）有限公司 | Universal license plate recognition method and device, computer equipment and storage medium |
| CN110942071A (en) * | 2019-12-09 | 2020-03-31 | 上海眼控科技股份有限公司 | License plate recognition method based on license plate classification and LSTM |
| CN111353500A (en) * | 2020-02-25 | 2020-06-30 | 上海其高电子科技有限公司 | Automatic identification method for double-row license plate |
| CN111797829A (en) * | 2020-06-24 | 2020-10-20 | 浙江大华技术股份有限公司 | License plate detection method and device, electronic equipment and storage medium |
| CN112132222A (en) * | 2020-09-27 | 2020-12-25 | 上海高德威智能交通系统有限公司 | License plate category identification method and device and storage medium |
| CN112215222A (en) * | 2020-10-12 | 2021-01-12 | 上海眼控科技股份有限公司 | License plate recognition method, device, equipment and storage medium |
| CN112348044A (en) * | 2019-08-09 | 2021-02-09 | 上海高德威智能交通系统有限公司 | License plate detection method, device and equipment |
| CN112580536A (en) * | 2020-12-23 | 2021-03-30 | 深圳市捷顺科技实业股份有限公司 | High-order video vehicle and license plate detection method and device |
| CN112950621A (en) * | 2021-03-29 | 2021-06-11 | 苏州科达科技股份有限公司 | Image processing method, apparatus, device and medium |
| CN113011423A (en) * | 2019-12-21 | 2021-06-22 | 北京师范大学珠海分校 | Text line structure optimization calculation method based on CTPN system and application thereof |
-
2021
- 2021-06-29 CN CN202110732946.7A patent/CN113569844A/en active Pending
Patent Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104809443A (en) * | 2015-05-05 | 2015-07-29 | 上海交通大学 | Convolutional neural network-based license plate detection method and system |
| CN105913058A (en) * | 2016-04-05 | 2016-08-31 | 浙江工业大学 | Method for positioning multiple license plates |
| US20180253618A1 (en) * | 2016-06-24 | 2018-09-06 | Ping An Technology (Shenzhen) Co., Ltd. | Method, system, electronic device, and medium for classifying license plates based on deep learning |
| US20190147304A1 (en) * | 2017-11-14 | 2019-05-16 | Adobe Inc. | Font recognition by dynamically weighting multiple deep learning neural networks |
| CN108446696A (en) * | 2018-02-09 | 2018-08-24 | 杭州雄迈集成电路技术有限公司 | A kind of end-to-end licence plate recognition method based on deep learning |
| US20190251369A1 (en) * | 2018-02-11 | 2019-08-15 | Ilya Popov | License plate detection and recognition system |
| CN110163199A (en) * | 2018-09-30 | 2019-08-23 | 腾讯科技（深圳）有限公司 | Licence plate recognition method, license plate recognition device, car license recognition equipment and medium |
| CN109670458A (en) * | 2018-12-21 | 2019-04-23 | 北京市商汤科技开发有限公司 | A kind of licence plate recognition method and device |
| CN112348044A (en) * | 2019-08-09 | 2021-02-09 | 上海高德威智能交通系统有限公司 | License plate detection method, device and equipment |
| CN110889323A (en) * | 2019-10-10 | 2020-03-17 | 平安科技（深圳）有限公司 | Universal license plate recognition method and device, computer equipment and storage medium |
| KR102089298B1 (en) * | 2019-10-21 | 2020-03-16 | 가천대학교 산학협력단 | System and method for recognizing multinational license plate through generalized character sequence detection |
| CN110942071A (en) * | 2019-12-09 | 2020-03-31 | 上海眼控科技股份有限公司 | License plate recognition method based on license plate classification and LSTM |
| CN113011423A (en) * | 2019-12-21 | 2021-06-22 | 北京师范大学珠海分校 | Text line structure optimization calculation method based on CTPN system and application thereof |
| CN111353500A (en) * | 2020-02-25 | 2020-06-30 | 上海其高电子科技有限公司 | Automatic identification method for double-row license plate |
| CN111797829A (en) * | 2020-06-24 | 2020-10-20 | 浙江大华技术股份有限公司 | License plate detection method and device, electronic equipment and storage medium |
| CN112132222A (en) * | 2020-09-27 | 2020-12-25 | 上海高德威智能交通系统有限公司 | License plate category identification method and device and storage medium |
| CN112215222A (en) * | 2020-10-12 | 2021-01-12 | 上海眼控科技股份有限公司 | License plate recognition method, device, equipment and storage medium |
| CN112580536A (en) * | 2020-12-23 | 2021-03-30 | 深圳市捷顺科技实业股份有限公司 | High-order video vehicle and license plate detection method and device |
| CN112950621A (en) * | 2021-03-29 | 2021-06-11 | 苏州科达科技股份有限公司 | Image processing method, apparatus, device and medium |
Non-Patent Citations (4)
| Title |
|---|
| 张利辉, 韩莉, 高庆吉, 徐海军: "多种字符混合图像的自动识别", 东北电力学院学报, no. 04, pages 43 - 46 * |
| 董洪义: "《深度学习之PyTorch物体检测实战》", 机械工业出版社, pages: 175 - 176 * |
| 郭克友;贾海晶;郭晓丽;: "卷积神经网络在车牌分类器中的应用", 计算机工程与应用, no. 14, pages 214 - 218 * |
| 马永杰;陈振宇;马芸婷;: "基于自适应阈值投影的双行车牌分割方法", 计算机工程与科学, no. 09, pages 99 - 107 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108388879B (en) | Target detection method, device and storage medium | |
| WO2019051941A1 (en) | Method, apparatus and device for identifying vehicle type, and computer-readable storage medium | |
| CN109413023A (en) | The training of machine recognition model and machine identification method, device, electronic equipment | |
| CN111797821A (en) | Text detection method and device, electronic equipment and computer storage medium | |
| CN111539425A (en) | License plate recognition method, storage medium and electronic equipment | |
| CN114266894A (en) | Image segmentation method and device, electronic equipment and storage medium | |
| CN114943750A (en) | Target tracking method and device and electronic equipment | |
| CN114972316A (en) | Battery case end surface defect real-time detection method based on improved YOLOv5 | |
| CN111275694B (en) | Attention mechanism guided progressive human body division analysis system and method | |
| CN112200772A (en) | Pox check out test set | |
| CN107886093B (en) | Character detection method, system, equipment and computer storage medium | |
| CN113569844A (en) | License plate detection method and device | |
| CN113051958A (en) | Driver state detection method, system, device and medium based on deep learning | |
| CN116580232A (en) | Automatic image labeling method and system and electronic equipment | |
| CN112801045B (en) | Text region detection method, electronic equipment and computer storage medium | |
| CN111815658B (en) | Image recognition method and device | |
| CN113887300A (en) | Method and device for detecting target, human face and human face key point and storage medium | |
| CN113469176A (en) | Target detection model training method, target detection method and related equipment thereof | |
| CN115424250A (en) | License plate recognition method and device | |
| CN115100419B (en) | Target detection method and device, electronic equipment and storage medium | |
| CN118314424B (en) | Vehicle-road collaborative self-advancing learning multi-mode verification method based on edge scene | |
| CN114565889B (en) | Method and device for determining vehicle line pressing state, electronic equipment and medium | |
| CN113051959B (en) | Deep learning-based driver state detection method, system, equipment and medium | |
| CN112633243B (en) | Information identification method, device, equipment and computer storage medium | |
| CN116206282A (en) | Data processing method, device, equipment and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |