CN103793708B - A kind of multiple dimensioned car plate precise positioning method based on motion correction - Google Patents

Abstract

The invention discloses a kind of multiple dimensioned car plate precise positioning method based on motion correction, " the car plate speckle " of candidate is detected first with car plate texture information and color characteristic, then each candidate " car plate " is carried out motion correction, then definition energy function precise positioning car plate position, finally removes noise and inverse goes back to artwork car plate position.The present invention can from the image fusion complex environment precise positioning car plate, and there is higher robustness, accuracy and high speed, the loss of car plate is close to zero.The present invention introduces motion correction the most peculiarly, solves the License Plate difficult problem brought because of image aspects well, also positions accurately for car plate and lay a good foundation.Meanwhile, the present invention defines energy function the most originally and evaluates and precise positioning car plate, and this method make use of more profound information being conducive to License Plate, makes License Plate more accurate, more stable, relatively reliable.

Description

A kind of multiple dimensioned car plate precise positioning method based on motion correction

Technical field

The invention belongs to pattern recognition and computer vision field, particularly relate to a kind of multiple dimensioned car plate based on motion correction accurate Localization method.

Background technology

Car plate detection and location technology is that computer vision, Digital Image Processing and mode identification technology are applied at intelligent transportation field One of important subject, it is widely used in road traffic monitoring, Auto Express-way Toll Collector System, parking lot management system Many traffic control systems such as system.Along with the fast development of streetscape map products in recent years, the consideration for protection privacy needs inspection Surveying positioning licence plate obfuscation, car plate detection and location technology the most more efficient, robust becomes hot research problem.

Car plate detection and location method mainly has two kinds of technology paths at present, and one is gray level image processing method based on spatial information, Another kind is color image processing method based on color characteristic.

Gray level image processing method based on spatial information.Owing to license plate area exists abundant marginal information in vertical direction, because of This can utilize this feature detection positioning licence plate.This is a kind of method based on texture information, has certain stability to illumination, But when circumstance complication, being difficult to remove the interference of non-car plate factor, effect is undesirable.

Color image processing method based on color characteristic.Car plate background color and the intrinsic collocation of characters on license plate color is utilized to detect fixed Position license plate area.This method is very big by illumination, weather, shooting angle effects, and car plate color and of a great variety, this Method has obvious limitation.

Meanwhile, some methods based on statistics and study also it has been proposed that, but efficiency and accuracy rate are difficult to balance, and missing inspection Rate and error rate are higher, are difficult to meet actual demand.Owing to the actual environment of car plate detection and location is more complicated, and there is car plate Visual angle change, illumination variation, weather conditions, motion blur etc. disturb, and car plate detection and location fast and accurately face the biggest challenge. Having multiple car plate simultaneously in partial image and picture size is relatively big, shared by car plate, regional resolution is low, there is partial occlusion, this Bring difficulty also to car plate detection and location.

Summary of the invention

In order to solve above-mentioned technical problem, energy is quickly, stable, from complex environment, detection and location go out car plate, the present invention exactly Provide a kind of multiple dimensioned car plate precise positioning method based on motion correction.

The technical solution adopted in the present invention is: a kind of multiple dimensioned car plate precise positioning method based on motion correction, its feature exists In, comprise the following steps:

Step 1: obtain original license plate image A；

Step 2: described original license plate image A is carried out rough detection, utilizes vertical texture information abundant for original license plate image A " car plate speckle " collection with strikingly color feature detection goes out candidate, then utilizes the geometric properties of original license plate image to described " the car plate speckle " of candidate tentatively filter, obtain " car plate speckle " collection B, and carry out following process；

Step 3: described " car plate speckle " collection B is carried out VLP correction, successively to each " car in " car plate speckle " collection B Board speckle " B_iCarry out motion correction, the license plate image C after being corrected_i；

Step 4: to the license plate image C after described correction_iCarry out car plate precise positioning, utilize car plate horizontal vertical gradient projection peak value, Be combined obtaining series of rectangular frame, then define energy function, by evaluate rectangle frame energy must sub-elect optimal rectangle Frame, its position just represents car plate position；

Step 5: utilize the parameter of VLP correction, obtains the position in described original license plate image A by described car plate position inverse Putting, optimal rectangle frame obtains " license plate frame " by inverse；

Step 6: judge, in " car plate speckle " collection B, whether each " car plate speckle " is disposed？

The most then perform following step 7；

If it is not, then turn round the step 3 described in performing；

Step 7: utilize non-maxima suppression method to remove part puppet car plate, in the preset range around each " license plate frame ", choosing " license plate frame " that take energy maximum is considered as car plate, and remaining " license plate frame " is considered as pseudo-car plate, is abandoned；

Step 8: result post processing, utilizes the spatial character between the inherent character of car plate itself and car plate, rejects pseudo-car plate further；

Step 9: output result.

As preferably, described in step 2, utilize vertical texture information abundant for original license plate image A and strikingly color special Levying " the car plate speckle " detecting candidate, it implements and includes following sub-step:

Step 2.1: described original license plate image A is carried out marginal information detection, utilizes the texture information pair of original license plate image A Described original license plate image A detects, and obtains car plate edge candidate's speckle figure；

Step 2.2: described original license plate image A is carried out color probability detection, obtains car plate color candidate's speckle figure；

Step 2.3: described car plate edge candidate's speckle figure and car plate color candidate's speckle figure are sought common ground；

Step 2.4: remove length-width ratio and the undesirable set of speckle of area, obtains " the car plate speckle " of candidate.

As preferably, described in step 2.1, original license plate image A is carried out marginal information detection, obtain car plate edge candidate Speckle figure, it implements and includes following sub-step:

Step 2.1.1: first described original license plate image A is carried out local enhancement, then carry out gray processing process, obtain gray scale Figure A₁；

Step 2.1.2: utilize gradient operator convolution gray-scale map A₁, try to achieve gray-scale map A₁X, Y-direction gradient map, respectively to X, Y-direction gradient map is first normalized, and then sets reasonable threshold value T₁, carry out binary conversion treatment, two can be respectively obtained Value figure A_2XAnd A_2Y, binary map A to gained_2XCarry out Gaussian smoothing and obtain marginal information density map, reject binary map Middle marginal density is less than predetermined threshold value T₂Part, the binary map after rejecting is as the initial edge of car plate edge candidate's speckle figure Testing result figure A₃；

Step 2.1.3: binary map A described in utilization_2XAnd A_2Y, carry out closing operation of mathematical morphology, then mean filter, choose suitable threshold Value T₃Carry out binary conversion treatment, after two images obtained first are carried out " with computing ", carry out morphological dilations computing, obtain new Binary map A₄, new binary map A described in utilization₄Initial edge testing result figure A described in constraint₃；

Step 2.1.4: by gray-scale map A₁Carry out edge contour detection, remove length more than preset range T₄Edge wheel profile, meter Calculate the minimum enclosed rectangle of each of the edges contour line, remove length and be wider than preset range T₅With area more than preset range T₆? Little boundary rectangle, detects edge contour the part remained, carries out closing operation of mathematical morphology, and the scope obtained regards as car plate and waits Constituency A₅, car plate candidate regions A described in utilization₅Initial edge testing result figure A described in constraint₃；

Step 2.1.5: by the initial edge testing result figure A after constraint₃, carry out closing operation of mathematical morphology, obtain car plate edge candidate Speckle figure.

As preferably, described in step 2.2, original license plate image A is carried out color probability detection, obtain car plate color candidate Speckle figure, it implements and includes following sub-step:

Step 2.2.1: utilize Color Statistical model that original license plate image A calculates such as blue, the back of the body of yellow class colors car plate respectively Scape and foreground color probability distribution graph；

Step 2.2.2: such as blue, the background of yellow class colors car plate and the foreground color probability distribution graph of gained are carried out Gauss and put down Slide and morphology operations, then choose appropriate threshold T₇Carry out binaryzation, obtain background and the prospect two of each color car plate Binary map, background and two binary map of prospect to each color car plate carry out " with computing " respectively, and its result represents often respectively Plant color license plate candidate area；

Step 2.2.3: each color license plate candidate area carries out union operation, and carries out expansion process, obtains car plate color and waits Select speckle figure.

As preferably, described in step 3, described " car plate speckle " collection B is carried out VLP correction, successively to " car plate Speckle " collect each " car plate speckle " B in B_iCarry out motion correction, the license plate image C after being corrected_i；It implements Including following sub-step:

Step 3.1: determine motion correction region；Take out " car plate speckle " B the most successively_i, calculate its minimum enclosed rectangle, by institute The minimum enclosed rectangle stated carries out region expansion, and rectangular area artwork expansion completed is taken out, and carries out gray processing process, as Affine transformation administrative division map B₁；

Step 3.2: by described affine transformation administrative division map B₁Carry out rotation transformation；By described affine transformation administrative division map B₁Carry out height This smoothing processing, removes parts of images noise, then utilizes gradient operator to carry out process of convolution, calculates every pixel X, Y side To gradient, try to achieve the gradient direction of every pixel, utilize statistics with histogram Gradient direction information, calculate gradient principal direction, root According to this principal direction to described affine transformation administrative division map B₁Carry out rotation transformation, obtain figure B₂；

Step 3.3: by described figure B₂Carry out Shear Transform；By described figure B₂, calculate its gradient master according to the method for step 3.2 Direction, then utilizes principal direction to figure B₂Shear Transform is carried out in X-direction；

Step 3.4: calculate affine transformation matrix；According to rotation transformation and Shear Transform parameter, obtain affine transformation matrix；

Step 3.5: utilize this matrix to " car plate speckle " B_iIt is corrected, the license plate image C after being corrected_i。

As preferably, described in step 4 to the license plate image C after motion correction_iCarrying out car plate precise positioning, it is specifically real Now include following sub-step:

Step 4.1: determine license plate image C_iRectangle frame combination；According to license plate image C_iX, Y-direction gradient projection peak value, really Fixed possible rectangle frame combination: first high-ranking officers license plate image C just_iGray processing, calculates its X, Y-direction gradient map, so After both horizontally and vertically projecting, statistics X, Y-direction projection peak value, combine these peak points, the point set that obtains is made For the vertex set of rectangle frame, arbitrarily choose at different 2 and may make up series of rectangular frame, choose wherein area and length-width ratio suitable Rectangle frame participates in subsequent calculations；

Step 4.2: from the beginning of first rectangle frame, defines energy function, the energy term of the rectangle frame described in calculating；Its energy term bag Include following four,

Marginal score: the computing formula of marginal score is:

score_edge=(p_x×f_x×p_y×f_y)^1/4(formula one)

Wherein, p_xRepresent horizontal gradient accurate rate or precision ratio, p_yRepresent vertical gradient accurate rate or precision ratio, f_xRepresent horizontal gradient Comprehensive evaluation index or F tolerance, f_yRepresent vertical gradient comprehensive evaluation index or F tolerance, the meter of precision ratio p and F tolerance f Calculation formula is:

$p=\frac{{\mathrm{\Σ}}_{\mathrm{Rect}}\mathrm{Gradient}}{{\mathrm{Area}}_{\mathrm{Rect}}},r=\frac{{\mathrm{\Σ}}_{\mathrm{Rect}}\mathrm{Gradient}}{{\mathrm{Area}}_{\mathrm{Rect}}+{\mathrm{Area}}_{\stackrel{\‾}{\mathrm{Rect}}}},f=2\×\frac{p\×r}{p+r}$ (formula two)

Wherein, r represents recall ratio or recall rate, ∑_RectGradient represents the gradient accumulated value in rectangle frame Rect, Area_RectTable Show the area of rectangle frame Rect,Represent the outer shared license plate image C of rectangle frame Rect_iArea；

Mixed Gauss model GMM classifies score: the computing formula of GMM score is:

${\mathrm{score}}_{\mathrm{GMM}}=1-\underset{n}{\mathrm{\Σ}}\frac{\min (P_{\mathrm{GMM}}^n\left(\stackrel{\‾}{\mathrm{Rect}}\right),P_{\mathrm{GMM}}^n\left(\mathrm{Rect}\right))}{{\mathrm{Area}}_{\mathrm{Rect}}+{\mathrm{Area}}_{\stackrel{\‾}{\mathrm{Rect}}}}$

Wherein, n represents the n-th Gaussian classification,Represent rectangle circle exterior pixel belong to the n-th class probability and,Represent pixel in rectangle circle and belong to the probability of the n-th class；

Color contrast score: the computing formula of color contrast score is:

${\mathrm{score}}_{\mathrm{cc}}={({\mathrm{score}}_{\mathrm{cc}}^1\×{\mathrm{score}}_{\mathrm{cc}}^2\×{\mathrm{score}}_{\mathrm{cc}}^3)}^{1/3}$ $\left\{\begin{array}{c}{\mathrm{score}}_{\mathrm{cc}}^1=\mathrm{dist}(H_{\mathrm{Rect}},H_{\stackrel{\‾}{\mathrm{Rect}}})\\ {\mathrm{score}}_{\mathrm{cc}}^2=1-\mathrm{dist}(H_{\mathrm{Left}},H_{\mathrm{Right}})\\ {\mathrm{score}}_{\mathrm{cc}}^3=1-\mathrm{dist}(H_{\mathrm{Top}},H_{\mathrm{Bottom}})\end{array}\right.$

Wherein, dist (H₁,H₂) represent rectangular histogram H₁And H₂Pasteur's distance, HRect is the color component of all pixels in representing rectangle frame Rectangular histogram,Represent the color component rectangular histogram of all pixels of outer rectangular frame, H_LeftRepresent in rectangle frame institute in left-half There are the color component rectangular histogram of pixel, H_RightRepresent in rectangle frame the color component rectangular histogram of all pixels in right half part, H_TopRepresent in rectangle frame the color component rectangular histogram of all pixels, H in top half_BottomRepresent the latter half in rectangle frame In the color component rectangular histogram of all pixels,Represent respectively rectangle frame in-out-snap, left and right, Color contrast score up and down；

Background and foreground color score: the computing formula of background and foreground color score is:

${\mathrm{score}}_{\mathrm{color}}^{\mathrm{bg}}=f_{\mathrm{bg}}\×{\left(p_{\mathrm{bg}}\right)}^{\mathrm{\α}}$

${\mathrm{score}}_{\mathrm{color}}^{\mathrm{fg}}=f_{\mathrm{fg}}\×{\left(p_{\mathrm{fg}}\right)}^{\mathrm{\β}}$

${\mathrm{score}}_{\mathrm{color}}={({\mathrm{score}}_{\mathrm{color}}^{\mathrm{bg}}\×{\mathrm{score}}_{\mathrm{color}}^{\mathrm{fg}})}^{1/2}$

Wherein, score_colorRepresent background and the comprehensive score of foreground color,Represent background color score,Generation Table foreground color score, p_bgRepresent background color accurate rate or precision ratio, f_bgRepresent background color comprehensive evaluation index or F degree Amount, p_fgRepresent foreground color accurate rate or precision ratio, f_fgRepresent foreground color comprehensive evaluation index or F tolerance, their meter Calculation method is with reference to formula two, α and β is nonnegative constant；

Step 4.3: the rectangle frame complex energy score described in calculating:

score_total=(score_edge×score_GMM×score_cc×score_color)^1/4

Step 4.4: judge, described license plate image C_iThe complex energy score of rectangle frame the most all calculate complete？

The most then perform following step 4.5；

If it is not, then turn round the step 4.2 described in performing；

Step 4.5: the complex energy score of each rectangle frame is ranked up, the rectangle frame selecting energy scores the highest is considered as license plate frame,

Record this rectangle frame position and score.

As preferably, the non-maxima suppression method that utilizes described in step 7 removes part puppet car plate, at each " license plate frame " In certain limit around, " license plate frame " of choosing energy maximum is considered as correct car plate, and remaining " license plate frame " is considered as pseudo-car Board, is abandoned；It implements and includes following sub-step:

Step 7.1: from the beginning of first " license plate frame ", takes out " license plate frame " one by one, calculates its minimum enclosed rectangle；

Step 7.2: this minimum enclosed rectangle is expanded preset range, at this range searching either with or without other " license plate frame "；

Step 7.3: " license plate frame " search obtained is ranked up according to its complex energy score, retains complex energy highest scoring " license plate frame ", remove " license plate frame " that complex energy score is low, " license plate frame " that remove is not involved in calculating described later；

Step 7.4: judging, described " license plate frame " is the most all disposed？

The most then perform following step 7.5；

If it is not, then turn round the step 7.1 described in performing；

Step 7.5: preserving result, flow process terminates.

As preferably, the spatial character between the inherent character of the car plate described in step 8 itself and car plate, consolidating of its car plate itself Having characteristic is the length-width ratio of car plate own, area, inclination angle scope etc., and the spatial character of its car plate itself is spatially at a distance car plate ratio The least.

Relative to prior art, the invention has the beneficial effects as follows: can from the image fusion complex environment precise positioning car plate Position, and there is higher robustness, accuracy and high speed, the loss of car plate is close to zero.When car plate coarse positioning, Make full use of the texture information that car plate is abundant so that this method has the strongest stability to illumination and weather, in conjunction with car plate distinctness Color characteristic, creative employs color probability model so that the interference of License Plate is greatly reduced by complex environment.Stricture of vagina The comprehensive stability making License Plate of reason information and colouring information and accuracy have had and have been greatly enhanced, also high degree Decrease missing inspection.The present invention introduces motion correction the most peculiarly, solves the car plate brought because of image aspects well fixed A position difficult problem, also positions accurately for car plate and lays a good foundation.

Moreover, the present invention defines energy function the most originally and evaluates and precise positioning car plate, and this method make use of More profound information being conducive to License Plate, make License Plate more accurate, more stable, relatively reliable.Edge energy Item has fully taken into account the gradient disparities of car plate and background environment；Gauss hybrid models classification energy term take into account car plate and environment Classification diversity in gauss hybrid models；Color contrast energy term both take into account the statistical difference opposite sex between car plate and environment, It is also contemplated that left-right parts, the statistical correlation of top and the bottom in car plate；Background and foreground color model energy item not only consider To the background information of car plate, the most innovatively introduce foreground model；The mutual balance of these energy terms, makes car plate detection more accurate Really, stable, reliably.

Accompanying drawing explanation

Fig. 1: for the flow chart of the inventive method embodiment.

Fig. 2: for the schematic flow sheet of the car plate rough detection step of the inventive method embodiment.

Fig. 3: for the schematic flow sheet of the car plate motion correction step of the inventive method embodiment.

Fig. 4: for the schematic flow sheet of the car plate precise positioning step of the inventive method embodiment.

Fig. 5: for the schematic flow sheet of the non-maxima suppression method pseudo-car plate step of rejecting part of the inventive method embodiment.

Detailed description of the invention

Below in conjunction with accompanying drawing and detailed description of the invention, the invention will be further described.

Referring to Fig. 1, Fig. 2, Fig. 3, Fig. 4 and Fig. 5, the technical solution adopted in the present invention is: a kind of based on motion correction Multiple dimensioned car plate precise positioning method, comprises the following steps:

Step 1: obtain original license plate image A；

Step 2: original license plate image A is carried out rough detection, utilize vertical texture information abundant for original license plate image A and Strikingly color feature detection goes out " the car plate speckle " of candidate, then utilizes the geometric properties " car to candidate of original license plate image Board speckle " tentatively filter, obtain " car plate speckle " collection B, and carry out following process；Wherein utilize original license plate image Vertical texture information that A is abundant and strikingly color feature detection go out " the car plate speckle " of candidate, its implement include following Sub-step:

Step 2.1: original license plate image A is carried out marginal information detection, utilizes the texture information of original license plate image A to former Beginning, license plate image A detected, and obtained car plate edge candidate's speckle figure；It implements and includes following sub-step:

Step 2.1.1: first original license plate image A is carried out local enhancement, then carry out gray processing process, obtain gray-scale map A₁；

Step 2.1.2: utilize gradient operator convolution gray-scale map A₁, try to achieve gray-scale map A₁X, Y-direction gradient map, the most right X, Y-direction gradient map are normalized, and then set reasonable threshold value T₁(the present embodiment is T₁=0.35), two-value is carried out Change processes, and obtains binary map A_2XAnd A_2Y, by binary map A of gained_2X, carrying out Gaussian smoothing, to obtain marginal information close Degree figure.Reject marginal density in binary map and be less than predetermined threshold value T₂(the present embodiment is T₂=0.25) part, two after rejecting Value figure is as the initial edge testing result figure A of car plate edge candidate's speckle figure₃；

Step 2.1.3: utilize binary map A_2XAnd A_2Y, carry out closing operation of mathematical morphology, then mean filter, choose certain threshold value T₃(the present embodiment is T₃=0.75), carry out morphological dilations computing after two images obtained first are carried out " with computing ", obtain New binary map A₄, utilize new binary map A₄Constraint initial edge testing result figure A₃；Generally seek common ground.

Step 2.1.4: by gray-scale map A₁Carry out edge contour detection, remove length more than preset range T₄Edge wheel profile, Calculate the minimum enclosed rectangle of each of the edges contour line, remove length and be wider than preset range T₅(the present embodiment is that width is more than 100 Or height is more than 60) and area excessive in preset range T₆The minimum enclosed rectangle of (the present embodiment is that area is more than 3000). Edge contour detecting the part remained, carries out closing operation of mathematical morphology, the scope obtained regards as car plate candidate regions A₅, profit With described car plate candidate regions A₅Initial edge testing result figure A described in constraint₃；Generally seek common ground.

Step 2.1.5: by the initial edge testing result figure A after constraint₃, carry out closing operation of mathematical morphology, obtain car plate edge Detection speckle figure.

Step 2.2: original license plate image A is carried out color probability detection, obtains car plate color candidate's speckle figure；It is specifically real Now include following sub-step:

Step 2.2.1: utilize Color Statistical model that original license plate image A calculates such as blue, the back of the body of yellow class colors car plate respectively Scape and foreground color probability distribution graph；

Step 2.2.2: such as blue, the background of yellow class colors car plate and the foreground color probability distribution graph of gained are carried out Gauss and put down Slide and morphology operations, then choose appropriate threshold T₇Carry out binaryzation, obtain background and the prospect two of each color car plate Binary map, background and two binary map of prospect to each color car plate carry out " with computing " respectively, and its result represents often respectively Plant color license plate candidate area；

Step 2.2.3: each color license plate candidate area carries out union operation, and carries out expansion process, obtains car plate color and waits Select speckle figure.

Step 2.3: car plate edge candidate's speckle figure and car plate color candidate blocks figure are sought common ground；

Step 2.4: remove length-width ratio and the undesirable set of speckle of area, obtains " the car plate speckle " of candidate.

Step 3: " car plate speckle " collection B carries out VLP correction, successively in " car plate speckle " collection B each " car plate speckle " B_iCarry out motion correction, the license plate image C after being corrected_i；It implements and includes following sub-step:

Step 3.1: determine motion correction region；Take out " car plate speckle " B the most successively_i, calculate its minimum enclosed rectangle, Only comprise candidate license plate region due to minimum enclosed rectangle, this region may be too small or inaccurate, thus need suitable Expand, minimum enclosed rectangle is carried out region expansion, expands original N to₁(the present embodiment N again₁=3), expansion is completed Rectangular area artwork C₁Take out, carry out gray processing process, as affine transformation administrative division map B₁.Meanwhile, the square after expanding is taken out Speckle figure in shape frame, is both horizontally and vertically carrying out a certain degree of morphological dilations computing according to its minimum enclosed rectangle size Obtain speckle figure M₁, in step 3 and step 4, all statistical information used and areal calculation only consider speckle figure M₁In Pixel.

Step 3.2: by affine transformation administrative division map B₁Carry out rotation transformation；By affine transformation administrative division map B₁Carry out at Gaussian smoothing Reason, removes parts of images noise, then utilizes gradient operator to carry out process of convolution, calculate every pixel X, Y-direction gradient d_x And d_y, try to achieve the gradient direction α=arctan (d of every pixel_y/d_x), utilize statistics with histogram Gradient direction information, calculate Go out gradient principal direction, according to this principal direction to affine transformation administrative division map B₁Carry out rotation transformation, obtain figure B₂, and retain rotation Matrix R_y。

Step 3.3: B will be schemed₂Carry out Shear Transform；B will be schemed₂, calculate its gradient principal direction according to the method for step 3.2, so After utilize principal direction to B₂Shear Transform is carried out in X-direction；And retain Shear Transform matrix R_x。

Step 3.4: calculate affine transformation matrix；According to rotation transformation and Shear Transform parameter, obtain affine transformation matrix M_affine=R_xR_y；

Step 3.5: utilize this matrix to " car plate speckle " B_iWith speckle figure M₁It is corrected, the car plate figure after being corrected As C_iWith speckle figure M₂.Utilize this matrix can also be by license plate image C complete for precise positioning_iInverse goes back to artwork position.

Step 4: to the license plate image C after correction_iCarry out car plate precise positioning, utilize car plate horizontal vertical gradient projection peak value, Be combined obtaining series of rectangular frame, then define energy function, by evaluate rectangle frame energy must sub-elect optimal rectangle Frame, the position of frame just represents car plate position；It implements and includes following sub-step:

Step 4.1: determine license plate image C_iRectangle frame combination；Owing to after affine transformation, license plate area is corrected into as upright Rectangular area, according to license plate image C_iX, Y-direction gradient projection peak value, determine the combination of possible rectangle frame: first will The license plate image C corrected_iGray processing, calculates its X, Y-direction gradient map, is then both horizontally and vertically projecting, Statistics X, Y-direction projection peak value, if projection peak value number is very little (less than N₂=2), returning is not car plate, if projection Peak value number is too much (more than N₃=20), then choose the front N that peak value is maximum₄Individual, generally can select 20.Combine this A little peak points, the point set obtained is considered as the vertex set of rectangle frame, arbitrarily chooses at different 2 and may make up series of rectangular frame, chooses Wherein area and the suitable rectangle frame of length-width ratio participate in subsequent calculations；

Step 4.2: from the beginning of the first rectangle frame, defines energy function, calculates the energy term of rectangle frame；Its energy term include with Lower four,

Marginal score: there is character in car plate, so having abundant edge in car plate, and car plate ambient background environment is relative to light Sliding, therefore edge lacks.Car plate and background environment is thus made to there is obvious gradient disparities, the horizontal and vertical of car plate part Edge has higher intensity in vertically and horizontally gradient map, and adding up the extraction correct to car plate of these information has positive effect.Limit The computing formula of edge score is:

score_edge=(p_x×f_x×p_y×f_y)^1/4(formula one)

Wherein, p_xRepresent horizontal gradient accurate rate or precision ratio (Precision), p_yRepresent vertical gradient accurate rate or precision ratio (Precision), f_xRepresent horizontal gradient comprehensive evaluation index or F tolerance (F-Measure), f_yRepresent that vertical gradient is comprehensively commented Valency index or F tolerance (F-Measure), the computing formula of accurate rate p and F tolerance f is:

$p=\frac{{\mathrm{\Σ}}_{\mathrm{Rect}}\mathrm{Gradient}}{{\mathrm{Area}}_{\mathrm{Rect}}},r=\frac{{\mathrm{\Σ}}_{\mathrm{Rect}}\mathrm{Gradient}}{{\mathrm{Area}}_{\mathrm{Rect}}+{\mathrm{Area}}_{\stackrel{\‾}{\mathrm{Rect}}}},f=2\×\frac{p\×r}{p+r}$ (formula two)

Wherein, r represents recall ratio or recall rate (Recall), ∑_RectGradient represents the gradient accumulated value in rectangle frame Rect, Area_RectRepresent the area of rectangle frame Rect,Represent the outer shared license plate image C of rectangle frame Rect_iArea；

Mixed Gauss model GMM classifies score: utilize gauss hybrid models classification to be separated with environment by car plate, it is only necessary to Ensureing that car plate classification is as much as possible in rectangle frame, environment classification at outer rectangular frame, obtains when misclassification probability minimum as far as possible The rectangle frame arrived is optimal.The computing formula of GMM score is:

${\mathrm{score}}_{\mathrm{GMM}}=1-\underset{n}{\mathrm{\Σ}}\frac{\min (P_{\mathrm{GMM}}^n\left(\stackrel{\‾}{\mathrm{Rect}}\right),P_{\mathrm{GMM}}^n\left(\mathrm{Rect}\right))}{{\mathrm{Area}}_{\mathrm{Rect}}+{\mathrm{Area}}_{\stackrel{\‾}{\mathrm{Rect}}}}$

Wherein, n represents the n-th Gaussian classification,Represent rectangle circle exterior pixel belong to the n-th class probability and,Represent pixel in rectangle circle and belong to the probability of the n-th class；

Color contrast score: in view of having the statistical difference opposite sex between car plate and environment, and left-right parts, top and the bottom in car plate Having statistical correlation, when disparity outer with frame in frame, with time-frame interpolation away from minimum, this rectangle frame is the most optimal.This reality Execute example principal statistical RBG and H component when statistical color information, complete by calculating histogrammic Pasteur's distance.Color The computing formula of contrast score is:

${\mathrm{score}}_{\mathrm{cc}}={({\mathrm{score}}_{\mathrm{cc}}^1\×{\mathrm{score}}_{\mathrm{cc}}^2\×{\mathrm{score}}_{\mathrm{cc}}^3)}^{1/3}$

$\left\{\begin{array}{c}{\mathrm{score}}_{\mathrm{cc}}^1=\mathrm{dist}(H_{\mathrm{Rect}},H_{\stackrel{\‾}{\mathrm{Rect}}})\\ {\mathrm{score}}_{\mathrm{cc}}^2=1-\mathrm{dist}(H_{\mathrm{Left}},H_{\mathrm{Right}})\\ {\mathrm{score}}_{\mathrm{cc}}^3=1-\mathrm{dist}(H_{\mathrm{Top}},H_{\mathrm{Bottom}})\end{array}\right.$

Wherein, dist (H₁,H₂) represent rectangular histogram H₁And H₂Pasteur (Bhattacharyya) distance, H_RectRepresent institute in rectangle frame There is the color component rectangular histogram of pixel,Represent the color component rectangular histogram of all pixels of outer rectangular frame, H_LeftRepresent rectangle The color component rectangular histogram of all pixels, H in frame left-half_RightRepresent the color of all pixels in rectangle frame right half part to divide Amount rectangular histogram, H_TopRepresent in rectangle frame the color component rectangular histogram of all pixels, H in top half_BottomRepresent rectangle frame The color component rectangular histogram of all pixels in interior the latter half,Represent respectively in rectangle circle Outward, left and right, upper and lower color contrast score.Here color component can be each component of rgb space, it is also possible to be that other are empty Between color component, if multiple component uses simultaneously, seek their geometrical mean.

Background and foreground color score: do not only have blue or yellow background in car plate, also have white or black character, introduce background Foreground model not only considers color background when evaluating rectangle frame, it is also considered that character information, so evaluates more objective and accurate.The back of the body The computing formula of scape and foreground color score is:

${\mathrm{score}}_{\mathrm{color}}^{\mathrm{bg}}=f_{\mathrm{bg}}\×{\left(p_{\mathrm{bg}}\right)}^{\mathrm{\α}}$

${\mathrm{score}}_{\mathrm{color}}^{\mathrm{fg}}=f_{\mathrm{fg}}\×{\left(p_{\mathrm{fg}}\right)}^{\mathrm{\β}}$

${\mathrm{score}}_{\mathrm{color}}={({\mathrm{score}}_{\mathrm{color}}^{\mathrm{bg}}\×{\mathrm{score}}_{\mathrm{color}}^{\mathrm{fg}})}^{1/2}$

Wherein, score_colorRepresent background and the comprehensive score of foreground color,Represent background color score,Generation Table foreground color score, p_bgRepresent background color accurate rate or precision ratio (Precision), f_bgRepresent background color overall merit Index or F tolerance (F-Measure), p_fgRepresent foreground color accurate rate or precision ratio (Precision), f_fgRepresent prospect face Color comprehensive evaluation index or F tolerance (F-Measure), their computational methods are with reference to formula two, α and β is nonnegative constant.

Step 4.3: calculating rectangle frame complex energy score:

score_total=(score_edge×score_GMM×score_cc×score_color)^1/4

Step 4.4: judge, license plate image C_iThe complex energy score of rectangle frame the most all calculate complete？

The most then perform following step 4.5；

If it is not, then revolution performs step 4.2；

Step 4.5: the complex energy score of each rectangle frame be ranked up, the rectangle frame selecting energy scores high is considered as car plate Frame, records this rectangle frame position and score.

Step 5: utilize the parameter of VLP correction, goes back to car plate position inverse to the position in original license plate image A；First check square Whether the complex energy score of shape frame is more than threshold value T₈(the present embodiment is T₈=0.2), if less than threshold value, it is considered as pseudo-car plate, deletes Removing, if more than threshold value, retaining, this is a simple denoising process.Then affine transformation matrix M is calculated_affineInverse square Battle arrayUtilize inverse matrix that car plate position is worked back to artwork position, and record.The most original " rectangle frame " Through inverse, can obtain on artwork position " license plate frame ", this " license plate frame " can be no longer rectangle.

Step 6: judge, if the license plate image B after each filtration is disposed？

The most then perform following step 7；

If it is not, then revolution performs step 3；

Step 7: utilize non-maxima suppression (Non-Maximum Suppression) method to remove part puppet car plate, often In individual " license plate frame " certain limit around, " license plate frame " of choosing energy maximum is considered as correct car plate, remaining " car plate Frame " it is considered as pseudo-car plate, abandoned；It implements and includes following sub-step:

Step 7.1: from the beginning of first " license plate frame ", take " license plate frame " one by one, calculates its minimum enclosed rectangle；

Step 7.2: this minimum enclosed rectangle is expanded preset range, and the present embodiment expands original 5 times to, at this range searching Either with or without other " license plate frame "；

Step 7.3: " license plate frame " search obtained is ranked up according to its complex energy score, retains complex energy score The highest " license plate frame ", removes " license plate frame " that complex energy score is low, and " license plate frame " that remove is not involved in calculating described later；

Step 7.4: judging, described " license plate frame " is the most all disposed？

The most then perform following step 7.5；

If it is not, then revolution performs step 7.1；

Step 7.5: preserving result, flow process terminates.

Step 8: result post processing, utilizes the spatial character between the inherent character of car plate itself and car plate to reject pseudo-car plate further. The own length-width ratio of license plate frame extracted should be close to 3:1, and area should be not excessive or too small, will not in the angle of inclination of car plate yet Too big, simultaneously from space from the point of view of, car plate at a distance can not big than nearby.By a series of post processing, one can be entered Step rejects pseudo-car plate.

Step 9: output result.Car plate testing result after constraint is drawn in artwork, is separately stored in a result picture, car Board positional information can be saved in text, may be used for other purposes.

Specific embodiment described herein is only to illustrate spirit of the present invention.The skill of the technical field of the invention Described specific embodiment can be made various amendment, supplements or use similar mode to substitute by art personnel, but will not Deviate the spirit of the present invention or surmount scope defined in appended claims.

Claims (5)

1. a multiple dimensioned car plate precise positioning method based on motion correction, it is characterised in that comprise the following steps:

Step 1: obtain original license plate image A；

Step 2: described original license plate image A is carried out rough detection, utilizes vertical texture abundant for original license plate image A Information and strikingly color feature detection go out " car plate speckle " collection of candidate, then utilize the geometric properties pair of original license plate image " the car plate speckle " of described candidate tentatively filters, and obtains " car plate speckle " collection B；

Vertical texture information abundant for original license plate image A and strikingly color feature detection is wherein utilized to go out " the car plate of candidate Speckle ", it implements and includes following sub-step:

Step 2.1: described original license plate image A is carried out marginal information detection, utilizes the texture of original license plate image A to believe Described original license plate image A is detected by breath, obtains car plate edge candidate's speckle figure；It implements and includes following son Step:

Step 2.1.1: first described original license plate image A is carried out local enhancement, then carry out gray processing process, obtain Gray-scale map A₁；

Step 2.1.2: utilize gradient operator convolution gray-scale map A₁, try to achieve gray-scale map A₁X, Y-direction gradient map, the most right X, Y-direction gradient map are first normalized, and then set reasonable threshold value T₁, carry out binary conversion treatment, can respectively obtain Binary map A_2XAnd A_2Y, binary map A to gained_2XCarry out Gaussian smoothing and obtain marginal information density map, reject two-value In figure, marginal density is less than predetermined threshold value T₂Part, the binary map after rejecting is as the initial edge of car plate edge candidate's speckle figure Edge testing result figure A₃；

Step 2.1.3: binary map A described in utilization_2XAnd A_2Y, carry out closing operation of mathematical morphology, then mean filter, choose conjunction Suitable threshold value T₃Carry out binary conversion treatment, after two images obtained first are carried out " with computing ", carry out morphological dilations computing, To new binary map A₄, new binary map A described in utilization₄Initial edge testing result figure A described in constraint₃；

Step 2.1.4: by gray-scale map A₁Carry out edge contour detection, remove length more than preset range T₄Edge wheel profile, Calculate the minimum enclosed rectangle of each of the edges contour line, remove length and be wider than preset range T₅With area more than preset range T₆'s Minimum enclosed rectangle, detects edge contour the part remained, carries out closing operation of mathematical morphology, and the scope obtained regards as car plate Candidate regions A₅, car plate candidate regions A described in utilization₅Initial edge testing result figure A described in constraint₃；

Step 2.1.5: by the initial edge testing result figure A after constraint₃, carry out closing operation of mathematical morphology, obtain car plate edge Candidate's speckle figure；

Step 2.2: described original license plate image A is carried out color probability detection, obtains car plate color candidate's speckle figure；

Step 2.3: described car plate edge candidate's speckle figure and car plate color candidate's speckle figure are sought common ground；

Step 2.4: remove length-width ratio and the undesirable set of speckle of area, obtains " the car plate speckle " of candidate；

Step 3: described " car plate speckle " collection B is carried out VLP correction, successively to each in " car plate speckle " collection B " car plate speckle " B_iCarry out motion correction, the license plate image C after being corrected_i；

Step 4: to the license plate image C after described correction_iCarry out car plate precise positioning, utilize car plate horizontal vertical gradient projection Peak value, is combined obtaining series of rectangular frame, then defines energy function, by evaluating must sub-electing of rectangle frame energy Good rectangle frame, its position just represents car plate position；

Described to the license plate image C after motion correction_iCarrying out car plate precise positioning, it implements and includes following sub-step:

Step 4.1: determine license plate image C_iRectangle frame combination；According to license plate image C_iX, Y-direction gradient projection peak value, Determine that possible rectangle frame combines: first high-ranking officers license plate image C just_iGray processing, calculates its X, Y-direction gradient map, Then both horizontally and vertically projecting, statistics X, Y-direction projection peak value, combining these peak points, the point set obtained As the vertex set of rectangle frame, arbitrarily choose at different 2 and may make up series of rectangular frame, choose wherein area and length-width ratio suitable Rectangle frame participate in subsequent calculations；

Step 4.2: from the beginning of first rectangle frame, defines energy function, the energy term of the rectangle frame described in calculating；Its energy Item includes following four,

Marginal score: the computing formula of marginal score is:

score_edge=(p_x×f_x×p_y×f_y)^1/4(formula one)

Wherein, p_xRepresent horizontal gradient accurate rate or precision ratio, p_yRepresent vertical gradient accurate rate or precision ratio, f_xExpression level Gradient comprehensive evaluation index or F tolerance, f_yRepresenting vertical gradient comprehensive evaluation index or F tolerance, precision ratio p and F measures f Computing formula be:

Wherein, r represents recall ratio or recall rate, ∑_RectGradient represents the gradient accumulated value in rectangle frame Rect, Area_Rect Represent the area of rectangle frame Rect,Represent the outer shared license plate image C of rectangle frame Rect_iArea；

Mixed Gauss model GMM classifies score: the computing formula of GMM score is:

${\mathrm{score}}_{GMM}=1-\underset{n}{\Σ}\frac{min\left(P_{GMM}^n\right(\stackrel{\‾}{\mathrm{Re}ct}),P_{GMM}^n(\stackrel{\‾}{\mathrm{Re}ct}\left)\right)}{{\mathrm{Area}}_{\mathrm{Re}ct}+{\mathrm{Area}}_{\stackrel{\‾}{\mathrm{Re}ct}}}$

Wherein, n represents the n-th Gaussian classification,Represent rectangle circle exterior pixel belong to the n-th class probability and,Represent pixel in rectangle circle and belong to the probability of the n-th class；

Color contrast score: the computing formula of color contrast score is:

${\mathrm{score}}_{cc}={({\mathrm{score}}_{cc}^1\×{\mathrm{score}}_{cc}^2\×{\mathrm{score}}_{cc}^3)}^{1/3}$

$\left\{\begin{array}{c}{\mathrm{score}}_{cc}^1=dist(H_{\mathrm{Re}ct},H_{\stackrel{\‾}{\mathrm{Re}ct}})\\ {\mathrm{score}}_{cc}^2=1-dist(H_{Left},H_{Right})\\ {\mathrm{score}}_{cc}^3=1-dist(H_{Top},H_{Bottom})\end{array}\right.$

Wherein, dist (H₁,H₂) represent rectangular histogram H₁And H₂Pasteur's distance, H_RectThe color of all pixels in representing rectangle frame Histogram of component,Represent the color component rectangular histogram of all pixels of outer rectangular frame, H_LeftRepresent left-half in rectangle frame In the color component rectangular histogram of all pixels, H_RightRepresent in rectangle frame the color component rectangular histogram of all pixels in right half part, H_TopRepresent in rectangle frame the color component rectangular histogram of all pixels, H in top half_BottomRepresent the latter half in rectangle frame In the color component rectangular histogram of all pixels,Represent respectively rectangle frame in-out-snap, left and right, Color contrast score up and down；

Background and foreground color score: the computing formula of background and foreground color score is:

${\mathrm{score}}_{color}^{bg}=f_{bg}\×{\left(p_{bg}\right)}^{\mathrm{\α}}$

${\mathrm{score}}_{color}^{fg}=f_{fg}\×{\left(p_{fg}\right)}^{\mathrm{\β}}$

${\mathrm{score}}_{color}={({\mathrm{score}}_{color}^{bg}\×{\mathrm{score}}_{color}^{fg})}^{1/2}$

Wherein, score_colorRepresent background and the comprehensive score of foreground color,Represent background color score, Represent foreground color score, p_bgRepresent background color accurate rate or precision ratio, f_bgRepresent background color comprehensive evaluation index or F Tolerance, p_fgRepresent foreground color accurate rate or precision ratio, f_fgRepresent foreground color comprehensive evaluation index or F tolerance, they Computational methods are with reference to formula two, α and β is nonnegative constant；

Step 4.3: the rectangle frame complex energy score described in calculating:

score_total=(score_edge×score_GMM×score_cc×score_color)^1/4

Step 4.4: judge, described license plate image C_iThe complex energy score of rectangle frame the most all calculate complete；

The most then perform following step 4.5；

If it is not, then turn round the step 4.2 described in performing；

Step 4.5: the complex energy score of each rectangle frame be ranked up, the rectangle frame selecting energy scores the highest is considered as car Board frame, records this rectangle frame position and score；

Step 5: utilize the parameter of VLP correction, obtains described car plate position inverse in described original license plate image A Position, optimal rectangle frame obtains " license plate frame " by inverse；

Step 6: judge, in " car plate speckle " collection B, whether each " car plate speckle " is disposed；

The most then perform following step 7；

If it is not, then turn round the step 3 described in performing；

Step 7: utilize non-maxima suppression method to remove part puppet car plate, in the preset range around each " license plate frame ", " license plate frame " of choosing energy maximum is considered as car plate, and remaining " license plate frame " is considered as pseudo-car plate, is abandoned；

Step 8: result post processing, utilizes the spatial character between the inherent character of car plate itself and car plate, rejects puppet further Car plate；

Step 9: output result.

Multiple dimensioned car plate precise positioning method based on motion correction the most according to claim 1, it is characterised in that: step Described in 2.2, original license plate image A being carried out color probability detection, obtain car plate color candidate's speckle figure, it implements Including following sub-step:

Step 2.2.1: utilize Color Statistical model that original license plate image A calculates such as blue, yellow class colors car plate respectively Background and foreground color probability distribution graph；

Step 2.2.2: such as blue, the background of yellow class colors car plate and the foreground color probability distribution graph of gained are carried out height This smooth and morphology operations, then chooses appropriate threshold T₇Carry out binaryzation, obtain background and the prospect of each color car plate Two binary map, background and two binary map of prospect to each color car plate are carried out " with computing " respectively, its result generation respectively Table each color license plate candidate area；

Step 2.2.3: each color license plate candidate area is carried out union operation, and carries out expansion process, obtain car plate face Color candidate's speckle figure.

Multiple dimensioned car plate precise positioning method based on motion correction the most according to claim 1, it is characterised in that: step The B that collects described " car plate speckle " described in 3 carries out VLP correction, successively to each " car in " car plate speckle " collection B Board speckle " B_iCarry out motion correction, the license plate image C after being corrected_i；It implements and includes following sub-step:

Step 3.1: determine motion correction region；Take out " car plate speckle " B the most successively_i, calculate its minimum enclosed rectangle, Described minimum enclosed rectangle is carried out region expansion, and rectangular area artwork expansion completed is taken out, and carries out gray processing process, As affine transformation administrative division map B₁；

Step 3.2: by described affine transformation administrative division map B₁Carry out rotation transformation；By described affine transformation administrative division map B₁Enter Row Gaussian smoothing, removes parts of images noise, then utilizes gradient operator to carry out process of convolution, calculate every pixel X, Y-direction gradient, tries to achieve the gradient direction of every pixel, utilizes statistics with histogram Gradient direction information, calculates gradient principal direction, According to this principal direction to described affine transformation administrative division map B₁Carry out rotation transformation, obtain figure B₂；

Step 3.3: by described figure B₂Carry out Shear Transform；By described figure B₂, calculate its ladder according to the method for step 3.2 Degree principal direction, then utilizes principal direction to figure B₂Shear Transform is carried out in X-direction；

Step 3.4: calculate affine transformation matrix；According to rotation transformation and Shear Transform parameter, obtain affine transformation matrix；

Step 3.5: utilize this matrix to " car plate speckle " B_iIt is corrected, the license plate image C after being corrected_i。

Multiple dimensioned car plate precise positioning method based on motion correction the most according to claim 1, it is characterised in that: step The non-maxima suppression method that utilizes described in rapid 7 removes part puppet car plate, in the certain limit around each " license plate frame ", " license plate frame " of choosing energy maximum is considered as correct car plate, and remaining " license plate frame " is considered as pseudo-car plate, is abandoned；Its tool Body realizes including following sub-step:

Step 7.1: from the beginning of first " license plate frame ", takes out " license plate frame " one by one, calculates its minimum enclosed rectangle；

Step 7.2: this minimum enclosed rectangle is expanded preset range, at this range searching either with or without other " license plate frame "；

Step 7.3: " license plate frame " search obtained is ranked up according to its complex energy score, retains complex energy score The highest " license plate frame ", removes " license plate frame " that complex energy score is low, and " license plate frame " that remove is not involved in calculating described later；

Step 7.4: judging, described " license plate frame " is the most all disposed；

The most then perform following step 7.5；

If it is not, then turn round the step 7.1 described in performing；

Step 7.5: preserving result, flow process terminates.

Multiple dimensioned car plate precise positioning method based on motion correction the most according to claim 1, it is characterised in that: step Spatial character between the inherent character of the car plate described in rapid 8 itself and car plate, the inherent character of its car plate itself is car plate itself Length-width ratio, area, inclination angle scope etc., the spatial character of its car plate itself is that spatially car plate ratio is the least at a distance.