CN112085024A - A method for character recognition on the surface of a tank - Google Patents

Abstract

The invention provides a method for identifying characters on the surface of a tank, which comprises the following steps: acquiring a tank surface image in real time; carrying out character positioning on the tank surface image acquired in real time by using a pre-established YOLO-V3 neural network detection model to obtain whether the tank surface image contains characters and character position information; outputting the position of the character prediction box, and cutting out a character area; the cut character area has the problem that the fit with the actual text area is not tight enough, and the correction is carried out by using a text correction algorithm; segmenting text characters of each line through histogram projection; identifying the cut text region by using an improved end-to-end indefinite length text CRNN model; according to the system condition at the time, if the mode is an online mode, the recognition result is uploaded to a database; if the mode is an off-line mode, the recognition result is stored locally. The method has the advantages of high robustness, high accuracy and high speed, and provides an intelligent character positioning detection scheme for milk powder tank production enterprises.

Description

A method for character recognition on the surface of a tank

technical field

The invention relates to the field of character recognition on automatic production lines, in particular to a character recognition method based on YOLO-V3 and improved CRNN.

Background technique

In the process of actual industrial production, character recognition technology is a very important aspect. Character recognition technology has been successfully applied to industrial production and assembly, such as character printing on the surface of many small electronic components, circuit boards and some large parts. And labels, manufacturers can identify and track product information through these codes and labels. Since it is difficult for the human eye to achieve high-efficiency detection for a long time, some harsh operating environments such as high temperature and high pressure have brought difficulties to manual detection and recognition. Therefore, the use of machine vision technology for automatic and accurate character positioning and recognition has become an industrial production process. an important part of the process.

Image processing methods based on contour detection and edge processing can also extract character regions, but each type of image requires targeted layout analysis, and the selection of contours is also greatly affected by the image background, so its generalization The ability is not strong. Using the deep convolutional neural network to locate and extract the text information in the layout can locate and extract the text area of various documents, so that the applicable scope of character recognition and localization is wider, and the image recognition of complex backgrounds is stronger.

The current OCR identification methods used in the industry mainly have the following problems:

1) Using traditional algorithms to extract character regions and segment character texts has poor adaptability, and it is difficult to accurately extract images with uneven brightness, character sticking, and character blurring;

2) It is necessary to manually design a feature extraction method for the extracted characters, and the generality of detecting new fonts is poor. Therefore, a method for character location and recognition based on YOLO-V3 and CRNN is provided.

SUMMARY OF THE INVENTION

In order to achieve the above purpose, the present invention provides a method for recognizing characters on the surface of a tank, which includes: collecting an image of the tank surface in real time; using a pre-established YOLO-V3 neural network detection model to perform character positioning on the image of the tank surface collected in real time, and obtaining information including whether it contains Character and character position information; output the position of the character prediction frame, and crop out the character area; the cropped character area is not close enough to the actual text area, use the text correction correction algorithm to correct it; through histogram projection, segment out Text characters of each line; use the improved end-to-end variable-length text CRNN model to identify the cropped text area; according to the system conditions at the time, if it is in online mode, the recognition result will be uploaded to the database; if it is in offline mode, the recognition result will be Save locally.

In the above solution, the method for recognizing characters on the surface of the tank further includes: the actuator switches to the next workpiece to be recognized.

In the above solution, the surface of the can includes: the surface of the milk powder can and the surface of the can.

In the above scheme, the establishment method of the YOLO-V3 neural network detection model includes:

S1: Collect a large number of character images on the surface of milk powder cans;

S2: Preprocessing the surface image of the milk powder can; the preprocessing includes normalizing the surface image of the milk powder can: filling the proportionally scaled surface image of the milk powder can into a blank image with a preset pixel size, and then adding the blank image All pixels except the image of the milk powder can are filled with the set color; the pixel size of the blank image is 416*416, and the blank image can cover the zoomed image of the milk powder can; the set color includes gray;

S3: Perform character area calibration on the preprocessed image to obtain character position labels of each image; perform character area calibration on the preprocessed milk powder can image by manual frame selection, and set the upper left corner and lower right corner of each calibration area Point coordinates and characters in the corresponding area are stored as character labels;

S4: Divide the image and its position information labels into training set samples and test set samples, with a division ratio of 9:1; use the training set samples and test machine samples, and use the computer graphics processor GPU to analyze the pre-built YOLO-V3 deep neural network. The network is trained to obtain the YOLO-V3 neural network detection model.

In the above solution, the preprocessing also includes data enhancement on the character images on the tank surface obtained after normalization; the data enhancement on the character images on the tank surface includes:

Geometric transformation class: including flipping, rotating, cropping, deforming, scaling;

Color transformation class: including noise, blur, color transformation, erase, fill;

Generate a sample image based on the image style simulation.

In the above scheme, training the pre-built YOLO-V3 neural network includes:

S1: Set the training parameters: set the number of iteration steps epochs to 10000, set the learning rate optimizer optimizer to 'adam', set the number of batch training samples batch_size to 64, and set the size of the 9 a priori box anchor boxes;

S2: Train the YOLO-V3 neural network: input the training set sample data and the corresponding character data on the surface of the milk powder can into the YOLO-V3 neural network for model training;

S3: Model test: After each training, use the test set samples to test the trained model. If the model has a defect detection rate of more than 95% in the test set samples and the detection accuracy is not less than 95%, the model will be tested. The model obtained from the last training is used as the final YOLO-V3 neural network detection model; otherwise, the model obtained from the last training is used as the current YOLO-V3 neural network to be trained, and steps S2-S3 are repeated until the final YOLO-V3 neural network detection model is obtained. Model.

In the above scheme, the size of the prior frame is set by the k-means clustering method, including: randomly selecting 9 character area rectangular frame objects as the clustering center of the training set, and according to the distance between each object and each clustering center, Allocate each object to the nearest cluster center, each time a sample is assigned, the cluster center is recalculated once, and the assignment process is repeated until no cluster center changes. The current values of the nine cluster centers are nine priors. the size of the box.

In the above scheme, the text correction algorithm includes:

S1: The cropped image is grayscaled, the characters are white, and the background is colored;

S2: Perform Gaussian blurring so that the text parts can be connected together, and then perform threshold processing on the image to obtain a binary image;

S3: Use the smallest rectangle to fit, and then filter the area of the text area by the size of the rectangle area, which is the smallest bounding rectangle of the text;

S4: Obtain the center point of the minimum circumscribed rectangle, and perform affine transformation on the text area to correct the oblique text.

In the above scheme, the histogram horizontal projection algorithm includes:

S1: Binarize the corrected image obtained in step 4;

S2: Calculate the pixel sum in the horizontal direction for the binarized image, and draw a histogram with the horizontal axis as the image and the vertical axis as the coordinate range from 0 to the image width, and the vertical axis as the pixel sum;

S3: Find the bottom part of the histogram, which is the dividing line;

S4: According to the dividing line, each line of text image is divided.

In the above scheme, the improved end-to-end variable-length text CRNN model for recognition includes:

S1: Grayscale processing of each line of text image, converting the three-channel image into a single-channel grayscale image;

S2: Length scaling and padding, fixing the size of the input image to 32*width, scaling or expanding proportionally, so that it can be similar to the training samples;

S3: Data label sparse matrix conversion, process the label matrix and convert it into the required data format;

S4: Build the model, change the structure, change the original idea of CNN+RNN+CTC transcription layer to CNN+CTC, and modify the backbone network on VGG to design a 7-layer convolutional layer and a 5-layer pooling layer. Small convolution Neural network, and adding batch regularization twice in the middle layer to avoid model gradient dispersion and accelerate convergence;

S5: Data post-processing, transform the subscript of the dictionary array corresponding to the real value into the actual value by doing a mapping transformation.

The method for recognizing characters on the surface of a tank provided by the present invention realizes real-time recognition of text characters by using the YOLO-V3 and CRNN deep learning method, and can quickly frame and select the area where the characters are located, and can recognize them. Because of the use of deep learning, this method has the advantages of high robustness, high accuracy and fast speed. It provides a detection solution for intelligent character positioning for milk powder can manufacturers, and solves the problems of low manual recognition efficiency, slow speed and high personnel cost. .

Description of drawings

Fig. 1 is the schematic flow chart of the character recognition process on the surface of the milk powder can provided by the embodiment of the present invention;

Fig. 2 is a YOLO-V3 training process diagram provided by an embodiment of the present invention;

FIG. 3 is a partial network structure diagram of the improved CRNN provided by an embodiment of the present invention.

Detailed ways

In order to be able to understand the features and technical content of the present invention in more detail, the implementation of the present invention is described in detail below with reference to the accompanying drawings, which are for reference only and are not intended to limit the present invention.

The present invention provides a method for recognizing characters on the surface of a can, as shown in FIG. 1 , which is a schematic diagram of a process for recognizing characters on the surface of a milk powder can provided by an embodiment of the present invention; the character recognition on the surface of the can includes: collecting images of the surface of the can in real time; The surface images of the milk powder cans are taken from as many angles as possible, such as cameras, to cover characters from various angles and different characters, so that the detection rate and accuracy of the trained model are relatively high.

Use the pre-established YOLO-V3 neural network detection model to locate the characters on the surface image of the tank collected in real time, and obtain the information including whether it contains characters and the position of the characters; output the position of the character prediction frame, and crop out the character area; the cropped character area exists The problem of not fitting closely with the actual text area is corrected by using a text correction correction algorithm; the text characters of each line are segmented through histogram projection; the cropped text area is processed using an improved end-to-end variable-length text CRNN model. Recognition; according to the system conditions at the time, if it is in online mode, the recognition result will be uploaded to the database; if it is in offline mode, the recognition result will be saved locally.

Wherein, the method for recognizing characters on the surface of the tank further includes: the actuator switches over the next workpiece to be recognized.

In the embodiment provided by the present invention, the surface of the can includes: the surface of the milk powder can, the surface of the can, and the surface of other cans.

In the embodiment provided by the present invention, the establishment method of the YOLO-V3 neural network detection model includes:

S1: Collect a large number of character images on the surface of milk powder cans;

S2: Preprocessing the surface image of the milk powder can; the preprocessing includes normalizing the surface image of the milk powder can: filling the proportionally scaled surface image of the milk powder can into a blank image with a preset pixel size, and then adding the blank image All pixels except the image of the milk powder can are filled with the set color; the pixel size of the blank image is 416*416, and the blank image can cover the zoomed image of the milk powder can; the normalization process is scaled to the corresponding image according to the original aspect ratio of the image. Size, fill in the middle of the 416*416 blank image, and fill all other pixels with white, so as to keep the original texture characteristics of the image as much as possible, and use the flipping and translation strategies to enhance the normalized defect image and set the color. including grey;

S3: Perform character area calibration on the preprocessed image to obtain character position labels of each image; perform character area calibration on the preprocessed milk powder can image by manual frame selection, and set the upper left corner and lower right corner of each calibration area The point coordinates and the characters in the corresponding area are stored as character labels; in the image calibration stage, the production of character image labels includes: labeling software can be used to manually demarcate the character area with a rectangular frame; The ordinate is stored in the txt file, and the labeling software can adopt existing graphics drawing modification software, labelImg and other software, only for the convenience of automatically or artificially obtaining the corresponding coordinate data after manual frame selection.

S4: Divide the image and its position information labels into training set samples and test set samples, and randomly divide the training set samples and test machine samples, and the division ratio is 9:1, which can float up and down; using training set samples and test machine samples, adopt The computer graphics processor GPU trains the pre-built YOLO-V3 deep neural network to obtain the YOLO-V3 neural network detection model.

In the embodiment provided by the present invention, the preprocessing further includes performing data enhancement on the character image on the tank surface obtained after the normalization process; performing data enhancement on the character image on the tank surface includes:

Geometric transformation class: including flipping, rotating, cropping, deforming, scaling;

Color transformation class: including noise, blur, color transformation, erase, fill;

Generate a sample image based on the image style simulation.

In the embodiment provided by the present invention, training the pre-built YOLO-V3 neural network includes:

S1: Set the training parameters: set the number of iteration steps epochs to 10000, set the learning rate optimizer optimizer to 'adam', set the number of batch training samples batch_size to 64, and set the size of the 9 a priori box anchor boxes;

S2: Train the YOLO-V3 neural network: input the training set sample data and the corresponding character data on the surface of the milk powder can into the YOLO-V3 neural network for model training;

S3: Model test: After each training, use the test set samples to test the trained model. If the model has a defect detection rate of more than 95% in the test set samples and the detection accuracy is not less than 95%, the model will be tested. The model obtained from the last training is used as the final YOLO-V3 neural network detection model; otherwise, the model obtained from the last training is used as the current YOLO-V3 neural network to be trained, and steps S2-S3 are repeated until the final YOLO-V3 neural network detection model is obtained. Model.

Afterwards, the final YOLO-V3 neural network detection model can be used to perform real-time detection of characters: the camera is used to collect the surface image of the milk powder can to be detected in real time, and according to the trained YOLO-V3 network, the position of the characters on the surface of the milk powder can is detected in real time.

In the embodiment provided by the present invention, the size of the prior frame is set by the k-means clustering method, including: randomly selecting 9 character area rectangular frame objects as the clustering centers of the training set, and according to each object and each clustering center The distance between each object is assigned to the nearest cluster center, each time a sample is assigned, the cluster center is recalculated, and the assignment process is repeated until no cluster center changes, and the current value of the 9 cluster centers That is, the size of the 9 a priori boxes.

In order to better understand the character position detection method based on the YOLO-V3 network, here is a brief description of the working principle of the YOLO-V3 network:

a. The YOLO-V3 network evenly divides the input image into S×S cells;

b. Each cell will predict B bounding boxes, and give the information of these bounding boxes in the form of vectors. The information of the bounding box includes position information (coordinates of the center point of the rectangular box, width and height), confidence and category information of the predicted object.

c. For the training data, after the image and label are input, the five parameters output by the cell (coordinates of the center point of the rectangular box, width and height, confidence) are substituted into the loss function (the loss function is used to calculate the value calculated by the YOLO-V3 network. The difference between the five parameters and the five marked parameters) is calculated, and the weight is adjusted by backpropagation, so that the confidence of the correct character area is increased, and the confidence of the incorrect character area is decreased; for the real-time collected data in the input After the YOLO-V3 network, five parameters (coordinates of the center point of the rectangular box, width and height, confidence) will also be calculated. After calculating these five data through the loss function, the bounding box with the lowest loss function value will be obtained. It is the final classification frame required, that is, the detection result.

In the embodiment provided by the present invention, the text correction algorithm includes:

S1: The cropped image is grayscaled, the characters are white, and the background is colored;

S2: Perform Gaussian blurring so that the text parts can be connected together, and then perform threshold processing on the image to obtain a binary image;

S3: Use the smallest rectangle to fit, and then filter the area of the text area by the size of the rectangle area, which is the smallest bounding rectangle of the text;

S4: Obtain the center point of the minimum circumscribed rectangle, and perform affine transformation on the text area to correct the oblique text.

In the embodiment provided by the present invention, the histogram horizontal projection algorithm includes:

S1: Binarize the corrected image obtained in step 4;

S2: Calculate the pixel sum in the horizontal direction for the binarized image, and draw a histogram with the horizontal axis as the image and the vertical axis as the coordinate range from 0 to the image width, and the vertical axis as the pixel sum;

S3: Find the bottom part of the histogram, which is the dividing line;

S4: According to the dividing line, each line of text image is divided.

In the embodiment provided by the present invention, the identification of the improved end-to-end variable-length text CRNN model includes:

S1: Grayscale processing of each line of text image, converting the three-channel image into a single-channel grayscale image;

S2: Length scaling and padding, fixing the size of the input image to 32*width, scaling or expanding proportionally, so that it can be similar to the training samples;

S3: Data label sparse matrix conversion, process the label matrix and convert it into the required data format;

S4: Build the model, change the structure, change the original idea of CNN+RNN+CTC transcription layer to CNN+CTC, and modify the backbone network on VGG to design a 7-layer convolutional layer and a 5-layer pooling layer. Small convolution Neural network, and adding batch regularization twice in the middle layer to avoid model gradient dispersion and accelerate convergence;

S5: Data post-processing, transform the subscript of the dictionary array corresponding to the real value into the actual value by doing a mapping transformation.

In the embodiment provided by the present invention, the network structure of the pre-built improved CRNN neural network is as follows:

S1. Its structure level includes convolutional neural network CNN, delete the original RNN structure, CNN includes 7 convolutional layers, except for the third and seventh convolutional layers, each convolutional layer will be followed by maximum pooling deal with;

S2. Adjust the parameters in the CRNN network, set the batch size to 16 or 32, the learning rate to 0.00001, and the epoch to 100;

S3. Classify the text images obtained in the previous step into the training set and the test set. According to the text features in the pictures, write codes to generate familiar pictures and text pictures of various situations, which will help to increase the generalization ability of the model. The text dataset is scrambled and randomly divided into two parts: 10,000 for the test set and 100,000 for the training set;

S4. Put the training set into the CRNN network for training, and test the current model while training;

S5. When the training and testing loss functions converge, stop training and obtain the CRNN model

S6. Use the CRNN recognition model to recognize the cropped text;'

S7. According to the system situation at the time, if it is in online mode, the recognition result will be uploaded to the database; if it is in offline mode, the recognition result will be saved locally;

S8. The actuator will switch to the next workpiece to be identified;

The method for recognizing characters on the surface of a tank provided by the present invention realizes real-time recognition of text characters by using the YOLO-V3 and CRNN deep learning method, and can quickly select the area where the characters are located, and can recognize them. Because of the use of deep learning, this method has the advantages of high robustness, high accuracy and fast speed. It provides a detection solution for intelligent character positioning for milk powder can manufacturers, and solves the problems of low manual recognition efficiency, slow speed and high personnel cost. .

As mentioned above, it is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited to this. should be included within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (10)

1. A method of canister surface character recognition, the method comprising:

acquiring a tank surface image in real time;

carrying out character positioning on the tank surface image acquired in real time by using a pre-established YOLO-V3 neural network detection model to obtain whether the tank surface image contains characters and character position information;

outputting the position of the character prediction box, and cutting out a character area;

the cut character area has the problem that the fit with the actual text area is not tight enough, and the correction is carried out by using a text correction algorithm;

segmenting text characters of each line through histogram projection;

identifying the cut text region by using an improved end-to-end indefinite length text CRNN model;

according to the system condition at the time, if the mode is an online mode, the recognition result is uploaded to a database; if the mode is an off-line mode, the recognition result is stored locally.

2. The can surface character recognition method of claim 1, further comprising: and the executing mechanism switches the next workpiece to be identified.

3. The can surface character recognition method of claim 1, wherein the can surface comprises: milk powder can surface, can surface.

4. The can surface character recognition method of claim 1, wherein the method for building the YOLO-V3 neural network detection model comprises:

s1: collecting a large number of character images on the surface of the milk powder can;

s2: preprocessing the surface image of the milk powder can; the pretreatment comprises the following steps of carrying out normalization treatment on the surface image of the milk powder can: filling the milk powder tank surface image after the equal scaling into a blank image with a preset pixel size, and then filling all pixels except the milk powder tank image on the blank image into a set color; the pixel size of the blank image is 416 x 416, and the blank image can cover the scaled milk powder can image; the set color comprises gray;

s3: carrying out character area calibration on the preprocessed images to obtain character position labels of the images; carrying out character region calibration on the preprocessed milk powder tank image in a manual frame selection mode, and storing coordinates of an upper left corner point and a lower right corner point of each calibration region and characters of corresponding regions as character tags;

s4: dividing the image and the position information label thereof into a training set sample and a test set sample, wherein the division ratio is 9: 1; and training the preset YOLO-V3 deep neural network by using a training set sample and a test machine sample and using a GPU (graphics processing Unit) to obtain a YOLO-V3 neural network detection model.

5. The tank surface character recognition method of claim 4, wherein the preprocessing further comprises data enhancement of the tank surface character image obtained after the normalization processing; the data enhancement of the can surface character image comprises the following steps:

class of geometric transformations: turning, rotating, cutting, deforming and zooming;

color transform class: including noise, blur, color transformation, erasure, padding;

and generating a sample picture according to the picture style simulation.

6. The method of pot surface character recognition according to claim 1, wherein the training of the pre-built YOLO-V3 neural network comprises:

s1: setting training parameters: setting the iteration step number epochs to 10000, setting the learning rate optimizer optizer to 'adam', setting the batch training sample number batch _ size to 64, and setting the sizes of 9 prior frames anchor box;

s2: training the YoLO-V3 neural network: inputting the sample data of the training set and the character data of the surface of the milk powder can corresponding to each sample into a YOLO-V3 neural network for model training;

s3: and (3) testing a model: after each training is finished, testing the trained model by using the test set sample, and if the detection rate of the model on flaws in the test set sample exceeds 95% and the detection accuracy is not lower than 95%, taking the model obtained by the last training as a final YoLO-V3 neural network detection model; otherwise, taking the model obtained by the last training as the YOLO-V3 neural network to be trained currently, and repeating the steps S2-S3 until the final YOLO-V3 neural network detection model is obtained.

7. The tank surface character recognition method of claim 6, wherein the size of the prior box is set by a k-means clustering method comprising: randomly selecting 9 character region rectangular frame objects as clustering centers of a training set, distributing each object to the closest clustering center according to the distance between each object and each clustering center, distributing a sample, recalculating the clustering centers, repeating the distribution process until no clustering center changes, and obtaining the value of the current 9 clustering centers, namely the size of 9 prior frames.

8. The can surface character recognition method of claim 1, wherein the text rectification algorithm comprises:

s1: carrying out gray processing on the cut image, wherein the character is white and the background is color;

s2: performing Gaussian blur to enable the text parts to be connected into a block, and performing threshold processing on the image to obtain a binary image;

s3: fitting by using a minimum rectangle, and screening to obtain the area of a text region part according to the area size of the rectangle, wherein the text region part is the minimum external rectangle of the text;

s4: and obtaining the central point of the minimum circumscribed rectangle, and carrying out affine transformation on the text region to realize the correction of the inclined text.

9. The tank surface character recognition method of claim 1, wherein the histogram horizontal projection algorithm comprises:

s1: binaryzation is carried out on the corrected image obtained in the step four;

s2: calculating pixel sums in the horizontal direction of the binarized image, and drawing a histogram with the horizontal axis as the image vertical axis coordinate range of 0-image width and the vertical axis as the pixel sums;

s3: finding out the valley part of the histogram, wherein the valley part is a division line;

s4: and segmenting each line of text image according to the segmentation line.

10. The method of canister surface character recognition according to claim 1, wherein the improved end-to-end indefinite length text CRNN model recognition comprises:

s1: performing graying processing on each line of text images, and converting three-channel images into a single-channel grayscale image;

s2: scaling and filling the length, fixing the size of the input image to 32 × width, scaling or expanding, which can be similar to the training sample;

s3: converting the data label sparse matrix, processing the label matrix, and converting the label matrix into a required data format;

s4: building a model, changing the structure, changing the idea of the original CNN + RNN + CTC transcription layer into CNN + CTC, modifying and designing 7 layers of convolution layers and 5 layers of small convolution neural networks on a VGG (variable gradient generator) by a main network, and adding two times of batch regularization in the middle layer to avoid gradient dispersion of the model and accelerate convergence;

s5: and (4) data post-processing, namely performing mapping transformation on the subscript of the dictionary array corresponding to the real value to obtain an actual value.

Images