CN113838034A - Candy packaging surface defect rapid detection method based on machine vision - Google Patents
Candy packaging surface defect rapid detection method based on machine vision Download PDFInfo
- Publication number
- CN113838034A CN113838034A CN202111137495.9A CN202111137495A CN113838034A CN 113838034 A CN113838034 A CN 113838034A CN 202111137495 A CN202111137495 A CN 202111137495A CN 113838034 A CN113838034 A CN 113838034A
- Authority
- CN
- China
- Prior art keywords
- model
- defect
- image
- detection
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 76
- 230000007547 defect Effects 0.000 title claims abstract description 64
- 235000009508 confectionery Nutrition 0.000 title claims abstract description 51
- 238000004806 packaging method and process Methods 0.000 title claims abstract description 28
- 238000012549 training Methods 0.000 claims abstract description 40
- 230000002950 deficient Effects 0.000 claims abstract description 20
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 14
- 238000004519 manufacturing process Methods 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 11
- 238000013135 deep learning Methods 0.000 claims abstract description 10
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 238000000034 method Methods 0.000 claims description 26
- 230000006870 function Effects 0.000 claims description 20
- 230000008569 process Effects 0.000 claims description 16
- 238000011156 evaluation Methods 0.000 claims description 15
- 238000012360 testing method Methods 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 7
- 210000002569 neuron Anatomy 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 230000001960 triggered effect Effects 0.000 claims description 3
- 230000007306 turnover Effects 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 230000002708 enhancing effect Effects 0.000 claims description 2
- 238000012854 evaluation process Methods 0.000 claims description 2
- 238000004880 explosion Methods 0.000 claims description 2
- 239000011159 matrix material Substances 0.000 claims description 2
- 238000005457 optimization Methods 0.000 claims description 2
- 230000010355 oscillation Effects 0.000 claims description 2
- 238000003860 storage Methods 0.000 claims description 2
- 230000008030 elimination Effects 0.000 abstract description 6
- 238000003379 elimination reaction Methods 0.000 abstract description 6
- 238000013136 deep learning model Methods 0.000 abstract description 2
- 238000005516 engineering process Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000008187 granular material Substances 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000011089 mechanical engineering Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000012634 optical imaging Methods 0.000 description 1
- 238000012858 packaging process Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10141—Special mode during image acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Biomedical Technology (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a candy packaging defect rapid detection method based on machine vision, which comprises the following steps: (1) collecting an image; (2) processing images, namely performing a defect rapid detection algorithm based on deep learning in an image processing system, wherein the algorithm is an improved VGG16 deep learning model, the algorithm is obtained by preprocessing, training, evaluating and optimizing a real candy data set on a production line, and a defective product signal is sent out when a defect is detected; (3) and (4) rejecting defective products, wherein the rejecting system receives the defective product signals and executes rejecting operation. The candy packaging defect online detection and elimination are realized, the image acquisition rate, the unqualified product identification rate and the elimination rate can meet the production rate of 10 candies per second, the problems of low manual detection precision and high cost are solved, and the defect detection efficiency and reliability are greatly improved; the defect online detection in other fields is easily expanded, and the defect online detection is the same as the defect online detection of type product packages and the defect online detection of mobile phone shells.
Description
Technical Field
The invention belongs to the field of artificial intelligence technology and rapid detection of product outer package defects based on machine vision, and particularly relates to a rapid detection method of candy package surface defects.
Background
In modern automatic production lines, defects may exist on the surface of the candy package, the candy package is unqualified, the unqualified product is reduced from flowing into the market, good sensory experience can be brought to consumers, and the competitiveness of the candy package is improved. The discharging speed of the candy packaging production line is generally very high and reaches 0.1 s/granule, and in the packaging process, the quality problems of packaging damage, empty bags, folds and the like can be caused due to mechanical vibration or other environmental factors. The traditional detection method is to detect whether the package has defects by a manual visual method, but the manual detection has the problems of low efficiency, low accuracy and high cost. On-line inspection according to the principles of machine vision can solve the above problems. Machine vision is an integrated technology including image processing, mechanical engineering, control, electrical light source illumination, optical imaging, sensors, analog and digital video technology, computer hardware and software technology (image enhancement and analysis algorithms, image cards, I/O cards, etc.). A typical machine vision application system comprises an image capture module, a light source system, an image digitization module, a digital image processing module, an intelligent judgment decision module and a mechanical control execution module.
The online identification of candy packaging defects needs to be matched with the real-time production speed of a production line and also needs to ensure certain accuracy, so that online detection has certain requirements on visual devices, software algorithms and rejection mechanisms. Firstly, in order to meet the requirement of real-time detection, triggering conditions for product photographing need to be set, and the resolution and the image transmission rate of the industrial camera also need to be considered. The image recognition adopts a deep convolutional network, deep learning is to learn the intrinsic rules and the expression levels of sample data, and information obtained in the learning process is greatly helpful to the interpretation of data such as characters, images and sounds. The final aim of the method is to enable the machine to have the analysis and learning capability like a human, and to recognize data such as characters, images and sounds. Deep learning is a complex machine learning algorithm, and achieves the effect in speech and image recognition far exceeding the prior related art.
The invention provides a candy packaging defect rapid detection method based on machine vision, which is characterized in that the core of the method is a defect rapid detection algorithm based on deep learning, the detection time of a single candy is less than 0.1s, and the online detection and defective product elimination of candy packaging defects are realized by matching with a hardware system.
Disclosure of Invention
1. Objects of the invention
The invention aims to provide a method for rapidly detecting the packaging defects of candies, which realizes the online detection and elimination of the packaging defects of the candies.
2. The technical scheme adopted by the invention
The invention provides a candy packaging defect rapid detection method based on machine vision, which comprises the following steps:
step 1: image acquisition
Adopting a color area array camera to match with a planar shadowless light source to perform lighting shooting to obtain a surface image of the candy moving at a high speed, and storing the image and caching the image by the camera;
step 2: image processing
Reading the image based on a defect rapid detection algorithm of deep learning, and sending a defective product signal to an eliminating system when packaging defect candies appear;
step 2.1: image preprocessing, specifically, converting a data format, adjusting the image size, dividing a data set and enhancing the data;
step 2.2: constructing a defect rapid detection algorithm and training a model, inputting the preprocessed data into a defect recognition network, training a defect detection model, and obtaining a candy packaging defect classification result;
step 2.3: evaluating the performance of the model, namely evaluating the performance of the model obtained after training;
step 2.4: model optimization, namely further optimizing the model by combining the evaluation result in the step 2.3;
and step 3: defective product removal
And receiving a defective product signal, quickly responding by the rejecting system, and overturning to reject the defective product.
Preferably, in the step 1, a color area array industrial camera is adopted, and the vertical position is adjusted to detect different types of product packages; the used plane shadowless light source can effectively eliminate the reflection generated by the aluminum-plastic package, and the accuracy of defect detection and classification is improved; the candy passing through is sensitively captured by an infrared opposite-emitting sensor, and a photo is triggered.
Preferably, in the step 2.1, the data format conversion and size adjustment process is to convert the format of the image data set into a data input format facing a Keras framework, the original format of the image acquired in the industrial camera is BYTE, and the image data set can be input into a neural network for processing only by converting into a Mat matrix, the pixel size of the image directly obtained by shooting is 1000 × 1000 × 3, and the size of the image is adjusted to 150 × 150 × 3 by image interpolation in order to improve the image processing speed and the recognition speed; the data set dividing process is to divide the data set into a training set and a test set according to the proportion of 8: 2; the data enhancement process is to rotate, translate, turn over, scale transform, scale and cut randomly the images in the training set, enlarge the scale of the training set and enhance the generalization ability of the model.
Preferably, in the step 2.2, classification detection is performed on the input image through a deep learning neural network, a VGG16 convolution base part is used as a pre-training network for feature extraction, a full connection layer is added as a classifier to form a defect identification network, and in the model training process, a backbone convolution network is firstly frozen, a training set is used for training a classifier and adjusting hyper-parameters; then loading the trained model, unfreezing the convolution block close to the classifier, and utilizing the training set to train the model unfrozen convolution layer and the classifier again, so that the accuracy of the model is further improved; the final model is saved in the folder in which the model is in the form of h5 file; the hyper-parameters comprise learning rate, batch size and iteration times; the training process of the VGG16 model is a weight updating process, and the weight updating process comprises the following activation function, loss function and optimizer; relu activation function is selected in the convolutional layer, and the function formula is as follows:
f(x)＝max(0,x)
wherein x is the output of each layer of the neural network; the full connection layer adopts a Softmax function, and displays the multi-classification result in a probability form, wherein the function formula is as follows:
the last layer of the VGG16 neural network outputs C neurons, where WyIs the weight of the ith neuron of the last layer, WcA weight for each neuron; selecting a categorical cross entropy function as a loss function, and comprising the following steps of:
n is the number of samples, m is the number of classifications, yimTo determine the probability that an input belongs to the mth class,
for the class to which the input actually belongs, the variable is 0 or 1; the cross entropy function is a multi-output loss function, and the loss value is also multiple;
…
using RMSprop as an optimizer, limiting oscillations in the vertical direction, allowing the model to converge quickly, the algorithm equation is as follows:
vdw＝β·vdw+(1-β)·dw2
vdb＝β·vdw+(1-β)·db2
dW and db are the differential at mini-batch, vdWAnd vdbIs an exponentially weighted average, beta represents a momentum value, and is set to 0.9; parameter e for preventing vdwAnd the weight explosion and gradient rise caused by approaching 0, and alpha is a hyperparameter.
Preferably, in step 2.3, in the defect classification precision evaluation process, a test set which does not participate in training is used as a data set for evaluation, and the final model is excellent in performance on the test set; specifically, Precision and Recall are used as evaluation indexes; the precision rate is measured by predicting the correct proportion in all the results which are predicted as positive examples; the recall rate measures how many actual positive examples are predicted as positive examples by the model; TP is the number of true positive examples, TN is the number of true negative examples, FP is the number of false positive examples, and FN is the number of false negative examples;
preferably, in step 2.3, the detection speed is evaluated, after the model stably runs for a period of time, the detection speed of a single picture is taken as an evaluation index, and the candy output per minute of a single machine in the production line is 500, so that the detection time of the single picture is required to be less than 0.1 s; the model file is loaded when the model is operated for the first time, so that the detection time of a single picture of the model after the model is operated for the second time and stably operated is used as an evaluation index for reasonably evaluating the quality of the model, and the model is acceptable if the detection time is less than 0.1 s.
Preferably, in step 2.4, the evaluation index in step 2.3 is analyzed, and the detection precision and the detection speed are a pair of mutually restricted indexes, so that the detection precision needs to be improved on the premise of ensuring the detection speed, so as to meet the real-time property and higher precision of detection.
A candy packaging defect rapid detection device based on machine vision comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the steps of the method when executing the computer program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method steps.
3. Advantageous effects adopted by the present invention
(1) The invention designs an on-line shooting visual hardware system triggered by a sensor, adopts a color area array industrial camera to clearly acquire surface images of high-speed moving candies, adopts a plane shadowless light source to effectively eliminate reflection generated by aluminum-plastic packaging, and can match the actual production speed of 2m/s of a candy production line in detection and elimination
(2) In order to better extract the defect characteristics of the candy pictures, a convolutional base part based on VGG16 is used as a characteristic extraction network, a classifier consisting of two fully-connected layers is added, the classifier is trained by a candy picture training set, partial rolling blocks are unfrozen, and the model is finely adjusted by using the same candy training set, so that the model more suitable for candy defect classification is obtained.
(3) The data enhancement technology utilized by the invention can enrich the distribution of training data, improve the generalization capability and robustness of the model, prevent overfitting and overcome the problem of poor model performance caused by incomplete candy defective product samples.
(4) And software and hardware data interaction, automatic detection and elimination and manpower liberation. The user interaction interface is arranged, the defective products can be traced, the generation of the defective products is reduced from the source, missing detection and false detection are avoided, the labor degree of operators is reduced, and the automation degree of the production line is improved.
Drawings
FIG. 1 is a schematic view of a candy packaging defect detection method.
FIG. 2 is a schematic structural view of a candy packaging defect detection device.
FIG. 3 is a flow chart for constructing a defect detection algorithm.
Figure 4 is a diagram of a defect detection network.
Detailed Description
The technical solutions in the examples of the present invention are clearly and completely described below with reference to the drawings in the examples of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without inventive step, are within the scope of the present invention.
The present invention will be described in further detail with reference to the accompanying drawings.
Example 1
Referring to fig. 1 and 2, a method for rapidly detecting a defect of a candy package based on machine vision includes the following steps:
step 1: and (5) image acquisition.
Specifically, after the candy is packaged in the production line and enters the image acquisition unit through the conveying belt, the candy reaches the position right below the plane shadowless light source and triggers the infrared sensor, so that a signal is sent out to trigger the industrial camera to take a picture and the picture is stored in the memory of the camera.
Step 2: and (5) processing the image.
And the image processing software automatically makes a judgment after receiving the picture in the memory of the processor, and quickly transmits a defective product signal to the rejecting mechanism if the image processing software judges that the image is not qualified.
And step 3: and removing defective products.
After receiving the defective product signal, transmitting a signal to the PLC, controlling the turnover device by the PLC to remove the defective product, and enabling the qualified product to enter the next procedure.
Referring to fig. 3, the core of the present invention is a deep learning-based package defect rapid detection algorithm, which includes the following steps:
step 2.1: and (5) image preprocessing.
Specifically, a folder path where an acquired image is located is taken as a parameter, and a batch data input format facing a Keras frame is obtained after data lifting and normalization, wherein the sizes of input pictures are all adjusted to be 150 × 150 × 3, and the batch size is 10. The data set was divided into 400 training sets (80 for 5 types) and 100 test sets (20 for 5 types). And randomly performing data enhancement operations such as rotation, translation, overturning, scale transformation, scaling, cutting and the like on the images of the training set, wherein the training set generated after enhancement comprises 1800 images.
Step 2.2: and training a defect recognition model.
The defect identification network adopts a Windows operating system, the programming language is Python, and the deep learning framework Keras is used for operation. Firstly, adding a full connection layer (classifier) on the basis of a VGG16 convolutional layer serving as a pre-training model to form a defect detection network, and developing deep learning model training as shown in FIG. 4, specifically: extracting features by using a VGG16 backbone convolution network, carrying out first training and super-parameter adjustment on a classifier on a candy defect training set, setting the learning rate to be 0.00002 when the super-parameters are adjusted, setting the batch size to be 10, and setting the iteration number to be 20. Then, the trunk convolutional neural network is unfrozen to be close to the classifier convolutional block 5, and the classifier and the unfrozen partial convolutional layer are subjected to fine tuning by using the same training set. The defect classification model after training is saved in the folder in which the model is located in the h5 file form.
Step 2.3: the trained models were evaluated on a candy defect test set. For the detection precision, calculating precision and recall rate as evaluation indexes, firstly calculating TP (true positive), FP (false positive), FN (false negative) and TN (true negative) values, and calculating the precision and recall rate by the following formulas:
and after the model stably runs for a period of time, outputting the detection speed of a single picture as a performance evaluation index.
In this process, a total of 100 pictures were taken as a test set, including 40 normal (front, back) and 60 defective (with sugar, with bare sugar, with a virtual seal). The average accuracy rate obtained by calculation is as follows: 84.4%, the average recall ratio is: 84.4 percent, wherein the virtual seal recall rate is lower and is 63.5 percent. And the detection speed of a single picture output after the model stably runs is 0.25 s.
Step 2.4: and (4) combining the evaluation result of the step 2.3, and further optimizing the model. Firstly, the requirement of production line real-time performance is met, a dropout layer with the parameter of 0.5 is added into a full connection layer, the model parameter is reduced, the model scale is reduced, the single prediction time is shortened from 0.25s to 0.07s, and the requirement of production line real-time performance is met. Secondly, aiming at the condition that the detection precision average value is low due to low virtual seal recall rate, 50 virtual seal defect picture data sets are shot again, the data sets are enhanced to 250, a training set retraining model is added, finally the virtual seal recall rate is increased to 83.2%, the average recall rate is increased to 88.3%, and the detection precision requirement is met.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (9)
1. A candy packaging defect rapid detection method based on machine vision is characterized in that:
step 1: image acquisition
Adopting a color area array camera to match with a planar shadowless light source to perform lighting shooting to obtain a surface image of the candy moving at a high speed, and storing the image and caching the image by the camera;
step 2: image processing
Reading the image based on a defect rapid detection algorithm of deep learning, and sending a defective product signal to an eliminating system when packaging defect candies appear;
step 2.1: image preprocessing, specifically, converting a data format, adjusting the image size, dividing a data set and enhancing the data;
step 2.2: constructing a defect rapid detection algorithm and training a model, inputting the preprocessed data into a defect recognition network, training a defect detection model, and obtaining a candy packaging defect classification result;
step 2.3: evaluating the performance of the model, namely evaluating the performance of the model obtained after training;
step 2.4: model optimization, namely further optimizing the model by combining the evaluation result in the step 2.3;
and step 3: defective product removal
And receiving a defective product signal, quickly responding by the rejecting system, and overturning to reject the defective product.
2. The machine vision-based candy packaging defect rapid detection method according to claim 1, characterized in that: in the step 1, a color area array industrial camera is adopted to detect different types of product packages by adjusting the vertical position; the used plane shadowless light source can effectively eliminate the reflection generated by the aluminum-plastic package, and the accuracy of defect detection and classification is improved; the candy passing through is sensitively captured by an infrared opposite-emitting sensor, and a photo is triggered.
3. The machine vision-based candy packaging defect rapid detection method according to claim 1, characterized in that: in the step 2.1, the conversion data format and the size adjustment process are to convert the format of the image data set into a data input format facing a Keras framework, the original format of the image acquired in the industrial camera is BYTE, and the image can be input into a neural network for processing only by converting into a Mat matrix, the size of the image pixel obtained by direct shooting is 1000 × 1000 × 3, and the size of the image is adjusted to 150 × 150 × 3 by image interpolation in order to improve the image processing speed and the recognition speed; the data set dividing process is to divide the data set into a training set and a test set according to the proportion of 8: 2; the data enhancement process is to rotate, translate, turn over, scale transform, scale and cut randomly the images in the training set, enlarge the scale of the training set and enhance the generalization ability of the model.
4. The machine vision-based candy packaging defect rapid detection method according to claim 3, characterized in that: in the step 2.2, classification detection is carried out on the input image through a deep learning neural network, a VGG16 convolution base part is used as a pre-training network for feature extraction, a full connection layer is added to be used as a classifier to form a defect identification network, and in the model training process, a main convolution network is firstly frozen, a classifier is trained by using a training set, and a hyper-parameter is adjusted; then loading the trained model, unfreezing the convolution block close to the classifier, and utilizing the training set to train the model unfrozen convolution layer and the classifier again, so that the accuracy of the model is further improved; the final model is saved in the folder in which the model is in the form of h5 file; the hyper-parameters comprise learning rate, batch size and iteration times; the training process of the VGG16 model is a weight updating process, and the weight updating process comprises the following activation function, loss function and optimizer; relu activation function is selected in the convolutional layer, and the function formula is as follows:
f(x＝max(0,x)
wherein x is the output of each layer of the neural network; the full connection layer adopts a Softmax function, and displays the multi-classification result in a probability form, wherein the function formula is as follows:
the last layer of the VGG16 neural network outputs C neurons, where WyIs the weight of the ith neuron of the last layer, WcA weight for each neuron; selecting a categorical cross entropy function as a loss function, and comprising the following steps of:
n is the number of samples, m is the number of classifications, yimTo determine the probability that an input belongs to the mth class,
for the class to which the input actually belongs, the variable is 0 or 1; the cross entropy function is a multi-output loss function, and the loss value is also multiple;
…
using RMSprop as an optimizer, limiting oscillations in the vertical direction, allowing the model to converge quickly, the algorithm equation is as follows:
vdw＝β·vdw+(1-β)·dw2
vdb＝β·vdw+(1-β)·db2
dW and db are the derivatives at mini-batch, vdW and vdb are exponentially weighted averages, and beta represents the momentum value and is set to 0.9; parameter e for preventing vdwWeight explosion caused by approaching 0 andthe gradient rises and alpha is a hyperparameter.
5. The machine vision-based candy packaging defect rapid detection method according to claim 4, characterized in that: in the step 2.3, in the defect classification precision evaluation process, a test set which does not participate in training is used as a data set for evaluation, and finally, the model is excellent in performance on the test set; specifically, Precision and Recall are used as evaluation indexes; the precision rate is measured by predicting the correct proportion in all the results which are predicted as positive examples; the recall rate measures how many actual positive examples are predicted as positive examples by the model; TP is the number of true positive examples, TN is the number of true negative examples, FP is the number of false positive examples, and FN is the number of false negative examples;
6. the machine vision-based candy packaging defect rapid detection method according to claim 5, characterized in that: in the step 2.3, the detection speed is evaluated, after the model stably runs for a period of time, the detection speed of a single picture is taken as an evaluation index, and the candy output per minute of a single machine in the production line is 500, so that the detection time of the single picture is required to be less than 0.1 s; the model file is loaded when the model is operated for the first time, so that the detection time of a single picture of the model after the model is operated for the second time and stably operated is used as an evaluation index for reasonably evaluating the quality of the model, and the model is acceptable if the detection time is less than 0.1 s.
7. The machine vision-based candy packaging defect rapid detection method according to claim 6, characterized in that: in step 2.4, the evaluation index in step 2.3 is analyzed, the detection precision and the detection speed are a pair of mutually restricted indexes, and the detection precision needs to be improved on the premise of ensuring the detection speed so as to meet the detection real-time performance and higher precision.
8. A candy packaging defect rapid detection device based on machine vision comprises a memory and a processor, wherein the memory stores a computer program and is characterized in that; the processor, when executing the computer program, realizes the method steps of any of claims 1-7.
9. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program implementing the method steps of any one of claims 1 to 7 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111137495.9A CN113838034B (en) | 2021-09-27 | 2021-09-27 | Quick detection method for surface defects of candy package based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111137495.9A CN113838034B (en) | 2021-09-27 | 2021-09-27 | Quick detection method for surface defects of candy package based on machine vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113838034A true CN113838034A (en) | 2021-12-24 |
| CN113838034B CN113838034B (en) | 2023-11-21 |
Family
ID=78971026
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111137495.9A Active CN113838034B (en) | 2021-09-27 | 2021-09-27 | Quick detection method for surface defects of candy package based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113838034B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114088730A (en) * | 2022-01-24 | 2022-02-25 | 心鉴智控(深圳)科技有限公司 | Method and system for detecting aluminum-plastic bubble cap defects by using image processing |
| CN114354628A (en) * | 2022-01-05 | 2022-04-15 | 威海若维信息科技有限公司 | Root agricultural product defect detection method based on machine vision |
| CN114463327A (en) * | 2022-04-08 | 2022-05-10 | 深圳市睿阳精视科技有限公司 | Multi-shooting imaging detection equipment and method for watermark defect of electronic product lining package |
| CN114577816A (en) * | 2022-01-18 | 2022-06-03 | 广州超音速自动化科技股份有限公司 | Hydrogen fuel bipolar plate detection method |
| CN115452842A (en) * | 2022-10-20 | 2022-12-09 | 颖态智能技术(上海)有限公司 | Fold detection method for valve bag packaging machine |
| CN116309597A (en) * | 2023-05-23 | 2023-06-23 | 成都工业学院 | Visual on-line detection method and device for medicine box mixed-loading defects |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20190088089A (en) * | 2017-12-26 | 2019-07-26 | 세종대학교산학협력단 | Apparatus and method for detecting defects on welding surface |
| CN110363253A (en) * | 2019-07-25 | 2019-10-22 | 安徽工业大学 | A kind of Surfaces of Hot Rolled Strip defect classification method based on convolutional neural networks |
| CN110610475A (en) * | 2019-07-07 | 2019-12-24 | 河北工业大学 | Visual defect detection method of deep convolutional neural network |
| WO2020048119A1 (en) * | 2018-09-04 | 2020-03-12 | Boe Technology Group Co., Ltd. | Method and apparatus for training a convolutional neural network to detect defects |
| CN111862025A (en) * | 2020-07-14 | 2020-10-30 | 中国船舶重工集团公司第七一六研究所 | PCB defect automatic detection method based on deep learning |
| CN112381787A (en) * | 2020-11-12 | 2021-02-19 | 福州大学 | Steel plate surface defect classification method based on transfer learning |
| CN213933621U (en) * | 2020-10-26 | 2021-08-10 | 力度工业智能科技(苏州)有限公司 | Candy packaging defect detection platform based on machine vision |
-
2021
- 2021-09-27 CN CN202111137495.9A patent/CN113838034B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20190088089A (en) * | 2017-12-26 | 2019-07-26 | 세종대학교산학협력단 | Apparatus and method for detecting defects on welding surface |
| WO2020048119A1 (en) * | 2018-09-04 | 2020-03-12 | Boe Technology Group Co., Ltd. | Method and apparatus for training a convolutional neural network to detect defects |
| CN110610475A (en) * | 2019-07-07 | 2019-12-24 | 河北工业大学 | Visual defect detection method of deep convolutional neural network |
| CN110363253A (en) * | 2019-07-25 | 2019-10-22 | 安徽工业大学 | A kind of Surfaces of Hot Rolled Strip defect classification method based on convolutional neural networks |
| CN111862025A (en) * | 2020-07-14 | 2020-10-30 | 中国船舶重工集团公司第七一六研究所 | PCB defect automatic detection method based on deep learning |
| CN213933621U (en) * | 2020-10-26 | 2021-08-10 | 力度工业智能科技(苏州)有限公司 | Candy packaging defect detection platform based on machine vision |
| CN112381787A (en) * | 2020-11-12 | 2021-02-19 | 福州大学 | Steel plate surface defect classification method based on transfer learning |
Non-Patent Citations (2)
| Title |
|---|
| 张志晟 等: "基于深度学习的易拉罐缺陷检测技术", 包装工程, vol. 41, no. 19, pages 259 - 266 * |
| 项宇杰 等: "基于深度卷积神经网络的木材表面缺陷检测系统设计", 系统仿真技术, vol. 15, no. 04, pages 253 - 257 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114354628A (en) * | 2022-01-05 | 2022-04-15 | 威海若维信息科技有限公司 | Root agricultural product defect detection method based on machine vision |
| CN114354628B (en) * | 2022-01-05 | 2023-10-27 | 威海若维信息科技有限公司 | Rhizome agricultural product defect detection method based on machine vision |
| CN114577816A (en) * | 2022-01-18 | 2022-06-03 | 广州超音速自动化科技股份有限公司 | Hydrogen fuel bipolar plate detection method |
| CN114088730A (en) * | 2022-01-24 | 2022-02-25 | 心鉴智控(深圳)科技有限公司 | Method and system for detecting aluminum-plastic bubble cap defects by using image processing |
| CN114463327A (en) * | 2022-04-08 | 2022-05-10 | 深圳市睿阳精视科技有限公司 | Multi-shooting imaging detection equipment and method for watermark defect of electronic product lining package |
| CN115452842A (en) * | 2022-10-20 | 2022-12-09 | 颖态智能技术(上海)有限公司 | Fold detection method for valve bag packaging machine |
| CN116309597A (en) * | 2023-05-23 | 2023-06-23 | 成都工业学院 | Visual on-line detection method and device for medicine box mixed-loading defects |
| CN116309597B (en) * | 2023-05-23 | 2023-08-01 | 成都工业学院 | Visual on-line detection method and device for medicine box mixed-loading defects |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113838034B (en) | 2023-11-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113838034B (en) | Quick detection method for surface defects of candy package based on machine vision | |
| CN106875373B (en) | Mobile phone screen MURA defect detection method based on convolutional neural network pruning algorithm | |
| CN113077450B (en) | Cherry grading detection method and system based on deep convolutional neural network | |
| CN113361326B (en) | Wisdom power plant management and control system based on computer vision target detection | |
| CN113469050A (en) | Flame detection method based on image subdivision classification | |
| CN108428324A (en) | The detection device of smog in a kind of fire scenario based on convolutional network | |
| CN110909672A (en) | Smoking action recognition method based on double-current convolutional neural network and SVM | |
| CN117095155A (en) | Multi-scale nixie tube detection method based on improved YOLO self-adaptive attention-feature enhancement network | |
| CN114898472A (en) | Signature identification method and system based on twin vision Transformer network | |
| CN116704267A (en) | Deep learning 3D printing defect detection method based on improved YOLOX algorithm | |
| CN112488213A (en) | Fire picture classification method based on multi-scale feature learning network | |
| CN113538342B (en) | Convolutional neural network-based aluminum aerosol can coating quality detection method | |
| Burkapalli et al. | TRANSFER LEARNING: INCEPTION-V3 BASED CUSTOM CLASSIFICATION APPROACH FOR FOOD IMAGES. | |
| CN115953719A (en) | Multi-target recognition computer image processing system | |
| Cao et al. | A spatial pyramid pooling convolutional neural network for smoky vehicle detection | |
| CN114972246A (en) | Die-cutting product surface defect detection method based on deep learning | |
| CN114821154A (en) | Grain depot ventilation window state detection algorithm based on deep learning | |
| CN112837281A (en) | Pin defect identification method, device and equipment based on cascade convolutional neural network | |
| CN117893732A (en) | Gangue detection and sorting method based on PLC and deep learning | |
| CN115546894A (en) | Behavior detection method based on lightweight OpenPose space-time diagram network | |
| CN116030033A (en) | Tobacco package appearance detection method and equipment, model training method and device | |
| Chen et al. | Development of an Intelligent Defect Detection System for Gummy Candy under Edge Computing | |
| CN111191575B (en) | Naked flame detection method and system based on flame jumping modeling | |
| Kuo et al. | Design and Implementation of AI aided Fruit Grading Using Image Recognition | |
| Deng et al. | Image classification and detection of cigarette combustion cone based on inception resnet V2 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |