CN113762023B - Object identification method and device based on article association relation - Google Patents
Object identification method and device based on article association relation Download PDFInfo
- Publication number
- CN113762023B CN113762023B CN202110188078.0A CN202110188078A CN113762023B CN 113762023 B CN113762023 B CN 113762023B CN 202110188078 A CN202110188078 A CN 202110188078A CN 113762023 B CN113762023 B CN 113762023B
- Authority
- CN
- China
- Prior art keywords
- package
- neural network
- article
- sequence
- subgraphs
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000001514 detection method Methods 0.000 claims abstract description 78
- 238000013528 artificial neural network Methods 0.000 claims abstract description 61
- 238000012549 training Methods 0.000 claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 20
- 238000003860 storage Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 8
- 238000012546 transfer Methods 0.000 claims description 8
- 239000000523 sample Substances 0.000 claims 6
- 230000000694 effects Effects 0.000 abstract description 10
- 238000007689 inspection Methods 0.000 description 24
- 239000011159 matrix material Substances 0.000 description 24
- 238000010586 diagram Methods 0.000 description 17
- 230000006870 function Effects 0.000 description 10
- 238000013527 convolutional neural network Methods 0.000 description 8
- 238000001914 filtration Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 238000013459 approach Methods 0.000 description 4
- 238000009826 distribution Methods 0.000 description 4
- 238000002372 labelling Methods 0.000 description 4
- 239000007788 liquid Substances 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000007476 Maximum Likelihood Methods 0.000 description 2
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 235000013399 edible fruits Nutrition 0.000 description 2
- 239000002360 explosive Substances 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 239000006260 foam Substances 0.000 description 2
- 238000011478 gradient descent method Methods 0.000 description 2
- 239000003550 marker Substances 0.000 description 2
- 239000008267 milk Substances 0.000 description 2
- 210000004080 milk Anatomy 0.000 description 2
- 235000013336 milk Nutrition 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 239000000843 powder Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 238000004904 shortening Methods 0.000 description 2
- 238000005406 washing Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 239000000383 hazardous chemical Substances 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/587—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/087—Inventory or stock management, e.g. order filling, procurement or balancing against orders
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Economics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Databases & Information Systems (AREA)
- Human Resources & Organizations (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Development Economics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Entrepreneurship & Innovation (AREA)
- Library & Information Science (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an object identification method and device based on an article association relation, and relates to the technical field of warehouse logistics. One embodiment of the method comprises the following steps: carrying out package detection on the acquired pictures to obtain package subregions of each package; processing the object subgraphs of all objects in the parcel subarea into a sequence, and inputting the sequence into a neural network; obtaining a subsequence of the object in the parcel zone by using the neural network; and performing object recognition, wherein the neural network is trained based on training samples: collecting pictures with a plurality of packages and including a plurality of items in the packages to generate the training sample; performing package detection on the picture; and processing the object subgraphs of the plurality of objects contained in the obtained package subgraphs into a sequence, and marking the corresponding objects and the positions thereof to establish a corresponding object position database. This embodiment has the technical effect of improving the object recognition rate while reducing the cost.
Description
Technical Field
The invention relates to the field of warehouse logistics, in particular to an object identification method and device based on article association relations.
Background
In public transportation sites such as aviation and subway, it is necessary to perform security inspection on freight or shipping articles. The dangerous goods identification of the security inspection machine is an important computer vision technology for security protection. The goods are subjected to non-unpacking safety inspection through x-rays of a safety inspection machine. Sometimes, a large number of articles pass through the security inspection machine in a certain period of time, and video data are required to be used for automatically assisting security inspection personnel in identifying dangerous articles so as to determine the types of the dangerous articles and packages with the dangerous articles, and at the moment, vision technology is used for assisting in judging the dangerous articles.
However, in the process of implementing the present invention, the inventors found that at least the following problems exist in the prior art: although the security inspection machine image recognition technology is mature, as the prior art utilizes target detection, image segmentation and the like, whether dangerous goods exist or not is directly found in the image, and in actual operation, some dangerous goods have smaller volume, unobvious characteristics and difficult recognition; some dangerous goods are difficult to identify because of larger shape difference; meanwhile, color imaging can also cause poor universality of a detection model obtained by training on a public training set due to the difference of model numbers of security check machines.
The applicant conducted intensive studies to improve the recognition rate of objects, and as a result found that: because the probability of the occurrence of various articles in the same package is different, the association relationship exists between the various articles. For example, a user who purchases a fruit knife is likely to purchase other kitchen utensils, such as a plate, and therefore the information of having a plate in the package is useful for determining whether the current item belongs to the fruit knife. That is, when it is determined that an item is within a package, it is helpful to confirm whether an object of a certain type is contained.
Disclosure of Invention
In view of the above, embodiments of the present invention provide a method and an apparatus for identifying objects based on association relationships between objects, which can fully utilize association relationships between objects, and improve the identification rate of objects while reducing the cost.
To achieve the above object, according to an aspect of the embodiment of the present invention, there is provided a method for identifying an object based on an association relationship between objects, including: carrying out package detection on an acquisition picture acquired by a shooting device, and acquiring package subregions corresponding to each package in the acquisition picture; processing object subgraphs of all objects in the parcel subarea into sequences according to a predetermined sequence, and inputting the sequences into a neural network; obtaining a subsequence of the object in the parcel subarea based on the article association relationship by using the neural network; and performing object recognition according to the subsequence, wherein the neural network is trained based on training samples: collecting pictures with a plurality of packages and including a plurality of items in the packages to generate the training sample; performing package detection on the picture; and processing the object subgraphs of the plurality of objects contained in the obtained package subgraphs into a sequence, and marking the corresponding objects and the positions thereof to establish a corresponding object position database based on the object association relationship.
One embodiment of the above invention has the following advantages or benefits: the invention directly uses the technical means of assisting object recognition by using the article association relation (namely, using the association matrix of SKU articles in initialization) of SKU (Stock Keeping Unit) sold by e-commerce, thereby reducing the cost of training and learning, and overcoming the technical problems of difficult recognition when the object volume is small or the shape difference is large and poor universality of a detection model in color imaging, thereby achieving the technical effect of improving the object recognition rate while reducing the cost.
In the object recognition method based on the article association relationship of the present invention, preferably, when training the neural network, the network for target detection in the neural network is trained according to the marked article and the position thereof.
In the object recognition method based on the article association relationship, preferably, when training the neural network, the corresponding article position database is updated by using conditional probability so as to tune a sub-neural network in the neural network.
In the object recognition method based on the item association relation of the present invention, preferably, when the item sub-images of all the items in the package sub-area are processed into a sequence, the sequence is sorted according to IoU among the specific items.
In the object recognition method based on the item association relation of the present invention, preferably, when the item sub-images of all the items in the package sub-area are processed into a sequence, the objects are sorted according to IoU and distances between specific items.
In the object recognition method based on the article association relation of the present invention, it is preferable that after sorting from large to small according to IoU, sorting is performed from far to near according to the distance.
In the object recognition method based on the article association relation, preferably, the updated article position database is used for an e-commerce recommending stage and a package mailing stage of logistics.
In order to achieve the above object, according to an aspect of an embodiment of the present invention, there is provided an apparatus for object recognition based on an article association relationship, including: the package detection module is used for carrying out package detection on the acquired pictures acquired by the shooting device and acquiring package subregions corresponding to each package in the acquired pictures; the sub-graph sorting module processes the object sub-graphs of all objects in the parcel sub-area into a sequence according to a predetermined sequence, and inputs the sequence into a neural network; the subsequence acquisition module is used for acquiring a subsequence of the object in the parcel subarea based on the article association relationship by using the neural network; and an object recognition module for performing object recognition according to the subsequence, wherein the neural network is trained based on training samples: collecting pictures with a plurality of packages and including a plurality of items in the packages to generate the training sample; performing package detection on the picture; and processing the object subgraphs of the plurality of objects contained in the obtained package subgraphs into a sequence, and marking the corresponding objects and the positions thereof to establish a corresponding object position database based on the object association relationship.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main flow of a method of object recognition based on item associations according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of the main modules of an apparatus for object recognition based on item associations according to an embodiment of the present invention;
FIG. 3 (a) is a schematic diagram of a security check machine inspecting an original view for explaining a package inspection action according to an embodiment of the present invention;
FIG. 3 (b) is a schematic diagram of a security check machine package inspection result diagram for illustrating a package inspection action according to an embodiment of the present invention;
FIG. 4 (a) is a schematic diagram illustrating an original view of a single hair wrap for SKU detection of a single wrap;
FIG. 4 (b) is a schematic diagram of a SKU detection chart illustrating a single package for SKU detection of a single package;
fig. 5 is a schematic diagram for explaining the structure of the neural network of the present application;
FIG. 6 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
Fig. 7 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic diagram of main flow of a method for object recognition based on an item association relationship according to an embodiment of the present invention, and as shown in fig. 1, the method includes: step S101: carrying out package detection on an acquisition picture acquired by a shooting device, and acquiring package subregions corresponding to each package in the acquisition picture; step S102: processing object subgraphs of all objects in the parcel subarea into sequences according to a predetermined sequence, and inputting the sequences into a neural network; step S103: obtaining a subsequence of the object in the parcel subarea based on the article association relationship by using the neural network; step S104: performing object recognition according to the subsequence, wherein the neural network is trained based on training samples; collecting pictures with a plurality of packages and including a plurality of items in the packages to generate the training sample; performing package detection on the picture; and processing the object subgraphs of the plurality of objects contained in the obtained package subgraphs into a sequence, and marking the corresponding objects and the positions thereof to establish a corresponding object position database based on the object association relationship. Hereinafter, each of the above steps will be specifically described.
Step S101: and carrying out package detection on the acquired pictures acquired by the shooting device, and acquiring package subregions corresponding to all packages in the acquired pictures. Step S101 is mainly used for package detection, and the more mature method commonly used in the prior art can be used for the step S101 in the present invention. There is no particular limitation as long as it can be used for package detection. However, considering that the types of the security inspection machines are different, the problem that the resolution of the security inspection machine is not high enough, so that a plurality of packages, invalid information and noise interference exist when dangerous goods are detected on the whole security inspection machine image exists. For example, as shown in fig. 3 (a) and 3 (b), a picture acquired by a camera of the security inspection machine is acquired, and all parcel sub-areas in the picture are intercepted by using fast R-CNN. Then, the picture of the parcel subarea is used for subsequent object recognition in a transmission mode and the like.
Step S102: and processing the object subgraphs of all objects in the parcel subarea into a sequence according to a predetermined sequence, and inputting the sequence into a neural network.
In step S102, for example, sequences [ X 1,...,X|D| ] are processed in the order IoU (Intersection over Union, overlap ratio) between SKU items in the picture, and the sequences [ X 1,...,X|D| ] are input into the model by calling the API. Sometimes also ordered by IoU and distances between SKU items in the picture.
The invention introduces the technical means of IoU and distance sorting, thus improving the speed of object recognition, and achieving the technical effect of improving the object recognition rate while saving the cost.
Step S103: and obtaining a subsequence of the object in the parcel subarea based on the article association relation by using the neural network.
In step S103, a part for detection using the target detection network and a part for obtaining a subsequence using a markov transfer chain are included.
The following description is made for the target detection network f (·). As shown in fig. 4 (a) and 4 (b), assuming that the set of SKUs in the warehouse is N and the set of article SKU region subgraphs appearing simultaneously in the package frame is D, for the article subgraph X i e D therein, the target detection network outputs a probability that the probability distribution is any one of |n| SKUs in X i, denoted as f (X i)＝[p1(i),p2(i),...,p|N| (i) ], where p j (i) is the probability of being article j contained in X i. The position of each article region subgraph X i in the package frame needs to be recorded, which isRepresented as coordinates of the upper left and lower right of the item area, respectively. For set D, the probability that the item corresponding to [ X 1,...,X|D| ] in set D is [ h 1,h2,...,h|D| ] isWhere p (h 1,h2,...,h|D|) is the joint probability of the co-occurrence of h 1,h2,...,h|D|. Here the number of the elements is the number,It can be obtained from the neural network f (·), the key here is how to calculate p (h 1,h2,...,h|D|) because of the conditional probability p(hj|h1,h2...hj-1,hj+1,...h|D|)∝p(h1,h2,...,h|D|),. The biggest difficulty among these is how to calculate the joint probability distribution here using a unified approach.
In order to solve the problem, the scheme relates to another module Markov transfer chain, and the object association relation is introduced to the model judgment. For example, assuming that there are multiple items within a picture, each item occurrence may depend on other items due to the consumer's buying habits and item accessory relationships. For example, the appearance of batteries is often an accessory for electronic products, while consumers purchasing milk powder typically purchase paper towel products at the same time. When two articles appear at the same time, a certain position relation exists, the computer battery can appear along with the whole computer, and when in packaging, the articles of the same type, such as the washing liquid and the bath foam, are stored in the same warehouse due to the same storage type, so that the articles are packaged into a whole. While a different type of article may be packaged as another part. So that the association between items that occur within the package actually includes the association between the skus.
Hereinafter, this object is described in detail by taking a dangerous article as an example. The method comprises the steps of forming a candidate set D k by using one or more articles with highest possibility of dangerous articles detected by a target detection network, calculating the positions of other articles, ioU of the positions of the dangerous articles and the sequence of distance from far to near according to a certain possible dangerous article D i, orderly arranging the articles according to IoU from large to small, and then sorting according to the sequence of distance from far to near. The ordered arrangement may also be performed using IoU alone. This part is calculated by the position information obtained by the object detection moduleAnd (3) the product is discharged. Thus, assuming that the result of the sorting of [ h 1,h2,...,h|D| ] by the sorting manner is h 1,h2,...,h|D|, the probability that the SKU corresponding to [ X 1,...,X|D| ] is [ h 1,h2,...,h|D| ] isWhere p (h i+1|hi) is h i and h i+1 occur simultaneously, and h i+1 is the first item in h i order.
As described above, the technical means of IoU and distance sorting are introduced into the invention, so that the dangerous goods identification rate is improved, and the technical effect of saving the cost and improving the dangerous goods identification rate is achieved.
As shown in fig. 4 (a), 4 (b), assuming that the values of all parameters in the target detection network f (·) and the combinations h i and h j,p(hj|hi) for any two items are known, then given the sequence of item subgraphs within the package [ X 1,...,X|D| ], the choice is made such that the probabilities areThe largest sequence of items, i.e./>Calculating this probability requires a high time complexity. To simplify the complexity of the model, the dependence relationship of the part of the stock quantity units containing the affiliated relationship with a certain Markov chain is utilized, such as a battery- > a remote controller- > a four-wheel drive.
And a dynamic programming method is used for solving, so that the operation efficiency is improved. Due to
Then can enableThe largest sequence necessarily involves making
The largest subsequence. Thus, the viterbi algorithm can be used to solve for
In this step, since the goal is to detect objects without completely calculating the joint probability of all items in the package, the problem is reduced to a partial sub-sequence problem, and only each item D i in the hazardous materials inventory set D k is considered and the nearest item D i+1 is ordered according to IoU, so the problem becomes a solution
Finally, the most likely dangerous article type subsequence is obtained.
The training process for the neural network mainly comprises the following steps:
collecting historical security inspection machine pictures, and selecting pictures in which a plurality of packages exist and possibly contain a plurality of articles;
Using fast R-CNN for package detection on all pictures, and extracting package subgraphs;
marking each package subgraph, sequencing pictures with a plurality of articles in one picture, and marking the corresponding articles and the positions thereof;
training the target detection network according to the marked corresponding articles and the positions of the corresponding articles; and
P (h j|hi) (i.e., condition profile or correlation matrix or database) is learned according to IoU sequence labeling, and neural network f (·) is tuned.
The neural network structure in the present invention will be described with reference to fig. 5. Here, the object is still described in detail as an example of a dangerous article. As shown in FIG. 5, the first layer from top to bottom represents the sub-graph [ X 1,...,X|D| ] obtained by detecting the original image, namely the target detection model, the second layer is the neural network f (·), and the third layer is the Markov chain.
In order to implement the above-described procedure, the first step is to train the parcel detection model. The package detection may be performed using a common target detection method, such as Faster R-CNN, and then learning parameters and p (h j|hi) in the target detection network. The learning process is divided into two parts of training and tuning. Wherein, the training part only trains the neural network f (·) and the tuning part learns the neural network f (·) and the probability p (h j|hi) together. Before learning p (h j|hi), the association relation of stock units, such as collaborative filtering matrix on e-commerce website, can be used for probability normalization as an initialization matrix, so that a transition matrix can be learned more quickly. The training process for f (·) is: selecting a certain amount of original detection pictures of the security inspection machine, and marking, wherein the marking contents are as follows: which items are contained in a single package map, the area of each item.
TABLE 1
| hj/hi | Battery cell | Remote controller | Toy car | Square button |
| Battery cell | -- | 0.7 | 0.25 | 0.05 |
| Remote controller | 0.23 | -- | 0.76 | 0.01 |
| Toy car | 0.12 | 0.84 | -- | 0.04 |
| Square button | 0.33 | 0.33 | 0.33 | -- |
TABLE 1 initial correlation matrix (collaborative filtering matrix)
After labeling, the optimization is performed by a random gradient descent method using a multitasking loss function. In the structure selection of f (-), faster R-CNN is selected, but the target detection method can also be selected in practical use. And in the tuning part, selecting a large number of marked single package data, extracting an article area subgraph from the single package data, sorting the single package data according to a IoU sorting mode by taking the selected minimum/maximum article as a sorting starting point, and marking the character articles in the single package data. For each tagged sequence [ X 1,...,X|D| ], assuming its corresponding stock unit is [ h 1,h2,...,h|D| ], it is a negative maximum likelihoodTraining was performed using random gradient descent for the loss function. And finally training the obtained model for identifying dangerous goods of the security inspection machine.
For example, the probability values for individual inventory units detected by a single package are shown in table 1, the smallest area inventory unit may be battery=0.5 or square button=0.5, remote control=0.78, toy vehicle=0.9; in combination with the initial correlation matrix shown in table 1, p (battery|remote control) =0.7, p (square button|remote control) =0.33, p (remote control|toy car) =0.76 can calculate the probability maximum subsequence:
Battery- > remote control- > toy car 0.5 x (0.7 x 0.78) x (0.76 x 0.9) = 0.1867
Square button- > remote control- > toy car 0.5 x (0.33 x 0.78), (0.76 x 0.9) = 0.1334
Due to the probability maximum subsequence: battery- > remote control- > toy vehicle, so at the time of prediction the outcome is: the battery- > remote controller- > the sku sequence of the toy car, the condition probability among each sku of the reverse iteration table, namely, the association matrix of the sku is updated by determining the maximum subsequence of the mark during training.
Since the structure of the Markov chain used in the present invention is much faster than the structure of the object relation module (Object Relation Module), especially in the inference phase, the approach to search is much closer than classification (refer to face recognition techniques). In addition, the invention adopts the technical means of modeling the conditional probability that a plurality of articles appear simultaneously in the form of Markov transition probability, thereby improving the accuracy of object identification, and achieving the technical effects of improving the object identification speed and the identification rate while saving the cost.
Step S104: and carrying out object recognition according to the subsequence.
Specifically, the maximum subsequence obtained in step S103 is automatically compared with a dangerous goods list which is an object and is recorded in advance, for example, the subsequence appears as (battery, remote controller, toy car); the list of dangerous goods is (pipe cutters, batteries, liquids, explosives). And when the battery is contained in the sub-sequence, an automatic alarm is given to prompt that dangerous goods exist.
In addition to the steps S101 to S104, the present application may further include step S105: and updating the corresponding item position database based on the item association relation by using the conditional probability so as to tune the sub-neural network in the neural network. Specifically, the conditional probability among the stock units of the iterative table is reversed by determining the maximum subsequence of the marker during training, i.e., updating the correlation matrix of the stock units. Thus, collaborative filtering matrices between SKU items for package shipping, such as content-based collaborative filtering matrices for various items purchased via an e-commerce, may be obtained after neural network training. Because of the split property of the Markov chain module, the article association matrix is iteratively updated at the same time, and a new article association matrix is obtained, so that the method can be utilized in the e-commerce recommending stage and the package mailing stage of logistics.
In addition, according to the above-described embodiment of the present application, as compared with a method using, for example, a relationship network (Relation Networks for Object Detection) for target detection or the like, since it is not necessary to add features of different objects in a picture together to a network structure of an object relationship module (Object Relation Module) of a subsequent design of target detection, it is possible to reduce the network structure of a model, thereby shortening the time for calculation.
Fig. 2 is a schematic diagram of main modules of an apparatus for object recognition based on an item association relationship according to an embodiment of the present invention, and as shown in fig. 2, the method includes: the package detection module 201: carrying out package detection on an acquisition picture acquired by a shooting device, and acquiring package subregions corresponding to each package in the acquisition picture; sub-graph ordering module 202: processing object subgraphs of all objects in the parcel subarea into sequences according to a predetermined sequence, and inputting the sequences into a neural network; the sub-sequence acquisition module 203: obtaining a subsequence of the object in the parcel subarea based on the article association relationship by using the neural network; object recognition module 204: performing object recognition according to the subsequence, wherein the neural network is trained based on training samples: collecting pictures with a plurality of packages and including a plurality of items in the packages to generate the training sample; performing package detection on the picture; and processing the object subgraphs of the plurality of objects contained in the obtained package subgraphs into a sequence, and marking the corresponding objects and the positions thereof to establish a corresponding object position database based on the object association relationship. Hereinafter, each of the above modules will be specifically described.
The package detection module 201: and carrying out package detection on the acquired pictures acquired by the shooting device, and acquiring package subregions corresponding to all packages in the acquired pictures. The package detection module 201 is mainly used for package detection, and the package detection module 201 in the invention can use a more mature module commonly used in the prior art. There is no particular limitation as long as it can be used for package detection. However, considering that the types of the security inspection machines are different, the problem that the resolution of the security inspection machine is not high enough, so that a plurality of packages, invalid information and noise interference exist when dangerous goods are detected on the whole security inspection machine image exists. For example, as shown in fig. 3 (a) and 3 (b), a picture acquired by a camera of the security inspection machine is acquired, and all parcel sub-areas in the picture are intercepted by using fast R-CNN. The picture of the parcel sub-region is then used in subsequent object recognition, such as sub-image ordering module 202, by transmission or the like.
Sub-graph ordering module 202: and processing the object subgraphs of all objects in the parcel subarea into a sequence according to a predetermined sequence, and inputting the sequence into a neural network.
In the sub-graph ordering module 202, sequences [ X 1,...,X|D| ] are processed, for example, in accordance with IoU (Intersection over Union, overlap ratio) ordering among SKU items in the picture, and the sequences [ X 1,...,X|D| ] are input into the model by way of calling an API. Sometimes also ordered by IoU and distances between SKU items in the picture.
The invention introduces the technical means of IoU and distance sorting, thus improving the speed of object recognition, and achieving the technical effect of improving the object recognition rate while saving the cost.
The sub-sequence acquisition module 303: and obtaining a subsequence of the object in the parcel subarea based on the article association relation by using the neural network.
In step S103, a part for detection using the target detection network and a part for obtaining a subsequence using a markov transfer chain are included.
The following description is made for the target detection network f (·). As shown in fig. 4 (a) and 4 (b), assuming that the set of SKUs in the warehouse is N and the set of article SKU region subgraphs appearing simultaneously in the package frame is D, for the article subgraph X i e D therein, the target detection network outputs a probability that the probability distribution is any one of |n| SKUs in X i, denoted as f (X i)＝[p1(i),p2(i),...,p|N| (i) ], where p j (i) is the probability of being article j contained in X i. The position of each article region subgraph X i in the package frame needs to be recorded, which isRepresented as coordinates of the upper left and lower right of the item area, respectively. For set D, the probability that the item corresponding to [ X 1,...,X|D| ] in set D is [ h 1,h2,...,h|D| ] isWhere p (h 1,h2,...,h|D|) is the joint probability of the co-occurrence of h 1,h2,...,h|D|. Here the number of the elements is the number,Can be obtained from the neural network f (·) due to conditional probability
p(hj|h1,h2...hj-1,hj+1,...h|D|)∝p(h1,h2,...,h|D|), Then the key here is how to calculate p (h 1,h2,...,h|D|). The biggest difficulty among these is how to calculate the joint probability distribution here using a unified approach.
In order to solve the problem, the scheme relates to another module Markov transfer chain, and the object association relation is introduced to the model judgment. For example, assuming that there are multiple items within a picture, each item occurrence may depend on other items due to the consumer's buying habits and item accessory relationships. For example, the appearance of batteries is often an accessory for electronic products, while consumers purchasing milk powder typically purchase paper towel products at the same time. When two articles appear at the same time, a certain position relation exists, the computer battery can appear along with the whole computer, and when in packaging, the articles of the same type, such as the washing liquid and the bath foam, are stored in the same warehouse due to the same storage type, so that the articles are packaged into a whole. While a different type of article may be packaged as another part. So that the association between items that occur within the package actually includes the association between the skus.
Hereinafter, this object is described in detail by taking a dangerous article as an example. The candidate set D k is formed by using one or a plurality of articles with highest possibility of dangerous articles detected by the target detection network, one possible dangerous article D i is used for calculating the positions of other articles, ioU (Intersection over Union, overlapping rate) of the positions of the dangerous articles and the sequence of distance, the other articles are orderly arranged according to IoU from large to small, and then the other articles are orderly arranged according to the sequence from far to near. The ordered arrangement may also be performed using IoU alone. This part is calculated by the position information obtained by the object detection moduleAnd (3) the product is discharged. Thus, assuming that the result of the sorting of [ h 1,h2,...,h|D| ] by the sorting manner is h 1,h2,...,h|D|, the probability that the stock quantity unit corresponding to [ X 1,...,X|D| ] is [ h 1,h2,...,h|D| ] is
Where p (h i+1|hi) is h i and h i+1 occur simultaneously, and h i+1 is the first item in h i order.
As described above, the technical means of IoU and distance sorting are introduced into the invention, so that the dangerous goods identification rate is improved, and the technical effect of saving the cost and improving the dangerous goods identification rate is achieved.
As shown in fig. 4 (a), 4 (b), assuming that the values of all parameters in the target detection network f (·) and the combinations h i and h j,p(hj|hi) for any two items are known, then given the sequence of item subgraphs within the package [ X 1,...,X|D| ], the choice is made such that the probabilities areThe largest sequence of items, i.e./>Calculating this probability requires a high time complexity. To simplify the complexity of the model, the dependence relationship of the part of the stock quantity units containing the affiliated relationship with a certain Markov chain is utilized, such as a battery- > a remote controller- > a four-wheel drive.
And a dynamic programming method is used for solving, so that the operation efficiency is improved. Due to
Then can enableThe largest sequence necessarily involves making
The largest subsequence. Thus, the viterbi algorithm can be used to solve for
In the subsequence acquisition module 203, since the goal is to detect objects and not to completely calculate the joint probability of occurrence of all items in the package, the problem is reduced to a partial subsequence problem, and only each item D i in the dangerous goods inventory set D k is determined to be considered, and the nearest item D i+1 is ordered according to IoU, so the problem becomes a solution
Finally, the most likely dangerous article type subsequence is obtained.
The training process for the neural network mainly comprises the following steps:
collecting historical security inspection machine pictures, and selecting pictures in which a plurality of packages exist and possibly contain a plurality of articles;
Using fast R-CNN for package detection on all pictures, and extracting package subgraphs;
marking each package subgraph, sequencing pictures with a plurality of articles in one picture, and marking the corresponding articles and the positions thereof;
training the target detection network according to the marked corresponding articles and the positions of the corresponding articles; and
P (h j|hi) (i.e., condition profile or correlation matrix or database) is learned according to IoU sequence labeling, and neural network f (·) is tuned.
The neural network structure in the present invention will be described with reference to fig. 5. Here, the object is still described in detail as an example of a dangerous article. As shown in FIG. 5, the first layer from top to bottom represents the sub-graph [ X 1,...,X|D| ] obtained by detecting the original image, namely the target detection model, the second layer is the neural network f (·), and the third layer is the Markov chain.
In order to implement the above-described procedure, the first step is to train the parcel detection model. The package detection may be performed using a common target detection method, such as Faster R-CNN, and then learning parameters and p (h j|hi) in the target detection network. The learning process is divided into two parts of training and tuning. Wherein, the training part only trains the neural network f (·) and the tuning part learns the neural network f (·) and the probability p (h j|hi) together. Before learning p (h j|hi), the association relation of stock units, such as collaborative filtering matrix on e-commerce website, can be used for probability normalization as an initialization matrix, so that a transition matrix can be learned more quickly. The training process for f (·) is: selecting a certain amount of original detection pictures of the security inspection machine, and marking, wherein the marking contents are as follows: which items are contained in a single package map, the area of each item.
TABLE 1
| hj/hi | Battery cell | Remote controller | Toy car | Square button |
| Battery cell | -- | 0.7 | 0.25 | 0.05 |
| Remote controller | 0.23 | -- | 0.76 | 0.01 |
| Toy car | 0.12 | 0.84 | -- | 0.04 |
| Square button | 0.33 | 0.33 | 0.33 | -- |
TABLE 1 initial correlation matrix (collaborative filtering matrix)
After labeling, the optimization is performed by a random gradient descent method using a multitasking loss function. In the structure selection of f (-), faster R-CNN is selected, but the target detection method can also be selected in practical use. And in the tuning part, selecting a large number of marked single package data, extracting an article area subgraph from the single package data, sorting the single package data according to a IoU sorting mode by taking the selected minimum/maximum article as a sorting starting point, and marking the character articles in the single package data. For each tagged sequence [ X 1,...,X|D| ], assuming its corresponding stock unit is [ h 1,h2,...,h|D| ], it is a negative maximum likelihoodTraining was performed using random gradient descent for the loss function. And finally training the obtained model for identifying dangerous goods of the security inspection machine.
For example, the probability values for individual inventory units detected by a single package are shown in table 1, the smallest area inventory unit may be battery=0.5 or square button=0.5, remote control=0.78, toy vehicle=0.9; in combination with the initial correlation matrix shown in table 1, p (battery|remote control) =0.7, p (square button|remote control) =0.33, p (remote control|toy car) =0.76 can calculate the probability maximum subsequence:
Battery- > remote control- > toy car 0.5 x (0.7 x 0.78) x (0.76 x 0.9) = 0.1867
Square button- > remote control- > toy car 0.5 x (0.33 x 0.78), (0.76 x 0.9) = 0.1334
Due to the probability maximum subsequence: battery- > remote control- > toy vehicle, so at the time of prediction the outcome is: the battery- > remote controller- > the sku sequence of the toy car, the condition probability among each sku of the reverse iteration table, namely, the association matrix of the sku is updated by determining the maximum subsequence of the mark during training.
Since the structure of the Markov chain used in the present invention is much faster than the structure of the object relation module (Object Relation Module), especially in the inference phase, the approach to search is much closer than classification (refer to face recognition techniques). In addition, the invention adopts the technical means of modeling the conditional probability that a plurality of articles appear simultaneously in the form of Markov transition probability, thereby improving the accuracy of object identification, and achieving the technical effects of improving the object identification speed and the identification rate while saving the cost.
Object recognition module 404: and carrying out object recognition according to the subsequence.
Specifically, the largest subsequence obtained in the subsequence obtaining module 303 is automatically compared with a dangerous article list which is an object and is recorded in advance, for example, the occurring subsequence is (battery, remote controller, toy car); the list of dangerous goods is (pipe cutters, batteries, liquids, explosives). And when the battery is contained in the sub-sequence, an automatic alarm is given to prompt that dangerous goods exist.
In addition to the above modules 201 to 204, the present application may further include a tuning module 205: and updating the corresponding article position database by using the conditional probability so as to tune the sub-neural network in the neural network. Specifically, the conditional probability among the stock units of the iterative table is reversed by determining the maximum subsequence of the marker during training, i.e., updating the correlation matrix of the stock units. Thus, collaborative filtering matrices between SKU items for package shipping, such as content-based collaborative filtering matrices for various items purchased via an e-commerce, may be obtained after neural network training. Because of the split property of the Markov chain module, the article association matrix is iteratively updated at the same time, and a new article association matrix is obtained, so that the method can be utilized in the e-commerce recommending stage and the package mailing stage of logistics.
In addition, according to the above-described embodiment of the present application, as compared with a method using, for example, a relationship network (Relation Networks for Object Detection) for target detection or the like, since it is not necessary to add features of different objects in a picture together to a network structure of an object relationship module (Object Relation Module) of a subsequent design of target detection, it is possible to reduce the network structure of a model, thereby shortening the time for calculation.
Fig. 6 illustrates an exemplary system architecture 600 of a method of item association based object recognition or an apparatus of item association based object recognition to which embodiments of the present invention may be applied.
As shown in fig. 6, the system architecture 600 may include terminal devices 601, 602, 603, a network 604, and a server 605. The network 604 is used as a medium to provide communication links between the terminal devices 601, 602, 603 and the server 605. The network 604 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 605 via the network 604 using the terminal devices 601, 602, 603 to receive or send messages, etc. Various communication client applications such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only) may be installed on the terminal devices 601, 602, 603.
The terminal devices 601, 602, 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 605 may be a server providing various services, such as a background management server (by way of example only) providing support for shopping-type websites browsed by users using terminal devices 601, 602, 603. The background management server may analyze and process the received data such as the product information query request, and feedback the processing result (e.g., the target push information, the product information—only an example) to the terminal device.
It should be noted that, the method for identifying an object based on an article association relationship according to the embodiment of the present invention is generally executed by the server 605, and accordingly, the device for identifying an object based on an article association relationship is generally disposed in the server 605.
It should be understood that the number of terminal devices, networks and servers in fig. 6 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 7, there is illustrated a schematic diagram of a computer system 700 suitable for use in implementing an embodiment of the present invention. The terminal device shown in fig. 7 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 7, the computer system 700 includes a Central Processing Unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, ROM 702, and RAM 703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
The following components are connected to the I/O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output portion 707 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like; a storage section 708 including a hard disk or the like; and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. The drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read therefrom is mounted into the storage section 708 as necessary.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 709, and/or installed from the removable medium 711. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 701.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, for example, as: a processor includes a package detection module, a sub-graph ordering module, a sub-sequence acquisition module, and an object identification module. The names of these modules do not limit the module itself in some cases, and for example, the package detection module may also be described as a "module for package detection of acquired images".
As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to perform comprising: carrying out package detection on an acquisition picture acquired by a shooting device, and acquiring package subregions corresponding to each package in the acquisition picture; processing object subgraphs of all objects in the parcel subarea into sequences according to a predetermined sequence, and inputting the sequences into a neural network; obtaining a subsequence of the object in the parcel subarea based on the article association relationship by using the neural network; and performing object recognition according to the subsequence, wherein the neural network is trained based on training samples: collecting pictures with a plurality of packages and including a plurality of items in the packages to generate the training sample; performing package detection on the picture; and processing the object subgraphs of the plurality of objects contained in the obtained package subgraphs into a sequence, and marking the corresponding objects and the positions thereof to establish a corresponding object position database based on the object association relationship.
According to the technical scheme provided by the embodiment of the invention, the technical means such as the correlation of the SKU articles sold by the electronic commerce (namely, the correlation matrix of the SKU articles is used in the initialization) is directly used for assisting the object recognition, so that the cost of training and learning is reduced, the technical problems that the object is difficult to recognize when the size of the object is small or the shape difference is large and the detection model is poor in universality when the color imaging is performed are solved, and the technical effect that the object recognition rate can be improved while the cost is reduced is achieved.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (8)
1. A method for object recognition based on article association, comprising:
Carrying out package detection on an acquisition picture acquired by a shooting device, and acquiring package subregions corresponding to each package in the acquisition picture;
Processing object subgraphs of all objects in the parcel subarea into sequences according to a predetermined sequence, and inputting the sequences into a neural network; the method specifically comprises the following steps: ordering by IoU between the particular items, or ordering by IoU and distance between the particular items;
Obtaining a subsequence of the object in the parcel subarea based on the article association relationship by using the neural network; the method specifically comprises the following steps: detecting the probability that the articles in the parcel subarea are warehouse SKUs by using a neural network, and obtaining the subsequences by using a Markov transfer chain, wherein the Markov transfer chain comprises article association relations; and
Object recognition is performed on the basis of the sub-sequences,
Wherein, neural network trains based on training sample and forms: collecting pictures with a plurality of packages and including a plurality of items in the packages to generate the training sample; performing package detection on the picture; and processing the object subgraphs of the plurality of objects contained in the obtained package subgraphs into a sequence, and marking the corresponding objects and the positions thereof to establish a corresponding object position database based on the object association relationship.
2. The method of claim 1, wherein the step of determining the position of the probe comprises,
When training the neural network, training a network for target detection in the neural network according to the marked article and the position of the marked article.
3. The method of claim 2, wherein,
And when training the neural network, updating the corresponding article position database by using conditional probability so as to tune the sub-neural network in the neural network.
4. The method of claim 1, wherein the step of determining the position of the probe comprises,
After sorting from large to small by the IoU, sorting from far to near by the distance.
5. The method of claim 3, wherein,
And using the updated corresponding article position database in an e-commerce recommending stage and a package mailing stage of logistics.
6. An apparatus for object recognition based on article association, comprising:
The package detection module is used for carrying out package detection on the acquired pictures acquired by the shooting device and acquiring package subregions corresponding to each package in the acquired pictures;
The sub-graph sorting module processes the object sub-graphs of all objects in the parcel sub-area into a sequence according to a predetermined sequence, and inputs the sequence into a neural network; the method is particularly used for: ordering by IoU between the particular items, or ordering by IoU and distance between the particular items;
The subsequence acquisition module is used for acquiring a subsequence of the object in the parcel subarea based on the article association relationship by using the neural network; the method is particularly used for: detecting the probability that the articles in the parcel subarea are warehouse SKUs by using a neural network, and obtaining the subsequences by using a Markov transfer chain, wherein the Markov transfer chain comprises an article association relation and an article association relation
An object recognition module for performing object recognition according to the subsequence,
Wherein, neural network trains based on training sample and forms: collecting pictures with a plurality of packages and including a plurality of items in the packages to generate the training sample; performing package detection on the picture; and processing the object subgraphs of the plurality of objects contained in the obtained package subgraphs into a sequence, and marking the corresponding objects and the positions thereof to establish a corresponding object position database based on the object association relationship.
7. An electronic device for object recognition based on article association, comprising:
One or more processors;
Storage means for storing one or more programs,
When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-5.
8. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110188078.0A CN113762023B (en) | 2021-02-18 | 2021-02-18 | Object identification method and device based on article association relation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110188078.0A CN113762023B (en) | 2021-02-18 | 2021-02-18 | Object identification method and device based on article association relation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113762023A CN113762023A (en) | 2021-12-07 |
| CN113762023B true CN113762023B (en) | 2024-05-24 |
Family
ID=78786719
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110188078.0A Active CN113762023B (en) | 2021-02-18 | 2021-02-18 | Object identification method and device based on article association relation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113762023B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114581803A (en) * | 2022-03-01 | 2022-06-03 | 北京京东振世信息技术有限公司 | Article identification processing method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102053940B1 (en) * | 2019-05-07 | 2019-12-11 | 주식회사 두잉랩 | System and method for estimating object using association analysis between multiple object |
| CN111428536A (en) * | 2019-01-09 | 2020-07-17 | 北京京东尚科信息技术有限公司 | Training method and device for detection network for detecting article category and position |
| WO2020164282A1 (en) * | 2019-02-14 | 2020-08-20 | 平安科技（深圳）有限公司 | Yolo-based image target recognition method and apparatus, electronic device, and storage medium |
| WO2020187077A1 (en) * | 2019-03-21 | 2020-09-24 | 长安大学 | Deep neural network-based security check system and method |
| CN112184626A (en) * | 2020-09-02 | 2021-01-05 | 珠海格力电器股份有限公司 | Gesture recognition method, device, equipment and computer readable medium |
| CN112348015A (en) * | 2020-11-09 | 2021-02-09 | 厦门市美亚柏科信息股份有限公司 | Character detection method and device based on cascade neural network and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018176017A1 (en) * | 2017-03-24 | 2018-09-27 | Revealit Corporation | Method, system, and apparatus for identifying and revealing selected objects from video |
-
2021
- 2021-02-18 CN CN202110188078.0A patent/CN113762023B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111428536A (en) * | 2019-01-09 | 2020-07-17 | 北京京东尚科信息技术有限公司 | Training method and device for detection network for detecting article category and position |
| WO2020164282A1 (en) * | 2019-02-14 | 2020-08-20 | 平安科技（深圳）有限公司 | Yolo-based image target recognition method and apparatus, electronic device, and storage medium |
| WO2020187077A1 (en) * | 2019-03-21 | 2020-09-24 | 长安大学 | Deep neural network-based security check system and method |
| KR102053940B1 (en) * | 2019-05-07 | 2019-12-11 | 주식회사 두잉랩 | System and method for estimating object using association analysis between multiple object |
| CN112184626A (en) * | 2020-09-02 | 2021-01-05 | 珠海格力电器股份有限公司 | Gesture recognition method, device, equipment and computer readable medium |
| CN112348015A (en) * | 2020-11-09 | 2021-02-09 | 厦门市美亚柏科信息股份有限公司 | Character detection method and device based on cascade neural network and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 基于Faster R-CNN的除草机器人杂草识别算法;李春明;逯杉婷;远松灵;王震洲;;中国农机化学报(12);全文 * |
| 基于Faster-RCNN的快速目标检测算法;曹之君;张良;;航天控制(04);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113762023A (en) | 2021-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11232324B2 (en) | Methods and apparatus for recommending collocating dress, electronic devices, and storage media | |
| CN107798571B (en) | Malice address/malice order identifying system, method and device | |
| US10282722B2 (en) | Machine learning system, method, and program product for point of sale systems | |
| CN107330445B (en) | User attribute prediction method and device | |
| CN107330752B (en) | Method and device for identifying brand words | |
| CA2917256C (en) | Screenshot-based e-commerce | |
| US11741094B2 (en) | Method and system for identifying core product terms | |
| CN110827112B (en) | Deep learning commodity recommendation method and device, computer equipment and storage medium | |
| US20220188660A1 (en) | Systems and methods for processing data for storing in a feature store and for use in machine learning | |
| US9892437B2 (en) | Digitization of a catalog of retail products | |
| CN110942035A (en) | Method, system, device and storage medium for acquiring commodity information | |
| CN111782850B (en) | Object searching method and device based on hand drawing | |
| CN113762023B (en) | Object identification method and device based on article association relation | |
| CN113781149A (en) | Information recommendation method and device, computer-readable storage medium and electronic equipment | |
| CN109993619A (en) | Data processing method | |
| CN118193806A (en) | Target retrieval method, target retrieval device, electronic equipment and storage medium | |
| CN111833085A (en) | Method and device for calculating price of article | |
| CN112395398A (en) | Question and answer processing method, device and equipment | |
| CN111753181A (en) | Image-based search method, apparatus, server, client, and medium | |
| CN110751496B (en) | Commodity price detection method and device | |
| CN108121969A (en) | For handling the method and apparatus of image | |
| Shinde et al. | ML-based smart shopping system with recipe recommendation | |
| CN112861684A (en) | Article display method and device | |
| CN113342969A (en) | Data processing method and device | |
| CN107341161A (en) | A kind of information query method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |