CN118097195B

CN118097195B - Commodity and price tag matching method, equipment and storage medium

Info

Publication number: CN118097195B
Application number: CN202410493289.9A
Authority: CN
Inventors: 杨恒; 龙涛; 余文炫; 李轩; 吴永杰; 李娟�; 陈序
Original assignee: Shenzhen Aimo Technology Co ltd
Current assignee: Shenzhen Aimo Technology Co ltd
Priority date: 2024-04-23
Filing date: 2024-04-23
Publication date: 2024-07-23
Anticipated expiration: 2044-04-23
Also published as: CN118097195A

Abstract

The invention relates to the technical field of commodity price tag matching, in particular to a matching method of commodities and price tags, which comprises the following steps: acquiring a goods shelf image of a goods shelf; processing the goods shelf image to obtain price tag images and price tag position information of each price tag of the plurality of price tags, and commodity images and commodity position information of each commodity of the plurality of commodities; performing text recognition processing on the price tag image of each price tag and the commodity image of each commodity to obtain text information of each price tag and text information of each commodity; obtaining a matching score between each price tag and each commodity according to the price tag position information and the text information of each price tag and the commodity position information and the text information of each commodity, and obtaining a matching table according to the matching score between each price tag and each commodity; and carrying out dynamic programming processing on the matching table to obtain a matching result of the target commodity of each price tag. The method improves the accuracy, the robustness and the reliability of the matching process.

Description

Commodity and price tag matching method, equipment and storage medium

Technical Field

The present invention relates to the field of commodity price tag matching technologies, and in particular, to a method, an apparatus, and a storage medium for matching a commodity with a price tag.

Background

In the retail industry or in the goods shelves application process, it is an important task to match goods accurately to their corresponding price tags. Due to the factors of excessive quantity and variety of commodities, quick change of stock and the like, the commodities are required to be manually matched with corresponding price tags. Manually matching merchandise and price tags is often time consuming and error prone. To solve this problem, computer vision technology has been widely used in recent years to automate the process of matching goods and price tags.

Then, the matching process of the commodity and the price tag is realized through a computer vision technology, and a single-dimension judgment standard is generally adopted. For example, the commodity and the price tag are matched by a criterion of the positional information of the commodity. Or matching the commodity with the price tag according to the text information judgment standard identified by the commodity. In the actual matching process of the price tag and the commodity, the problem of price tag placement position deviation exists, namely one price tag is correspondingly placed at the middle position of the two commodities. Therefore, the matching error of the commodity and the price tag or the problem of low matching precision is easy to occur by the existing matching method of the commodity and the price tag.

Therefore, the prior art for realizing the matching process of the commodity and the price tag has the problem of insufficient accuracy and robustness.

Disclosure of Invention

The embodiment of the application solves the technical problem of insufficient accuracy and robustness in the matching process of the commodity and the price tag in the prior art by providing the matching method, the device and the storage medium of the commodity and the price tag, realizes the process of matching the commodity and the price tag quickly and accurately, improves the accuracy, the robustness and the reliability of the matching process, and improves the matching efficiency and other technical effects.

In a first aspect, an embodiment of the present invention provides a method for matching a commodity with a price tag, including:

acquiring a goods shelf image of a goods shelf;

Performing target detection processing on the shelf image to obtain price tag images and price tag position information of each price tag in the plurality of price tags, and performing semantic segmentation processing on the shelf image to obtain commodity images and commodity position information of each commodity in the plurality of commodities, wherein each commodity is an independent commodity or a commodity class;

performing text recognition processing on the price tag image of each price tag and the commodity image of each commodity to obtain text information of each price tag and text information of each commodity;

Obtaining a matching score between each price tag and each commodity according to the price tag position information and the text information of each price tag and the commodity position information and the text information of each commodity, and obtaining a matching table according to the matching score between each price tag and each commodity;

and carrying out dynamic programming processing on the matching table to obtain a matching result of the target commodity of each price tag.

Preferably, the dynamically planning the matching table to obtain a matching result of the target commodity of each price tag includes:

sorting the plurality of price tags according to a preset sequence;

Under the current state of the matching table, carrying out dynamic matching processing on one current price tag in the plurality of price tags to obtain a target commodity of the current price tag and a matching table updated by the current price tag, and taking the matching table updated by the current price tag as the next state of the current state of the matching table;

Under the next state of the current state of the matching table, carrying out dynamic matching processing on the next price of the current price to obtain a target commodity of the next price and a matching table updated by the next price, and taking the matching table updated by the next price as the next state of the matching table;

And in the next state of the matching table, carrying out dynamic matching processing on the next price tag until the last price tag in the plurality of price tags is subjected to dynamic matching processing in the last state of the matching table, and obtaining a matching result of the target commodity of each price tag.

Preferably, the dynamic matching process includes:

In the current state of the matching table, aiming at each current price tag, finding a target commodity of the current price tag according to the matching score between the current price tag and each commodity, wherein the target commodity is the commodity with the highest matching score with the current price tag;

Setting the matching score between the target commodity and each price tag except the current price tag as a preset value, and updating the matching table to obtain a matching table updated by the current price tag, wherein the preset value indicates that no matching relationship exists between the commodity and the price tag.

Sorting the plurality of commodities according to a preset sequence;

Under the current state of the matching table, carrying out dynamic matching processing on one current commodity in the plurality of commodities to obtain a target price tag of the current commodity and a matching table updated by the current commodity, and taking the matching table updated by the current commodity as the next state of the current state of the matching table;

Under the next state of the current state of the matching table, carrying out dynamic matching processing on the next commodity of the current commodity to obtain a target price tag of the next commodity and a matching table updated by the next commodity, and taking the matching table updated by the next commodity as the next state of the matching table;

And in the next state of the matching table, carrying out dynamic matching processing on the next commodity until each price tag in the matching table is subjected to dynamic matching processing, and obtaining a matching result of the target price tag of each commodity.

Preferably, the obtaining the matching score between each price tag and each commodity according to the price tag position information and the text information of each price tag and the commodity position information and the text information of each commodity includes:

aiming at each price tag and each commodity, obtaining the cross ratio between the price tag and the commodity according to the price tag position information of the price tag and the commodity position information of the commodity;

Obtaining a text distance between the price tag and the commodity according to the commodity name in the text information of the price tag and the commodity name in the text information of the commodity;

Obtaining the center point distance between the price tag and the corresponding commodity according to the price tag position information of the price tag and the commodity position information of the commodity corresponding to the price tag, and setting the center point distance between the price tag and the commodity except the corresponding commodity as a center point distance appointed value;

Obtaining a matching score between the price tag and the commodity according to the intersection ratio, the text distance and the center point distance between the price tag and the commodity;

And executing the matching score acquisition process on each price tag and each commodity to obtain the matching score between each price tag and each commodity.

Preferably, the text recognition processing is performed on the price tag image of each price tag and the commodity image of each commodity to obtain text information of each price tag and text information of each commodity, including:

Performing text recognition processing on the price tag image of each price tag through an OCR model to obtain text information of each price tag;

And carrying out text recognition processing on the commodity images of each commodity through an OCR model or an image recognition model to obtain text information of each commodity.

Preferably, the performing object detection processing on the shelf image to obtain a price tag image and price tag position information of each price tag in the plurality of price tags includes:

The target detection processing is carried out on the shelf images, so that a detection frame of each price tag is obtained;

And performing perspective correction processing on the detection frames of each price tag to obtain corrected detection frames of each price tag, and obtaining price tag images and price tag position information of each price tag based on the corrected detection frames of each price tag.

Preferably, the semantic division processing is performed on the shelf image to obtain a commodity image and commodity position information of each commodity in the plurality of commodities, including:

Under the condition that each commodity is a commodity type, carrying out semantic segmentation processing on the shelf image to obtain a segmented region image of each independent commodity and position coordinates of the segmented region image of each independent commodity, wherein the segmented region image of each independent commodity is used as a commodity image of each independent commodity, and the position coordinates of the segmented region image of each independent commodity are used as commodity position information of each independent commodity;

and carrying out continuous package calculation processing according to the segmented region images of each independent commodity to obtain a commodity image of each commodity in a plurality of commodity classes, and obtaining commodity position information of each commodity class according to the commodity image of each commodity class.

Based on the same inventive concept, in a second aspect, the present invention provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, said processor implementing the steps of the matching method of goods and price tags when executing said program.

Based on the same inventive concept, in a third aspect, the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of a matching method of goods and price tags.

One or more technical solutions in the embodiments of the present invention at least have the following technical effects or advantages:

In the embodiment of the invention, after the goods shelf image of the goods shelf is acquired, the goods shelf image is subjected to target detection processing to obtain the price tag image and the price tag position information of each price tag in the plurality of price tags, and the goods shelf image is subjected to semantic segmentation processing to obtain the goods image and the goods position information of each goods in the plurality of goods, wherein each goods is an independent goods or a goods class. And then carrying out text recognition processing on the price tag image of each price tag and the commodity image of each commodity to obtain text information of each price tag and text information of each commodity. And obtaining the matching score between each price tag and each commodity according to the price tag position information and the text information of each price tag and the commodity position information and the text information of each commodity. And obtaining a matching table according to the matching score between each price tag and each commodity. In this way, the direct matching degree of the price tag and the commodity in the three dimensions is obtained based on the respective position information of the price tag and the commodity, the image characteristic information of the commodity, and the respective commodity names of the price tag and the commodity. And the matching score between each price tag and each commodity is obtained by fully utilizing the related information identified by the price tag and the related information identified by the commodity, so that the matching efficiency between the price tag and the commodity is improved. And the obtained matching score between each price tag and each commodity has high precision, and the precision, the robustness and the reliability of the matching process are improved. Therefore, under the background of matching the commodity and the price tag, the multi-information fusion can help comprehensively utilize various information sources such as images, positions, characters and the like, so that the robustness and the accuracy of matching are improved.

And then, carrying out dynamic programming processing on the matching table to obtain a matching result of the target commodity of each price tag. Thus, the matching score between each commodity and each price tag is obtained based on multi-information fusion. And the target commodity of each price tag is obtained based on the matching score and the dynamic programming technology between each commodity and each price tag, so that the matching calculation amount is greatly reduced, the matching efficiency is improved, and the complexity of the space and time required by matching is saved. And the matching precision and accuracy between the commodity and the price tag are further improved, so that the matching robustness and reliability are improved.

Drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also throughout the drawings, like reference numerals are used to designate like parts. In the drawings:

FIG. 1 is a schematic flow chart of the steps of a method for matching a commodity with a price tag according to an embodiment of the present invention;

FIG. 2 illustrates a schematic view of an item on a shelf image in an embodiment of the present invention;

FIG. 3 illustrates an exemplary diagram of 5 price tags and 9 products identified on a shelf image in an embodiment of the invention

FIG. 4 illustrates an exemplary schematic diagram of 5 items and 3 price tags identified by a shelf image in an embodiment of the present invention;

FIG. 5 is a schematic diagram showing a matching relationship obtained by a first dynamic programming processing manner in an embodiment of the present invention;

fig. 6 is a schematic diagram showing a matching relationship obtained by the second dynamic programming processing manner in the embodiment of the present invention.

Detailed Description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

Example 1

A first embodiment of the present invention provides a method for matching a commodity with a price tag, as shown in FIG. 1, including:

S101, acquiring a goods shelf image of a goods shelf;

S102, carrying out target detection processing on a shelf image to obtain price tag images and price tag position information of each price tag in a plurality of price tags, and carrying out semantic segmentation processing on the shelf image to obtain commodity images and commodity position information of each commodity in a plurality of commodities, wherein each commodity is an independent commodity or a commodity class;

s103, performing text recognition processing on the price tag image of each price tag and the commodity image of each commodity to obtain text information of each price tag and text information of each commodity;

S104, obtaining a matching score between each price tag and each commodity according to the price tag position information and the text information of each price tag and the commodity position information and the text information of each commodity, and obtaining a matching table according to the matching score between each price tag and each commodity;

s105, carrying out dynamic programming processing on the matching table to obtain a matching result of the target commodity of each price tag.

In this embodiment, after a shelf image of a commodity shelf is obtained, target detection processing is performed on the shelf image to obtain a price tag image and price tag position information of each price tag in the plurality of price tags, and semantic segmentation processing is performed on the shelf image to obtain a commodity image and commodity position information of each commodity in the plurality of commodities, where each commodity is an independent commodity or a commodity class. And then carrying out text recognition processing on the price tag image of each price tag and the commodity image of each commodity to obtain text information of each price tag and text information of each commodity. And obtaining the matching score between each price tag and each commodity according to the price tag position information and the text information of each price tag and the commodity position information and the text information of each commodity. And obtaining a matching table according to the matching score between each price tag and each commodity. In this way, the direct matching degree of the price tag and the commodity in the three dimensions is obtained based on the respective position information of the price tag and the commodity, the image characteristic information of the commodity, and the respective commodity names of the price tag and the commodity. And the matching score between each price tag and each commodity is obtained by fully utilizing the related information identified by the price tag and the related information identified by the commodity, so that the matching efficiency between the price tag and the commodity is improved. And the obtained matching score between each price tag and each commodity has high precision, and the precision, the robustness and the reliability of the matching process are improved. Therefore, under the background of matching the commodity and the price tag, the multi-information fusion can help comprehensively utilize various information sources such as images, positions, characters and the like, so that the robustness and the accuracy of matching are improved.

The following describes in detail the specific implementation steps of the matching method for the commodity and the price tag provided in this embodiment with reference to fig. 1:

First, step S101 is executed to acquire a shelf image of a commodity shelf. Specifically, a shelf image of the commodity shelf is captured by a camera or other imaging device. The goods shelf image is provided with a plurality of goods baskets, a plurality of goods are placed on each row of goods baskets, a plurality of price tags are placed in price tag tracks of each row of goods baskets, and goods and price tags are placed in one-to-one correspondence. And matching the commodity on the shelf image with the price tag through the subsequent steps so as to monitor the purchase condition and the placement condition of the commodity on the shelf of the shelf image in real time, thereby realizing the phenomena of timely commodity replenishment or distinguishing whether the target price tag corresponding to the commodity is placed incorrectly or distinguishing whether the target commodity corresponding to the price tag is placed incorrectly and the like.

Then, step S102 is executed to perform target detection processing on the shelf image to obtain a price tag image and price tag position information of each price tag in the plurality of price tags, and perform semantic segmentation processing on the shelf image to obtain a commodity image and commodity position information of each commodity in the plurality of commodities, where each commodity is an independent commodity or a commodity class.

Specifically, the specific process of obtaining the price tag image and the price tag position information of each price tag is to obtain a detection frame of each price tag by performing target detection processing on the shelf image. The object detection process is a process performed by an object detection model technology, for example, object detection process is performed on the shelf image by an object detection model, and the object detection model can be set according to actual requirements. And then carrying out perspective correction processing on the detection frames of each price tag, namely processing the detection frames of each price tag through a perspective correction algorithm to obtain corrected detection frames of each price tag. And obtaining price tag images and price tag position information of each price tag based on the corrected detection frames of each price tag. The price tag image of each price tag is an image framed by the corrected detection frame of each price tag. The price tag position information of each price tag carries coordinate information for the corrected detection frame of each price tag, for example, the corrected detection frame of each price tag carries coordinate information including the coordinates of the upper left corner and the lower right corner of the corrected detection frame of each price tag and/or the center point coordinates of the corrected detection frame of each price tag. Therefore, the price tag image and the price tag information of each price tag can be obtained rapidly, efficiently and accurately by performing target detection on the shelf image through the target detection technology.

The specific process of obtaining the commodity image and commodity position information of each commodity of the plurality of commodities is divided into two cases. The first case is that each commodity in this embodiment is an independent commodity, for example, a commodity is a box of oral liquid on a commodity shelf image of a certain medicine or a bag of potato chips on a commodity shelf image of a certain supermarket. The second case is that each commodity of the present embodiment is a commodity class, and one commodity class includes a plurality of independent commodities having the same commodity standard. For example, one commodity is an A-brand oral liquid on an image on a commodity shelf of a certain medicine, a plurality of A-brand oral liquids are placed on the commodity shelf of the medicine, and the A-brand oral liquids are regarded as one commodity, namely, the whole.

Taking fig. 2 as an example, fig. 2 is a shelf image, on which three rows of storage baskets are provided, and a plurality of commodities are placed on each storage basket. The first basket, the uppermost basket in fig. 2, is arranged to hold a plurality of triangular articles and a plurality of square articles in a top-down order, the second basket is arranged to hold a plurality of circular articles and a plurality of diamond articles, and the third basket is arranged to hold a plurality of trapezoidal articles. Each article of the present embodiment may then be an independent article, i.e. a triangular article or a square article or a circular article or a diamond article or a trapezoidal article. Each commodity of the present embodiment may be one commodity, i.e., one commodity formed by a plurality of triangular commodities or one commodity formed by a plurality of square commodities or one commodity formed by a plurality of circular commodities or one commodity formed by a plurality of diamond commodities or one commodity formed by a plurality of trapezoidal commodities.

In the case where each commodity is an independent commodity, the split area image of each commodity and the position coordinates of the split area image of each commodity in the plurality of commodities are obtained by performing semantic splitting processing on the shelf image. The position coordinates of the divided area image of each commodity include the upper left corner coordinates, the lower right corner coordinates, and the center point coordinates of the divided area image of each commodity. And the segmented region image of each commodity is taken as a commodity image of each commodity, and the position coordinates of the segmented region image of each commodity are taken as commodity position information of each commodity. Therefore, the commodity image of each commodity on the shelf image is accurately and efficiently segmented through the semantic segmentation technology, the obtained commodity image and commodity position information of each commodity are high in accuracy, subsequent matching processing is facilitated, and matching efficiency of the commodity and the price tag is improved. It should be further noted that, the semantic segmentation process is a process performed by a semantic segmentation technology, for example, the semantic segmentation process is performed on the shelf image by using a semantic segmentation model, and the semantic segmentation model may be set according to actual requirements.

In the case where each commodity is one commodity class in this embodiment, the position coordinates of the divided area image of each independent commodity and the divided area image of each independent commodity in the plurality of independent commodities are obtained by performing semantic division processing on the shelf image, and the divided area image of each independent commodity is taken as the commodity image of each independent commodity, and the position coordinates of the divided area image of each independent commodity are taken as the commodity position information of each independent commodity. And carrying out continuous package calculation processing according to the segmented region images of each independent commodity to obtain a commodity image of each commodity in a plurality of commodity classes, and obtaining commodity position information of each commodity class according to the commodity image of each commodity class. The commodity position information of each commodity class is the position coordinates of the commodity image of each commodity class, and the commodity position information of each commodity class includes the upper left corner coordinates, the lower right corner coordinates, and the center point coordinates of the commodity image of each commodity class.

The continuous package calculation processing comprises the steps of firstly identifying the commodity image of each independent commodity through an image identification model to obtain the image characteristics of each independent commodity. And performing perspective transformation processing on the shelf image to ensure that each row of commodities on the shelf image are parallel, and dividing the commodity images of two adjacent independent commodities with the same image characteristics into commodity images of the same commodity class along the X-axis direction and the Y-axis direction of the shelf image, namely dividing the two adjacent independent commodities with the same image characteristics into the same commodity class to obtain commodity images of each commodity class of a plurality of commodity classes. Two independent goods that have the same image characteristics and are adjacent represent that both independent goods have consistent image characteristics and that no other independent goods barriers exist between the two independent goods.

Under the condition that each commodity is a commodity class, not only the commodity image of each independent commodity on the shelf image is accurately and efficiently segmented through a semantic segmentation technology, but also the commodity image of each independent commodity is subjected to continuous package calculation, so that the commodity image and commodity position information of each commodity class are obtained. Therefore, the calculation amount for matching each commodity and each price tag in the follow-up process is reduced, and the matching efficiency and the matching precision of the commodity and the price tag are improved.

Next, step S103 is executed to perform text recognition processing on the price tag image of each price tag and the commodity image of each commodity, and obtain text information of each price tag and text information of each commodity.

Specifically, text recognition processing is performed on the price tag image of each price tag through an OCR (Optical Character Recognition ) model, so as to obtain text information of each price tag. The text information of each price tag includes, but is not limited to, commodity name, commodity rule, price information, and date of listing. And carrying out text recognition processing on the commodity images of each commodity through the OCR model or the image recognition model to obtain text information of each commodity. The text information for each commodity includes, but is not limited to, commodity name and commodity rule. It should be noted that whether the text information of each commodity obtained by the OCR model or the text information of each commodity obtained by the image recognition model is adopted in the subsequent step S104 depends on the confidence level of the OCR model and the confidence level of the image recognition model. If the confidence of the recognition result of the OCR model is high, text information of each commodity obtained by the OCR model is used. If the confidence of the recognition result of the image recognition model is high, the text information of each commodity obtained by the image recognition model is sampled.

In the embodiment, the text information of each price tag and the text information of each commodity can be accurately and efficiently identified through the OCR model, so that the data processing capacity is improved, the accuracy of each text information is enhanced, the subsequent matching processing is facilitated, and the matching efficiency is improved for the matching method of the embodiment. The text information of each commodity is identified through the image identification model, and the accuracy of the text information of each commodity can be improved.

Then, step S104 is executed to obtain a matching score between each price tag and each commodity according to the price tag position information and text information of each price tag and the commodity position information and text information of each commodity, and obtain a matching table according to the matching score between each price tag and each commodity.

Specifically, for each price tag and each commodity, obtaining the intersection ratio between the price tag and the commodity according to the price tag position information of the price tag and the commodity position information of the commodity; obtaining the text distance between the price tag and the commodity according to the commodity name in the text information of the price tag and the commodity name in the text information of the commodity; obtaining the center point distance between the price tag and the corresponding commodity according to the price tag position information of the price tag and the commodity position information of the commodity corresponding to the price tag, and setting the center point distance between the price tag and the commodity except the corresponding commodity as a center point distance appointed value; and obtaining the matching score between the price tag and the commodity according to the intersection ratio, the text distance and the center point distance between the price tag and the commodity.

And executing the matching score acquisition process for each price tag and each commodity to obtain the matching score between each price tag and each commodity.

By way of example, suppose that 5 price tags and 9 products are identified from the shelf image shown in FIG. 3. The 5 price tags are respectively marked as B1, B2, B3, B4 and B5, and the 9 goods are respectively marked as C1, C2, C3. First, a matching score between B1 and each of the 9 products is calculated. And obtaining the intersection ratio IOU _B1C1 between B1 and C1 according to the price tag position information of B1 and the commodity position information of C1. And obtaining the text distance TD _B1C1 between B1 and C1 according to the commodity name in the text information of B1 and the commodity name in the text information of C1.

The price tags are usually placed in price tag tracks of corresponding object placing baskets, the commodities are placed on the corresponding object placing baskets, and price tags of the commodities are placed on the price tag tracks of the corresponding object placing baskets. Therefore, in the same basket, the commodity and the price tag are positioned close to the corresponding placement. In this embodiment, the commodity corresponding to the price tag represents the commodity in the closest row to the price tag. The shelf image shown in fig. 3 is that in the same basket, the goods are placed on the basket, and the price tag is placed on the price tag track below the goods position. Since C1 is located in the row of products above B1, C1 is the product corresponding to B1. And obtaining the center point distance O _B1C1 between the B1 and the C1 according to the price tag position information of the B1 and the commodity position information of the C1.

The matching score MCTCH _B1C1 between B1 and C1, MCTCH _B1C1=w1×IOU_B1C1+w2×TD_B1C1+w3×O_B1C1, is obtained from IOU _B1C1、TD_B1C1 and O _B1C1. The distance-dependent weights w1 and w3 may be set according to human experience. The w2 weight may reference the recognition confidence.

Similarly, a match score MCTCH _B1C2 between B1 and C2, a match score MCTCH _B1C3 between B1 and C3, a match score MCTCH _B1C4 between B1 and C4, a match score MCTCH _B1C5 between B1 and C5, a match score MCTCH _B1C6 between B1 and C6, a match score MCTCH _B1C7 between B1 and C7, a match score MCTCH _B1C8 between B1 and C8, and a match score MCTCH _B1C9 between B1 and C9 can be obtained.

In the process of calculating the center point distance between B1 and C5 or C6 or C7 or C8 or C9, taking C5 as an example, since C5 is not located in the row of commodities above B1, C5 is not the commodity corresponding to B1. Therefore, the center point distance between C5 and B1 is set to the center point distance specified value zero, i.e., O _B1C5 =0. The specified value of the center point distance can also be set according to actual requirements. The matching score MCTCH _B1C5 between B1 and C5, MCTCH _B1C5=w1×IOU_B1C5+w2×TD_B1C5+w3×O_B1C5=w1×IOU_B1C5+w2×TD_B1C5 +w3X10, is derived from the intersection ratio between B1 and C5, the text distance TD _B1C1 between B1 and C5, and the center point distance O _B1C5 between B1 and C5.

Similarly, a match score between B2 and each of the 9 articles, a match score between B3 and each of the 9 articles, a match score between B4 and each of the 9 articles, and a match score between B5 and each of the 9 articles are calculated. Thus, a matching score between each price tag and each commodity is obtained.

In addition, in the process of obtaining the matching score between each price tag and each commodity, the cross-union ratio between the price tag and the commodity is obtained according to the price tag position information of one price tag and the commodity position information of one commodity, and the specific mode is as follows: the area of the price tag is obtained according to the price tag position information of the price tag, and the area of the commodity is obtained according to the commodity position information of the commodity. And obtaining the cross-linking ratio between the price tag and the commodity according to the area of the price tag and the area of the commodity. According to the commodity name in the text information of the price tag and the commodity name in the text information of the commodity, the text distance between the price tag and the commodity is obtained by the following specific modes: and obtaining a text public subsequence of the commodity name according to the commodity name in the text information of the price tag and the commodity name in the text information of the commodity. Dividing the text public subsequence of the commodity name by the length of the commodity name in the text information of the price tag to obtain the text distance between the price tag and the commodity. According to the price tag position information of the price tag and the commodity position information of the commodity corresponding to the price tag, the center point distance between the price tag and the corresponding commodity is obtained by the following specific modes: and obtaining the center point coordinates of the price tag according to the price tag position information of the price tag. And obtaining the center point coordinates of the corresponding commodity according to the commodity position information of the commodity corresponding to the price tag. And obtaining the center point distance between the price tag and the corresponding commodity according to the center point coordinates of the price tag and the center point coordinates of the corresponding commodity.

After the matching score between each price tag and each commodity is obtained, a matching table is obtained according to the matching score between each price tag and each commodity. Specifically, each price tag, each commodity, and the matching score between each price tag and each commodity are formed into a table. The matches between each price tag and each commodity obtained according to fig. 3 are divided into the match tables shown in table 1.

In table 1, the first row represents the number of each commodity, the first column represents the number of each price tag, and the lattice corresponding to each price tag and each commodity places a matching score between each price tag and each commodity. The format of the table arrangement and the variations of the rows and columns are not limited thereto, and table 1 is only an example.

In this embodiment, based on the respective position information of the price tag and the commodity name identified by the image feature information of the commodity, the respective commodity names of the price tag and the commodity, that is, the intersection ratio between the price tag and the commodity, the text distance between the price tag and the commodity, and the center point distance between the price tag and the commodity, which are identified by the OCR model, the direct matching degree of the price tag and the commodity in the three dimensions is obtained. According to the method and the device, the matching score between each price tag and each commodity is obtained by fully utilizing the related information identified by the price tag and the related information identified by the commodity, so that the matching efficiency between the price tag and the commodity is further improved. And the obtained matching score between each price tag and each commodity has high precision, so that the precision, the robustness and the reliability of the matching process are further improved. Therefore, under the background of matching the commodity and the price tag, the multi-information fusion can help comprehensively utilize various information sources such as images, positions, characters and the like, so that the robustness and the accuracy of matching are improved.

Finally, step S105 is executed to perform dynamic programming processing on the matching table, so as to obtain a matching result of the target commodity of each price tag.

In particular, the matching table may be dynamically programmed from two dimensions. The first dynamic programming processing mode is to dynamically program the matching table according to the dimension of the price tag. The second dynamic programming processing mode is to dynamically program the matching table according to the dimension of the commodity.

The specific process of the first dynamic programming processing mode is as follows: a1, sorting the price tags according to a preset sequence. The preset sequence can be set according to actual requirements. And b1, under the current state of the matching table, carrying out dynamic matching processing on one current price tag in the plurality of price tags to obtain a target commodity of the current price tag and the updated matching table of the current price tag, and taking the updated matching table of the current price tag as the next state of the current state of the matching table. The target commodity is a commodity matched with the price tag, and the target commodity of the current price tag is a commodity matched with the current price tag. And c1, dynamically matching the next price tag of the current price tag under the next state of the current state of the matching table to obtain a target commodity of the next price tag and the updated matching table of the next price tag, and taking the updated matching table of the next price tag as the next state of the matching table. d1, in the next state of the matching table, carrying out dynamic matching processing on the next price tag until in the last state of the matching table, carrying out dynamic matching processing on the last price tag in the plurality of price tags, and obtaining a matching result of the target commodity of each price tag.

The dynamic matching process comprises the following steps: in the current state of the matching table, for each current price tag, finding a target commodity of the current price tag according to the matching score between the current price tag and each commodity, wherein the target commodity is the commodity with the highest matching score with the current price tag. And setting the matching score between the target commodity and each price tag except the current price tag as a preset value, and updating the matching table to obtain the updated matching table of the current price tag, wherein the preset value indicates that the commodity and the price tag do not have a matching relationship. Specific values of the preset values can be set according to actual requirements.

The specific procedure of the first dynamic programming approach is set forth in the matching table shown in table 1. The price tags are ordered according to a preset sequence. For example, in the order of the number of tags, B1, B2, B3, B4 and B5. And under the current state of the matching table, B1 is the current price tag, and the dynamic matching processing is carried out on B1. The dynamic matching process is to find the target commodity of B1 according to the matching score between B1 and each commodity in the 9 commodities, namely, the commodity with the highest matching score with B1 is determined as the target commodity of B1. If the matching score between B1 and C1 is highest, then C1 is the target commodity of B1. Setting the matching score of C1 and each price tag except B1 to be zero or null, which indicates that C1 is no longer matched with any price tag except B1, namely that C1 does not have a matching relationship with any price tag except B1, and B1 and C1 have locked matching relationship. After the matching score of C1 and each price tag except B1 is set to be a preset value, updating the matching table to obtain a B1 updated matching table. The updated matching table of B1 is shown in table 2.

The updated matching table of B1 is the next state of the current state of the matching table and is also the next state of the matching table. The next state of the matching table is taken as the current state of the matching table. In the next state of the matching table, the next price tag B2 of B1 is the current price tag at this time. And in the matching table updated by the B1, the current price tag is B2, and the dynamic matching processing is carried out on the B2. The dynamic matching process is that in the updated matching table of B1, that is, in table 2, according to the matching score between B2 and each of the remaining 8 products, the target product of B2 is found, that is, the product with the highest matching score with B2 is determined as the target product of B2. The matching score between B2 and C3 is highest, then C3 is the target commodity for B2. Setting the matching score of C3 and each price tag except B2 to be a preset value. And after the matching score of the C3 and each price tag except the B2 is set to be a preset value, updating the matching table to obtain a B2 updated matching table. The updated matching table of B2 is shown in table 3.

The updated matching table of B2 is the next state of the current state of the matching table, i.e. table 3, and is also the next state of the matching table. The next state of the matching table is taken as the current state of the matching table. In the next state of the matching table, the next price tag of B2 is the current price tag at that time. And pushing the price label until the last price label in the plurality of price labels is subjected to dynamic matching processing in the last state of the matching table, and obtaining a matching result of the target commodity of each price label.

The dynamic programming process works on the principle that when one commodity is matched by price tags, the commodity cannot be matched by other price tags. The matching relationship of each price tag is determined by the matching score between the current price tag and each commodity in the current state of the matching table. The price tag and the commodity which have the matching relation are not matched any more, and the matching score of the commodity which has the matching relation and other price tags is set to be a preset value of 0 or null.

The specific process of the second dynamic programming processing mode is as follows: a2, sorting the plurality of commodities according to a preset sequence. And b2, under the current state of the matching table, carrying out dynamic matching processing on one current commodity in the plurality of commodities to obtain a target price tag of the current commodity and the matching table updated by the current commodity, and taking the matching table updated by the current commodity as the next state of the current state of the matching table. And c2, dynamically matching the next commodity of the current commodity under the next state of the current state of the matching table to obtain a target price tag of the next commodity and a matching table updated by the next commodity, and taking the matching table updated by the next commodity as the next state of the matching table. d2, carrying out dynamic matching processing on the next commodity in the next state of the matching table until each price tag in the matching table is subjected to dynamic matching processing, and obtaining a matching result of the target price tag of each commodity. The principle of the dynamic matching process in the second dynamic programming process mode is identical to that in the first dynamic programming process mode, and will not be described herein.

The specific procedure of the second dynamic programming approach is set forth in the matching table shown in table 4. Table 4 shows the following:

In Table 4, E1, E2 and E3 are price tags, and F1, F2 and F3 are commercial products. The method comprises the steps of firstly sorting a plurality of commodities according to a preset sequence. For example, ordered by the serial number of the merchandise, ordered as F1, F2, and F3. Under the current state of the matching table, F1 is the current commodity, and dynamic matching processing is carried out on F1. The dynamic matching process is to find the target price tag of F1 according to the matching score between F1 and each price tag in the 3 price tags, namely, the price tag with the highest matching score with F1 is determined as the target price tag of F1. The matching score between F1 and E1 is highest, then E1 is the target price tag for F1. Setting the matching score of E1 and each commodity except F1 to be zero or null, which indicates that E1 is no longer matched with any commodity except F1, i.e. E1 does not have a matching relationship with any commodity except F1, and E1 and F1 have locked matching relationship. After the matching score of E1 and each commodity except F1 is set to be a preset value, updating the matching table to obtain an F1 updated matching table. The updated matching table for F1 is shown in table 5.

The updated matching table F1 is the next state of the current state of the matching table, i.e. table 5, and is also the next state of the matching table. The next state of the matching table is taken as the current state of the matching table. In the next state of the matching table, the next commodity of F1 is the current commodity at this time. And the same goes on until each price tag in the matching table is subjected to dynamic matching processing, and then a matching result of the target price tag of each commodity is obtained.

In this embodiment, a matching score between each commodity and each price tag is obtained based on multi-information fusion. And the target commodity of each price tag is obtained based on the matching score and the dynamic programming technology between each commodity and each price tag, so that the matching calculation amount is greatly reduced, the matching efficiency is improved, and the complexity of the space and time required by matching is saved. And the matching precision and accuracy between the commodity and the price tag are further improved, so that the matching robustness and reliability are improved.

In addition, the first dynamic programming processing mode is more reasonable than the second dynamic programming processing mode, namely, the matching process based on price tag dimension is higher in matching efficiency and matching precision than the matching process based on commodity dimension, and the matching effect is better. Therefore, the first dynamic programming approach is generally preferred.

If the specific procedure of the second dynamic programming approach is set forth in the matching table shown in Table 1. In the current state of the matching table shown in table 1, dynamic matching processing is performed on the current commodity C1. And C1, resetting the matching score of the B1 and each other commodity to a preset value to obtain a C1 updated matching table. The updated matching table of C1 is shown in table 6.

The updated matching table of C1 is the next state of the current state of the matching table, i.e. table 6, and is also the next state of the matching table. The next state of the matching table is taken as the current state of the matching table. In table 6, the dynamic matching process is performed on the current commodity C2. And B3, resetting the matching score of the B3 and each other commodity to a preset value to obtain a C2 updated matching table. The updated matching table of C2 is shown in table 7. And the same goes on until each price tag in the matching table is subjected to dynamic matching processing, and then a matching result of the target price tag of each commodity is obtained.

Based on the first dynamic programming processing mode, the target commodity of the price tag B2 is C3. And based on the second dynamic programming processing mode, the target price tag of the commodity C2 is C3. In connection with the shelf image shown in fig. 3, the commodity corresponding to B2 should be C3. It is apparent that the first dynamic programming approach is more reasonable than the second dynamic programming approach. Therefore, the matching process based on the price tag dimension has higher precision and accuracy of the matching result and better matching effect.

The rationality of the first dynamic programming approach and the second dynamic programming approach is further illustrated by the matching of price tags to merchandise shown in fig. 4.

In fig. 4 there are 5 products and 3 tags, with 3 tags pt1, pt2 and pt3, respectively, and 5 products cg1, cg2, cg3, cg4 and cg5, respectively. The matching table of these 5 goods and 3 price tags is shown in table 8.

Based on the first dynamic programming processing mode, namely matching is carried out according to the dimension of the price tag, the target commodity of pt1 is cg4, namely the matching score of pt1 and cg4 is the largest. The target commodity of pt2 is cg3, and the target commodity of pt3 is cg5. The matching result based on the first dynamic programming method is shown in fig. 5. Where cg2 and cg4 are commercial products without a price tag.

Assuming that each of the commodities cg1 to cg5 is an independent commodity, and assuming that cg1, cg2 and cg3 are the same commodity, that is, the same commodity class, after the matching result shown in fig. 5 is obtained, cg1, cg2 and cg3 may be classified into one class by a continuous packet calculation process. Therefore, when each commodity is an independent commodity, after the matching result of the target commodity of each price tag is obtained, the final matching result can be obtained through continuous package calculation processing.

Based on the matching result obtained in the first dynamic programming processing method, whether cg1, cg2, cg3 are the same commodity or not, the matching relationship can be reasonably processed according to table 8.

And matching is performed based on a second dynamic programming processing mode according to the dimension of the commodity. In Table 8, the matching score of cg1 and pt2 is the largest. cg2 and pt2 have the greatest match score. If cg1 and cg2 are a commodity class, pt2 can be matched to both cg1 and cg2 as appropriate. The final matching result is similar to that of fig. 3. If cg1 and cg2, cg3 are not the same commodity, i.e., the three are different commodities. One price tag can only be matched with one commodity. Based on the second dynamic programming approach, cg2 and cg3 cannot match pt2 after cg1 matches pt 2. The matching result obtained based on the second dynamic programming processing mode is that the target price tag of cg1 is pt2, the target price tag of cg2 is pt1, and the target price tag of cg3 is pt3. The matching result based on the second dynamic programming process is shown in fig. 6.

It is apparent that the matching results of fig. 6 are much more reasonable than those of fig. 5 for the matching cases where all of the different products are different. Therefore, the matching process based on the price tag dimension has higher matching result precision and better matching effect. If the second dynamic programming method is to be sampled, the possible application environment and conditions are more severe, for example, the method is only applicable to each commodity which is a commodity class, and the number of the identified commodities is consistent with the number of price tags. Therefore, the preferred sampling is based on the first dynamic programming approach, namely, the preferred sampling price tag dimension performs the matching process of the goods and price tags.

It should be noted that, when each commodity is an independent commodity, after the matching result of the target commodity of each price tag is obtained, the final matching result can be obtained through continuous package calculation processing. In the case where each commodity is a commodity class, the package calculation processing has been performed in step S102 in the process of obtaining the matching result of the target commodity of each price tag.

In this embodiment, the method for matching the commodity and the price tag based on the dynamic programming of multi-information fusion has the following advantages: first, the dynamic programming technology can effectively reduce the time complexity of the matching algorithm and improve the accuracy and precision of matching through reasonable problem decomposition and optimal substructure properties. And secondly, the multi-information fusion can comprehensively utilize different information sources to provide more comprehensive and accurate commodity and price tag characterization, so that the matching robustness and reliability are improved. Finally, the method can adapt to matching requirements of commodities and price tags in different scenes and environments, and has certain universality and flexibility.

Therefore, the matching method solves the problems of the traditional matching method and improves the accuracy and the instantaneity of matching. By applying the dynamic programming and multi-information fusion technology to the combination of feature extraction, model training and matching ratio equivalent steps, the method can automatically realize the accurate matching of commodities and price tags, thereby improving the efficiency and accuracy of retail industry and logistics field.

Example two

Based on the same inventive concept, the third embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of any one of the matching methods of the commodity and the price tag when the program is executed.

Since the apparatus described in this embodiment is an apparatus for implementing the matching method of the commodity and the price tag in the first embodiment of the present application, based on the matching method of the commodity and the price tag described in the first embodiment of the present application, those skilled in the art can understand the specific implementation manner of the apparatus of this embodiment and various modifications thereof, so how the apparatus implements the method in the first embodiment of the present application will not be described in detail herein. As long as the person skilled in the art implements the apparatus for matching the commodity and the price tag in the first embodiment of the present application, the apparatus falls within the scope of protection of the present application.

Example III

Based on the same inventive concept, the fourth embodiment of the present invention further provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of any one of the matching methods of goods and price tags described in the previous embodiment.

It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

1. A method of matching a commodity with a price tag, comprising:

acquiring a goods shelf image of a goods shelf;

Performing dynamic programming processing on the matching table to obtain a matching result of the target commodity of each price tag, wherein the dynamic programming processing comprises the following steps:

sorting the plurality of price tags according to a preset sequence;

2. The method of claim 1, wherein the process of dynamic matching comprises:

in the current state of the matching table, finding a target commodity of the current price tag according to the matching score between the current price tag and each commodity, wherein the target commodity is the commodity with the highest matching score with the current price tag;

3. The method of claim 1, wherein the specific process of dynamically programming the matching table to obtain the matching result of the target commodity of each price tag is replaced by the following steps:

Sorting the plurality of commodities according to a preset sequence;

4. The method of claim 2, wherein the obtaining the matching score between each price tag and each commodity according to the price tag position information and the text information of each price tag and the commodity position information and the text information of each commodity comprises:

5. The method of claim 2, wherein the text recognition processing is performed on the price tag image of each price tag and the commodity image of each commodity to obtain the text information of each price tag and the text information of each commodity, and the method comprises the following steps:

6. The method of claim 1, wherein the performing the object detection process on the shelf image to obtain a price tag image and price tag location information for each of a plurality of price tags comprises:

7. The method of claim 1, wherein the performing semantic segmentation on the shelf image to obtain a commodity image and commodity location information for each of a plurality of commodities comprises:

8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the matching method of goods and price tags as claimed in any of claims 1-7 when the program is executed.

9. A computer readable storage medium having stored thereon a computer program, characterized in that the program when executed by a processor performs the steps of the matching method of goods and price tags as claimed in any of claims 1-7.