CN109993595B

CN109993595B - Method, system and equipment for personalized commodity and service recommendation

Info

Publication number: CN109993595B
Application number: CN201711483236.5A
Authority: CN
Inventors: 李炜明; 汪昊; 喻冬东; 刘洋; 王强; 王再冉; 考月英; 安民修; 洪性勋
Original assignee: Samsung Electronics Co Ltd
Current assignee: Samsung Electronics Co Ltd
Priority date: 2017-12-29
Filing date: 2017-12-29
Publication date: 2024-06-21
Anticipated expiration: 2037-12-29
Also published as: CN109993595A; KR102664008B1; KR20190082048A

Abstract

The application relates to a method, a system and equipment for personalized commodity and service recommendation, wherein the method comprises the following steps: acquiring or inputting at least one image about a customer; identifying shopping tendencies of the customer according to the acquired or input images; recommending goods or services to the customer according to the identified shopping tendency; and transmitting or displaying the recommended goods or services information to the customer. With the proposed method and system that can identify consumer shopping tendencies, high quality personalized recommendations can be made to consumers for related products and services in an augmented reality shopping application based on the identified shopping tendencies.

Description

Method, system and equipment for personalized commodity and service recommendation

Technical Field

The application relates to the field of pattern recognition and computer vision, in particular to a method, a system and equipment for personalized commodity and service recommendation.

Background

Existing merchandise recommendation techniques often make recommendations using data related to the purchase history of the customer (e.g., shopping records, browsing records of merchandise, etc.). Hereinafter, some existing patent documents on commodity recommendation technologies and related technologies will be listed.

In some patent documents, some personalized recommendation methods are proposed. In these methods, based on the online purchase record or online browsing record of the customer, the customer is recommended with goods similar to the goods previously purchased or browsed.

However, this personalized recommendation method is only applicable to online purchasing behavior of customers. These recommendation methods are not applicable to the purchasing behavior of customers at physical stores.

In addition, when a customer shops in a physical store, a worker can make a recommendation to the customer through a purchase record of the customer. However, when a customer first accesses a certain physical store, the customer cannot be properly recommended because of the lack of past purchase or browsing records.

Augmented reality technology may be used to overlay display related information for a user in a real scene, thereby enhancing the user's experience. In the existing application field, augmented reality is mainly applied to three aspects including terminal display, content generation and user interaction. The application adopts the user interaction technology in the augmented reality technology. Specifically, in augmented reality shopping, in order to select information of interest to a customer from data concerning a large number of commodities and display the information, it is necessary to recognize shopping tendencies of the customer.

Disclosure of Invention

The object of the present application is to solve at least one of the above technical drawbacks, by means of which a system and a method are proposed which can identify a shopping tendency of a customer, from which a high quality personalized recommendation of related products and services can be made to the customer in an augmented reality shopping application.

According to an aspect of the present application, there is provided a method of personalized recommendation of goods and services, comprising: acquiring or inputting at least one image about a customer; identifying shopping tendencies of the customer according to the acquired or input images; recommending goods or services to the customer according to the identified shopping tendency; and transmitting or displaying the recommended goods or services information to the customer.

According to another aspect of the present application, there is provided a system for personalized recommendation of goods and services, comprising: an image acquisition unit for acquiring or inputting at least one image about a customer; the identification unit is used for identifying shopping tendency of the customer according to the acquired or input image; a judging unit for recommending goods or services to the customer according to the identified shopping tendency; and a network unit for transmitting or displaying the recommended goods or service information to the customer.

According to another aspect of the present application, there is provided an apparatus for personalized recommendation of goods and services, comprising: a processor; and a memory in which are stored instructions executable by a processor in fact, which when executed by the processor, cause the processor to perform the method steps of: acquiring or inputting at least one image about a customer; identifying shopping tendencies of the customer according to the acquired or input images; recommending goods or services to the customer according to the identified shopping tendency; and transmitting or displaying the recommended goods or services information to the customer.

Drawings

The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description when taken in conjunction with the accompanying drawings in which:

FIG. 1 illustrates a schematic diagram of a method of making a recommendation of a commodity to a customer in accordance with the inventive concept of the present application;

Fig. 2 is a flowchart illustrating a method of acquiring and inputting an image according to the inventive concept;

FIG. 3 illustrates a schematic diagram of a method for identifying shopping tendencies of a customer in accordance with the inventive concepts of the present application;

FIG. 4 illustrates a schematic diagram of a method for modeling shopping tendencies through an online shopping database in accordance with the inventive concepts of the present application;

FIG. 5 illustrates a flowchart of a shopping tendency model extraction method based on deep learning according to the inventive concept;

FIG. 6 shows a schematic diagram of a deep neural network recommendation model for fashion class merchandise in accordance with the inventive concepts of the present application;

FIG. 7 is a schematic diagram of a method of establishing feature vectors for each category of merchandise for each customer group, respectively, in accordance with the inventive concept of the present application;

FIG. 8 illustrates a schematic diagram of a method of modeling shopping tendencies through a physical store shopping database in accordance with the inventive concepts of the present application;

FIG. 9 is a schematic diagram of a method of analyzing customer images based on a neural network according to the inventive concept;

FIG. 10 is a schematic diagram of a method of matching customer images and merchandise images based on a neural network according to the inventive concept of the present disclosure;

FIG. 11 illustrates a flow chart of a method of making a recommendation of goods or services based on an identified shopping tendency of a customer in accordance with the inventive concept;

FIG. 12 is a schematic diagram showing a method of making merchandise recommendations to an enhanced display device of a customer of a physical store in accordance with the inventive concept of the present application;

FIG. 13 illustrates a schematic diagram of a method of displaying merchandise recommendations to a display in the vicinity of a customer of a physical store in accordance with the inventive concept;

fig. 14 is a schematic view showing a method of displaying customer information and merchandise recommendations to an augmented reality display device of a sales assistant in a physical store according to the inventive concept of the present application; and

Fig. 15 is a schematic view showing a method of displaying customer information and merchandise recommendation on a face image displayed by a browser according to the inventive concept of the present application.

Detailed Description

Fig. 1 illustrates a schematic view of a method of recommending goods to a customer according to the inventive concept of the present application. It should be noted that this diagram is merely an example and is not intended to limit the scope of the claimed application.

As shown in fig. 1, the method for recommending goods to a customer includes the following steps:

in step S110, at least one image about the customer is acquired or inputted;

In step S120, the shopping tendency of the customer is identified by the acquired or inputted image;

In step S130, a recommendation of goods or services is made according to the identified shopping tendencies of the customer;

in step S140, the recommended goods or services information is transmitted or displayed to the customer.

Fig. 2 is a flowchart illustrating a method of acquiring and inputting an image according to the inventive concept. The drawings are merely examples and are not intended to limit the scope of the claimed application.

As shown, the method of capturing and inputting images is performed by a device comprising one or more cameras, wherein the device is in a physical store and the field of view of the one or more cameras may cover all areas of the physical store.

In the method, the device first detects a customer in a shopping area of a physical store and tracks the detected customer, wherein after detecting the customer, the customer is identified and a matching customer ID is searched for; if no match is found, a customer ID is established for the customer. If a customer leaves the field of view of a certain camera, the customer continues to be detected and tracked by cameras within the device that are adjacent to the field of view of the camera. If it is determined from the tracking information that the customer has left the shopping area of the brick-and-mortar store, the tracking of the customer is ended and the corresponding information is saved under the customer ID.

During the entire process of detection and tracking, the image of the customer and the customer ID are output to a method for identifying the shopping tendency of the customer, and corresponding recommendation is made to the customer. Depending on the needs of the shopping trend identification method, one or more images may need to be input from the detection and tracking process, and these images may be input at different times of tracking.

When the customer leaves the shopping area, the corresponding information retained may include the customer ID, the image data of the customer, and an actual purchase record of the customer is saved. The purchase records of the customer may include, but are not limited to, the type of goods purchased, brands, specifications, quantity, and prices of the goods, among others.

Fig. 3 illustrates a schematic diagram of a method for recognizing shopping tendencies of customers according to the inventive concept. The drawings are merely examples and are not intended to limit the scope of the claimed application.

In a method for identifying shopping tendencies of a customer, two sub-processes are included, one sub-process being performed offline for constructing a shopping tendencies model database; another sub-process is performed online, wherein after receiving an image of a customer, a best matching shopping trend model is automatically looked up from the shopping trend model database based on the customer's visual appearance characteristics and used as the customer's shopping trend model.

As shown in FIG. 3, the shopping tendency model database contains a plurality of shopping tendency models, and the models are obtained by the shopping behavior database through the statistical analysis of the algorithm in an off-line mode. Wherein each shopping trend model describes a representative class of average shopping trends of customers, and each shopping trend model is composed of a plurality of feature vectors, including but not limited to fashion class commodity feature vectors, customer image feature vectors, non-fashion class commodity feature vectors and the like. The customer image feature vector is analyzed to obtain personal information about the age, sex and the like of the customer, and the shopping behavior of the customer in a physical store is counted; in addition, by analyzing the customer image feature vector, the customer image may be divided, and based on the analysis of different clothing parts (such as coat, trousers, etc.), the features of the clothing may be extracted and the customer's feature preference for fashion type goods may be determined. The non-fashion category commodity feature vector may include a plurality of component vectors, each component corresponding to a different category of non-fashion commodity.

For a target customer, the customer's features may be extracted from the image of the customer. Based on these features, a best matching shopping tendency model can be found in the shopping tendency database as the shopping tendency model of the customer based on features such as fashion class commodity feature vectors, customer image feature vectors, and the like.

Fig. 4 illustrates a schematic diagram of a method for building a shopping tendency model through an online shopping database according to the inventive concept of the present application. The drawings are merely examples and are not intended to limit the scope of the claimed application.

One possible way to build a shopping trend model database is to build a shopping trend model from an online shopping database, as shown in fig. 4. An online shopping database may include online shopping records for a large number of customers, including but not limited to: customer ID, merchandise purchased by the customer, a browsing record of the customer, etc. The items purchased by the customer may include fashion class items, non-fashion class items, and the like.

Taking fashion category goods as an example, customers who purchase similar fashion category goods can be divided into groups of different categories by performing statistical learning with respect to preference feature vectors of customers for fashion category goods. For each customer group, the purchase data of different types of commodities can be counted, and feature vectors of the different types of commodities are respectively established. The shopping trend model that can be built for each customer group includes: fashion category commodity feature vectors, non-fashion category commodity feature vectors.

Specifically, the feature vector of each customer can be determined by first performing statistical learning on the online shopping records of the customers, and then utilizing matrix decomposition according to the purchase record matrix of the customers' fashion category commodities (the elements of the ith row and the jth column of the matrix represent the purchase record of the customer i for the commodity j); the customers are then clustered according to their feature vectors to classify the customers into different customer groups, wherein the similarity between the customers can be determined by the inner product between the feature vectors of the customers.

After different customer groups are established, the feature vector of each customer group for a commodity of a certain category is determined according to the purchase record matrix of the commodity of the certain category of the different customer groups. Taking fashion class merchandise as an example, feature vectors of the fashion class merchandise may be determined.

Fig. 5 illustrates a flowchart of a shopping tendency model extraction method based on deep learning according to the inventive concept. The drawings are merely examples and are not intended to limit the scope of the claimed application.

As shown in fig. 5, first, a "customer-commodity" association vector is extracted from an online shopping database, wherein the vector is a binary vector having a length equal to the number of commodity categories; for a certain customer, if the customer purchases a commodity belonging to a certain commodity category, the vector bit corresponding to the commodity category is 1, otherwise, the vector bit is 0. Merchandise images of fashion category merchandise may also be obtained from the online shopping database. For fashion category goods, a deep neural network recommendation model for the fashion category goods can be trained by utilizing a large number of customer-goods association vectors and fashion category goods images. The recommendation model may predict whether a customer will purchase a fashion class commodity. Specifically, the customer-commodity correlation vector of the customer and the image of the fashion class commodity are input, and the recommendation model outputs a decimal between 0 and 1, which is the probability that the customer purchased the fashion class commodity. The hidden layer of the neural network may represent shopping tendency attributes of one customer and output as shopping tendency feature vectors of the customer.

According to the shopping trend feature vectors of a large number of customers in the online shopping database, the customers can be clustered to obtain customer groups. Wherein for each customer group there is a similar tendency for fashion class merchandise shopping. Each customer group is considered a virtual representative customer whose shopping record may be obtained from an online shopping database. From the online shopping database, a "customer-commodity" correlation matrix can be constructed for each representative customer of each customer group, and the correlation matrix can be decomposed to obtain shopping tendency feature vectors of each representative customer on different types of commodities.

Fig. 6 shows a schematic diagram of a deep neural network recommendation model for fashion class goods according to the inventive concept of the present application. The drawings are merely examples and are not intended to limit the scope of the claimed application.

As shown in fig. 6, the input of the recommendation model includes two paths, wherein the first path of input is a "customer-commodity" association vector of a customer, and the vector is firstly mapped from original data to a customer feature vector through a dimension-reducing projection; the dimension-reducing projection can be realized by a matrix or a multi-layer neuron network; for example, if the customer-commodity correlation vector before the dimension reduction is N-dimensional, and the feature vector after the dimension reduction is M-dimensional, the dimension reduction can be achieved by the method in Matrix Factorization Techniques for Recommender Systems (y. Koren et al, august 2009,IEEE Computer), or by a multi-layer perceptron (Multiple Layer Perceptron, MLP) technique using a multi-layer neuronal network. And when the second path is input, an image of a commodity in a fashion class is firstly subjected to visual feature extraction through a deep convolution neural network, and commodity feature vectors are obtained. The customer characteristic vector obtained by the first path of input and the commodity characteristic vector obtained by the second path of input are connected in series, a characteristic vector is obtained, the obtained characteristic vector is input into a coding neuron network, the number of neurons in the coding neuron network is reduced layer by layer, and finally a numerical value between 0 and 1 is input, wherein the numerical value represents the probability that the customer purchases the fashion type commodity.

A large amount of training time for the model described above may be generated by an online shopping database and the model trained using a back propagation algorithm.

Fig. 7 is a schematic diagram showing a method of establishing feature vectors of each category of goods for each customer group, respectively, according to the inventive concept of the present application. The drawings are merely examples and are not intended to limit the scope of the claimed application.

As shown, items purchased by each customer belonging to the customer group may be looked up from an online purchase database. Considering the entire customer group as one representative customer, it is possible to obtain which goods the representative customer purchased. For these commodities, shopping tendency feature vectors are respectively established according to commodity categories. For example, for a commodity of a commodity category other than a fashion class commodity, a purchase matrix of different representative customers with different commodity categories may be established taking into account the "customer-commodity" association vector. The purchase matrix is decomposed by using a collaborative filtering method, so that the shopping tendency characteristic vectors of different types of commodities can be obtained. These feature vectors constitute a shopping tendency model for the representative customer (i.e., the customer group).

Fig. 8 illustrates a schematic diagram of a method of building a shopping tendency model through a physical store shopping database according to the inventive concept of the present application. The drawings are merely examples and are not intended to limit the scope of the claimed application.

The above describes how the shopping trend model is built by an online shopping database. In FIG. 8, it is shown how shopping trend models are built through a physical store shopping database. Here, the physical store shopping database includes a customer ID of a customer, an image of the customer, a shopping record of the customer, and the like.

Information that may be obtained by image analysis of a customer includes, but is not limited to: appearance information of the customer, shopping behavior information of the customer, and the like. Wherein, the appearance information of the customer may include: age, sex, hairstyle, apparel characteristics, jewelry characteristics, ornamental brand characteristics, etc. of the customer; shopping behavior characteristics of a customer may include: the trajectory of the customer within the physical store, the residence time of the customer in different areas within the store, including but not limited to shopping areas, etc.

The customers can be separated into different customer groups based on features extracted from the customer images in the shopping trend model constructed in fig. 3. For each customer group, purchase data of different commodity categories can be counted, and feature vectors of the commodities of different categories are respectively established. For each customer group, a shopping trend model may be constructed comprising: feature vectors of fashion type commodities, feature vectors of customer images, feature vectors of non-fashion type commodities, and the like.

For a target customer, after the image of the customer is acquired, the image features of the customer may be extracted. From these image features, the best matching shopping tendency model can be found in the shopping tendency model database as the shopping tendency model of the customer based on features such as feature vectors of fashion class merchandise, feature vectors of customer images, and the like. One implementation method for finding the shopping trend model that best matches the customer is to calculate the inner products of the image feature vector of the customer and the vision-related feature vectors in all shopping trend models, compare all the inner products obtained, and use the shopping trend model corresponding to the maximum inner product as the shopping trend model that matches the customer.

Fig. 9 is a schematic diagram illustrating a method for analyzing a customer image based on a neural network according to the inventive concept of the present disclosure. The drawings are merely examples and are not intended to limit the scope of the claimed application.

As shown in fig. 9, in the analysis method, each pixel of the customer image is classified using a deep learning-based method, including but not limited to: background, hat, hair, face, sunglasses, scarf, coat, dress, coat, glove, trousers, coveralls, short skirt, left arm, right arm, left leg, right leg, left foot, right foot, sock, bag, jewelry, watch, etc.

According to the classification result of the pixels, image windows corresponding to different fashion types of commodities can be extracted from the customer image, and noise caused by the color, texture and the like of a background object can be removed by removing pixels classified as the background.

Fig. 10 is a schematic diagram showing a method of matching customer images and merchandise images based on a neural network according to the inventive concept of the present application. The drawings are merely examples and are not intended to limit the scope of the claimed application.

After the images of the target customer are acquired, clothing images worn on the customer are extracted, and for these images, images having the most matching style are searched for among fashion-type commodity images managed by each shopping tendency model.

One problem to be considered here is that, in the case of the same laundry style, there is a clear difference in image characteristics between the laundry in the worn state and the laundry in the display state of the online shopping website. Here, the present application adopts a neural network-based method to evaluate the degree of style matching between a set of customer images and fashion class merchandise images.

As shown in fig. 10, the method adopts a multi-task twin convolution network structure, branches of two convolution networks respectively correspond to a customer image and a fashion class commodity image, and share neuron network parameters; at the output layer, branches of the two convolution networks respectively output types of fashion type commodities (such as clothes, trousers, bags, sunglasses and the like) in the images, meanwhile, branches of the two convolution networks respectively output embedded vectors, and the similarity of styles of the two input images can be compared by calculating an inner product.

Fig. 11 is a flowchart illustrating a method of making a recommendation of goods or services according to an identified shopping tendency of a customer according to the inventive concept. The drawings are merely examples and are not intended to limit the scope of the claimed application.

As shown in fig. 11, when a recommendation is made to a customer detected by the system, a commodity category to be recommended is first determined. The category of the commodity to be recommended can be specified by the customer through an interactive interface, and can be automatically determined by the system through calculation, for example, the principle based on the commodity category can comprise a shopping area where the customer is located, a sales strategy of a mall and the like.

When the commodity category to be recommended is determined, the commodity information of the category can be retrieved from a commodity database, wherein the commodity information comprises a characteristic vector which is stored in the commodity database and is used for abstractly describing the attribute of the commodity; according to the 'collaborative filtering' algorithm, the feature vector of the commodity can be obtained through a matrix decomposition method, and the specific calculation method can refer to 'Matrix Factorization Techniques for Recommender Systems'. Aiming at target customers, the shopping tendency feature vectors of the corresponding commodity categories in the models are found by identifying the corresponding shopping tendency models, and matching scores of the customers and the commodities are obtained by calculating inner products of the shopping tendency feature vectors and the feature vectors of the commodities. And selecting a plurality of commodities with highest matching scores from the commodities to recommend the commodities to the customer.

After the goods or services information to be recommended is determined for a certain customer, it can be transmitted or displayed to the customer in several ways.

Fig. 12 is a schematic diagram illustrating a method of recommending goods to an augmented reality device of a customer of a physical store according to the inventive concept of the present application. The drawings are merely examples and are not intended to limit the scope of the claimed application.

As shown in fig. 12, in the recommendation system, a camera of a physical store detects a customer and transmits a push invitation to the customer through a wireless signal. When an augmented reality device (e.g., helmet, glasses, cell phone, etc.) worn by the customer detects the invitation, it is decided by the augmented reality management application whether to accept the invitation. If the invitation is accepted, the system predicts the shopping tendency model according to the image of the customer and recommends the commodity. During tracking, recommended content may be updated based on the status of the customer. For example, when a customer is in a fruit area of a physical store, fruit-based merchandise is recommended; when a customer is in a file area of a physical store, a file class commodity is recommended.

Fig. 13 is a schematic diagram showing a method of displaying a recommendation of a commodity to a display near a customer of a physical store according to the inventive concept of the present application. The drawings are merely examples and are not intended to limit the scope of the claimed application.

As shown in fig. 13, in the system, a camera in a physical store is installed near a shelf, and when a customer is detected, the system predicts a shopping tendency model based on an image of the customer, and makes a recommendation of a commodity, and the recommended commodity information is displayed on a display near the customer. During tracking, recommended content may be updated based on the status of the customer. For example, when a customer in front of the display changes, a recommendation is made according to the shopping tendency model of the new customer.

Fig. 14 is a schematic diagram showing a method of displaying customer information and merchandise recommendations to an augmented reality display device of a sales assistant in a physical store according to the inventive concept of the present application. The drawings are merely examples and are not intended to limit the scope of the claimed application.

As shown in fig. 14, in the system, a sales assistant in a physical store wears an augmented reality device, and when a camera on the augmented reality device detects a customer, the system predicts a shopping tendency model from an image of the customer, and makes a commodity recommendation for the predicted shopping tendency model, and the recommended commodity information is displayed on the augmented reality display device of the sales assistant. During tracking, recommended content may be updated based on the status of the customer. For example, when a customer before sales assistance changes, a recommendation is made according to the shopping tendency model of the new customer.

The sales assistant in the system may also be a robotic agent with the functions of displaying images and conversations, etc., which provides shopping guide services to customers in the brick and mortar store based on the received recommendation information.

Fig. 15 is a schematic view showing a method of displaying customer information and merchandise recommendation on a face image displayed by a browser according to the inventive concept of the present application. The drawings are merely examples and are not intended to limit the scope of the claimed application.

As shown in fig. 15, when a user displays a character image on a user device, a shopping tendency model prediction can be made and the shopping tendency can be displayed for a character selected in the image to make a commodity recommendation targeted and the recommended commodity information can be displayed on the image.

As can be seen from the above description of the various embodiments, the present application first builds a set of shopping trend models based on a large number of customer shopping times. For a target customer, the application shoots the image information of the customer through a camera, and analyzes the visual characteristics of the customer by utilizing a deep learning method, so as to find a best matched customer trend model, and recommending goods or services to the customer according to the best matched customer trend model.

Further, the present application proposes an implementation using this technique in several different augmented reality shopping scenarios, including: the customer uses the augmented reality device to shop at the physical store, provides auxiliary information for the shopping assistant of the physical store, and augmented reality the shopping tendency information of the customer in the image on the browser.

Unlike the prior art, in the technique of the present application, when making a recommendation, no historical shopping data of the customer may be required, nor does the customer make any input or select any merchandise. The application can recommend commodities, including but not limited to fashion class commodities and other commodities with image visual characteristics, and also can be commodities or services without visual distinguishing characteristics.

Those skilled in the art will appreciate that the present application includes apparatuses related to performing one or more of the operations described herein. These devices may be specially designed and constructed for the required purposes, or may comprise known devices in general purpose computers. These devices have computer programs stored therein that are selectively activated or reconfigured. Such a computer program may be stored in a device (e.g., a computer) readable medium or any type of medium suitable for storing electronic instructions and respectively coupled to a bus, including, but not limited to, any type of disk (including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks), ROMs (Read-Only memories), RAMs (Random Access Memory, random access memories), EPROMs (Erasable Programmable Read-Only memories), EEPROMs (ELECTRICALLY ERASABLE PROGRAMMABLE READ-Only memories), flash memories, magnetic cards, or optical cards. That is, a readable medium includes any medium that stores or transmits information in a form readable by a device (e.g., a computer).

It will be understood by those within the art that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by computer program instructions. Those skilled in the art will appreciate that the computer program instructions can be implemented in a processor of a general purpose computer, special purpose computer, or other programmable data processing method, such that the blocks of the block diagrams and/or flowchart illustration are implemented by the processor of the computer or other programmable data processing method.

Those of skill in the art will appreciate that the various operations, methods, steps in the flow, acts, schemes, and alternatives discussed in the present invention may be alternated, altered, combined, or eliminated. Further, other steps, means, or steps in a process having various operations, methods, or procedures discussed herein may be alternated, altered, rearranged, disassembled, combined, or eliminated. Further, steps, measures, schemes in the prior art with various operations, methods, flows disclosed in the present invention may also be alternated, altered, rearranged, decomposed, combined, or deleted.

The foregoing is only a partial embodiment of the present invention, and it should be noted that it will be apparent to those skilled in the art that modifications and adaptations can be made without departing from the principles of the present invention, and such modifications and adaptations are intended to be comprehended within the scope of the present invention.

Claims

1. A method of recommending items, comprising:

Acquiring an image of a customer using a camera;

Generating a segmented image by segmenting the acquired image, wherein the segmented image indicates a classification of one or more pixels of the image corresponding to the customer;

Extracting partial image feature vectors of the customer according to the segmented images;

For each of one or more shopping trend models of a shopping trend model database, calculating a degree of correlation between a customer and a shopping trend model based on similarity between partial image feature vectors and feature vectors determined for the shopping trend model;

selecting a shopping tendency model with the highest correlation degree from the one or more shopping tendency models;

Determining recommended items for the customer according to the selected shopping trend model; and

Information related to the recommended item is provided.

2. The method of claim 1, wherein determining recommended items for the customer based on the selected shopping trend model comprises:

determining shopping tendencies of the customer based on the selected shopping trend model; and

And recommending the items to the customers according to the determined shopping tendency.

3. The method of claim 1, wherein capturing an image of a customer comprises:

at least one image of the customer is acquired by a camera.

4. The method of claim 3, wherein the field of view of the camera covers at least a portion of a shopping area of the physical store.

5. The method of claim 1, wherein capturing an image of a customer comprises:

At least one image of the customer is obtained from an image displayed on the user device.

6. The method of claim 1, wherein:

the feature vector determined for the shopping tendency model comprises at least one of a fashion category item feature vector, a customer image feature vector and a non-fashion category item feature vector;

Wherein calculating the correlation between the customer and the shopping trend model comprises:

Based on the partial image feature vector, calculating the correlation between the customer and each shopping tendency model according to the fashion category item feature vector, the customer image feature vector and the non-fashion category item feature vector.

7. The method of claim 1, further comprising:

And constructing a shopping tendency model through an online shopping database, so as to construct the shopping tendency model database.

8. The method of claim 7, wherein building a shopping trend model via an online shopping database comprises:

Clustering according to the feature vectors of the customers, and dividing the customers into different customer groups; and

For each customer group, feature vectors for different item categories are determined based on purchase data for the different item categories.

9. The method of claim 8, further comprising training the shopping trend model through statistics of online shopping records of customers, decomposing a purchase history matrix indicating customer purchase item histories, and determining feature vectors of customers.

10. The method of claim 1, further comprising:

and constructing an offline shopping tendency model through the physical store shopping database, so as to construct the shopping tendency model database.

11. The method of claim 10, wherein building an offline shopping trend model through a physical store shopping database comprises:

Extracting information from the physical store shopping database including at least one of: customer ID, customer's image, and customer's shopping record;

The method comprises the steps of analyzing images of customers to obtain appearance information of the customers and shopping behavior information of the customers; and

Classifying the customers into different customer groups through information acquired from the images of the customers; and

For each customer group, feature vectors of different item categories are established according to purchase data of the different item categories.

12. The method of claim 1, wherein determining recommended items for the customer based on the shopping trend model of the customer comprises:

Matching the image of the customer with the image of the item for each category of item in the selected shopping tendency model; and

A recommended item is determined based on a matching score between the customer's image and the item image.

13. The method of any of claims 1-12, wherein providing information related to the recommended item comprises:

sending a push invitation to a customer device comprising an Augmented Reality (AR) display; and

Information is received indicating whether to accept the push invitation from the customer device.

14. The method of claim 13, wherein providing information related to the recommended item comprises:

in response to the customer device accepting the push invitation, information related to recommending items is sent to the customer device.

15. The method of claim 13, wherein providing information related to the recommended item comprises:

information related to the recommended item is sent to a display located within a threshold distance of the customer's location.

16. The method of claim 13, wherein providing information related to the recommended item comprises:

information related to the recommended item is sent to one or any combination of the work terminal and the customer device.

17. The method of any of claims 1 to 12, wherein determining the recommended item for the customer based on the acquired image comprises:

and determining the recommended items for the customer according to the shopping area where the customer is located and the acquired image.

18. The method of claim 12, wherein the matching score is calculated based on an inner product between a feature vector of a single item in the selected shopping trend model and a feature vector of an item worn by the customer.

19. The method of claim 1, wherein the information related to the recommended item is provided using a display.

20. A non-transitory computer readable storage medium having stored thereon instructions that, when executed by a processor, cause the processor to perform the method of any of claims 1 to 19.

21. A system for recommending items, comprising:

An image acquisition unit for acquiring an image of a customer;

A determining unit for determining the position of the object,

Generating a segmented image by segmenting the acquired image, wherein the segmented image is indicative of a classification of one or more pixels of the image corresponding to the customer;

for selecting a shopping tendency model having a highest degree of correlation from the one or more shopping tendency models;

For determining recommended items for the customer based on the selected shopping trend model; and

And the network unit is used for providing information related to the recommended project.

22. The system of claim 21, wherein the determining unit determines shopping tendencies of the customer according to the selected shopping tendencies model, and makes item recommendations to the customer according to the determined shopping tendencies.

23. The system of claim 21, wherein the image acquisition unit acquires at least one image of the customer via a camera.

24. The system of claim 23, wherein the field of view of the camera covers at least a portion of a shopping area of a physical store.

25. The system of claim 21, wherein the image acquisition unit acquires at least one image of the customer from an image displayed on the user device.

26. The system of claim 21, wherein:

27. The system of claim 21, further comprising:

28. The system of claim 27, wherein building a shopping trend model via an online shopping database comprises:

29. The system of claim 28, further comprising training the shopping trend model through statistics of online shopping records of customers;

A purchase history matrix indicating a history of purchase items by a customer is decomposed to determine feature vectors of the customer.

30. The system of claim 21, further comprising:

31. The system of claim 30, wherein building an offline shopping trend model via the physical store shopping database comprises:

32. The system of claim 21, wherein the determining unit matches the image of the customer with the image of the item for each category of item in the selected shopping tendency model, and determines the recommended item based on a matching score between the image of the customer and the image of the item.

33. The system of any of claims 21-32, wherein providing information related to the recommended item comprises:

34. The system of claim 33, wherein providing information related to the recommended item comprises:

35. The system of claim 33, wherein providing information related to the recommended item comprises:

36. The system of claim 33, wherein providing information related to the recommended item comprises:

37. The system according to any one of claims 21 to 32, wherein the determining unit determines the item recommended for the customer based on a shopping area in which the customer is located and the acquired image.

38. The system of claim 32, wherein the matching score is calculated based on an inner product between a feature vector of a single item in the selected shopping trend model and a feature vector of an item worn by the customer.

39. The system of claim 21, wherein the network element provides information related to the recommended item using a display.

40. An apparatus for recommending items, comprising:

A processor; and

A memory in which instructions executable by the processor are stored, which when executed by the processor, cause the processor to perform the method of any one of claims 1 to 22.