KR102009130B1 - The System Providing Diagnosis of Makeup and Question and Answer Service - Google Patents

Abstract

According to the present invention, a system for providing a real-time makeup diagnosis and question and answer service comprises: a subscription processing module including a user subscription part for receiving user information from a user and an expert subscription part for receiving expert information from an expert; a question input module for receiving one or more photographed images and related questions wherein the one or more photographed images are made by photographing a face part of the user from the user; a mentor selection module including a list generating part for generating a mentor list listing a plurality of experts, a list providing part for providing the mentor list to the user, and a mentor selecting part for selecting at least one among the experts belonging to the mentor list from the user; a question providing module for providing the user′s photographed image and related questions to the expert selected from the user; and a mentoring performing module including a comment input part for receiving a mentor comment from the expert and a comment delivery part for providing the mentor comment to the user.

Description

The System Providing Diagnosis of Makeup and Question and Answer Service}

The present invention is a real-time makeup diagnosis and question and answer service providing system, in addition to being able to receive a diagnosis and question and answer service for their makeup from a makeup professional such as a makeup artist or a beauty youtuber in real time, big data analysis and artificial It provides a real-time make-up diagnosis and question-and-answer service providing system that enables automatic analysis and evaluation of the makeup of users using the service through intelligence.

With the development of the beauty industry, interest in cosmetics, make-up, and the like of users is increasing. Accordingly, the user's desire for cosmetics, makeup and the like is diversified. For example, since the skin color, face shape, and appearance of the neck may vary for each user, makeup suitable for the individual may be different. Therefore, the user may have difficulty in selecting a makeup that is suitable for them. In addition, the user may wonder whether the makeup is good after makeup, and what part should be supplemented.

In accordance with this trend, recently, an application for applying virtual makeup to a user's face has been developed. In this case, although it may cause curiosity and interest of the user, there is a limit that the individual must determine which makeup is suitable for the user. In other words, currently provided beauty services do not include the collaboration of makeup specialists, or even based on limited data, even if the collaboration of makeup specialists. Therefore, it is difficult to provide a personalized service for each user.

Therefore, as a prior art for overcoming such difficulties, Korean Patent Registration No. 10-1872635 discloses a 'make-up automatic evaluation system and its operation method'.

The automatic makeup evaluation system comprising a mobile terminal and a makeup server according to the present invention stores a mobile terminal and a makeup score data for photographing a face image and transmitting the photographed face image to the makeup server, and the face image from the mobile terminal. When receiving the data may include a makeup server for calculating the makeup score of the face image based on the makeup score data, and transmits the calculated makeup score to the mobile terminal.

Therefore, not only this provides a more reliable makeup evaluation service, but also a service that can receive a makeup evaluation similar to that of a makeup expert is currently being implemented and provided.

However, in addition to this, in addition to providing real-time diagnosis and Q & A services for makeup by a makeup artist such as a makeup artist or beauty youtuber, the user who uses this service through big data analysis and artificial intelligence It provides not only automatic analysis and evaluation of makeup, but also a real-time makeup diagnosis and question answering service providing system that analyzes makeup based on three-dimensional images of the user's face instead of simple two-dimensional images. The need to provide is emerging.

The present invention has been made in order to overcome the problems of the above-described technology, real-time makeup diagnosis and question and answer to receive a diagnosis and question and answer service for their makeup from a makeup professional such as a makeup artist or influencer in real time The main purpose is to provide services.

Furthermore, it aims to enable automatic analysis and evaluation of makeup of users using the service through big data analysis and artificial intelligence.

In addition, by providing a photographing guide for uploading an image for make-up diagnosis, a user can upload an appropriate image to facilitate make-up diagnosis.

In addition, by dividing the user's face into a plurality of areas, and make-up analysis and evaluation of each area can be made more detailed and detailed analysis and diagnosis.

In addition, the facial part is recombined into a three-dimensional image rather than a simple two-dimensional image, thereby enabling make-up diagnosis and analysis, thereby enabling more three-dimensional make-up diagnosis and analysis.

In order to achieve the above object, the real-time makeup diagnosis and question answering service providing system of the present invention, a subscription processing module including a user subscription unit for receiving user information from the user, and a professional subscription unit for receiving expert information from the expert; A question input module for receiving at least one photographed image and a related question photographing the face part of the user from the user; A list generator for generating a mentor list listing a plurality of the experts, a list provider for providing the mentor list to the user, and a mentor selected by the user from at least one of the experts belonging to the mentor list A mentor selection module including a selection unit; A question providing module for providing a photographed image and a related question of the user to the expert selected from the user; And a mentoring performing module including a comment input unit for receiving a mentor comment from the expert, and a comment transfer unit for providing the mentor comment to the user.

In addition, the system further includes a guide providing module including a guide generation unit for generating a guide image for the photographed image, and a guide providing unit for providing the guide image to the user, wherein the question input module includes: At least one photographed image and a related question of photographing the face of the user may be input from the user provided with the guide image.

In addition, the system, a composition analysis unit for analyzing the composition of each of the photographed image and the guide image, a similarity analysis unit for analyzing the similarity between the composition of the analyzed image and the composition of the guide image, and And a guide analyzing module including an additional image requesting unit requesting the user to re-enter the photographing image according to the similarity between the photographing image and the composition of the guide image being less than or equal to a preset reference similarity. do.

The system may further include an area detector configured to detect the at least one makeup area from the photographed image by analyzing the photographed image, an area analyzer configured to generate color information and application information for each of the makeup areas, and the color. And an area calculation unit for calculating an area score for each of the makeup areas according to information and the application information, and a comprehensive score calculation unit for calculating a comprehensive score for the photographed image based on the area points. It further comprises an automatic analysis module, wherein the comment transmitting unit, characterized in that for providing the user with the mentor comment and the overall score.

The real-time makeup diagnosis and question and answer service providing system according to the present invention,

1) To provide real-time makeup diagnosis and question and answer service to make-up artists such as make-up artists and influencers in order to receive diagnosis and question and answer service for their makeup,

2) Through big data analysis and artificial intelligence, we made automatic analysis and evaluation of user's make-up possible.

3) By providing a shooting guide for uploading an image for make-up diagnosis, the user can upload an appropriate image to make makeup diagnosis easier and at the same time, if the guide does not match, re-upload the image. ,

4) By dividing the user's face into a plurality of areas and enabling makeup analysis and evaluation for each area, more detailed and detailed analysis and diagnosis is possible.

5) The face part is recombined into a three-dimensional image rather than a simple two-dimensional image, and accordingly, makeup diagnosis and analysis are possible, thereby enabling more three-dimensional makeup diagnosis and analysis.

1 is a conceptual diagram showing a schematic configuration of a system for providing real-time makeup diagnosis and question and answer service of the present invention.
Figure 2 is a block diagram showing the basic configuration of the system for providing real-time makeup diagnosis and question and answer service of the present invention.
Figure 3 is a block diagram showing the detailed configuration of a real-time makeup diagnosis and question and answer service providing system of the present invention.
Figure 4 is a conceptual diagram showing an example of the guide image of the present invention.
5 is a conceptual diagram showing an example of makeup analysis of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The accompanying drawings are not drawn to scale, and like reference numerals in each of the drawings refer to like elements.

1 is a conceptual diagram showing a schematic configuration of a system 10 for providing real-time makeup diagnosis and question and answer service according to the present invention.

Referring to Figure 1 will be described the basic components constituting the real-time makeup diagnosis and question answer service providing system 10 of the present invention.

First, in the present invention, the user 20 refers to a subject who is diagnosed and evaluated for his makeup, and generally means a makeup beginner who is poor at makeup. Or even if you are not a make-up beginner, all the targets who want to improve their makeup skills and make excellent makeup may be the user 20. The user 20 is not limited to a specific gender as a male or a female, and the age group wearing makeup is also various, so the age range is not limited either.

Most of the users 20, that is, the general public who wears make up most of the cases have a worry about their makeup. Simple thoughts about whether this makeup looks good on you, you're not doing the wrong thing, or what kind of makeup method suits you better, what kind of makeup is popular for you, or good skin to make your makeup better What is the secret of having it? Therefore, the users 20 are subjects who have a need to receive a diagnosis about their makeup as well as various makeup worries.

The expert 30 generally refers to an influencer (make-up power blower or beauty youtuber), a make-up artist, or a beauty specialist with excellent makeup skills. It is obvious that such an expert 30 has excellent knowledge compared to the general public (normal user 20) with respect to makeup and beauty. Therefore, the expert 30 diagnoses the makeup of the user 20 and performs a function of presenting an answer to a problem or question related to makeup and beauty that the user 20 has.

The main server 40 is a series of subjects (central control server) for implementing the make-up diagnosis and question-answer service providing system 10 of the present invention, and includes a server PC and a network communication network. The main server 40 basically delivers the data received from the user 20 to the expert 30, and serves to provide the user 20 with the material received from the expert 30. In addition, it receives the revenue generated through the system 10, that is, the fee that the user 20 pays from the user 20, and serves to provide it to the expert 30.

In addition, the makeup diagnosis and question and answer service may be implemented in the form of a website, a program, or an application. In this case, in order for the user 20 to upload his makeup image, the user 20 may be linked with a smartphone on an image file upload method or an application. Uploading may be performed after shooting with the camera.

2 is a block diagram showing the basic configuration of the system 10 for providing real-time makeup diagnosis and question and answer service according to the present invention.

With reference to Figure 2 will be described in more detail with respect to the basic configuration included in the real-time makeup diagnosis and question and answer service providing system 10 of the present invention.

First, the system 10 of the present invention can be implemented in the form of a website, a program, an application, or the like. Therefore, the registration procedure of the user 20 and the expert 30 for the service may be necessary, and the subscription processing module 100 is included therefor.

The subscription processing module 100 includes a user subscription unit 110 and an expert subscription unit 120, respectively. The user subscription unit 110 includes a name, sex, age, skin type (intelligence / Complex / dry) and further, the user information including contact information (e-mail, mobile phone number) for providing a solution is performed. The expert subscription unit 120 receives expert information from the expert 30, wherein the expert information may include not only a name, age and gender, but also a portfolio of the user, a blog address, a YouTube address, or a current job. Information about the makeup shop may be included as expert information.

Furthermore, in the case of the user sign-up unit 110, the sign-up process may be performed for all users 20 who input user information. In the case of the expert sign-up unit 120, only a part of the experts 30 who input the expert information may be registered. Only the expert 30 who has processed and proven his or her ability or reputation may register with the system 10 of the present invention to act as the expert 30.

The mentor selection module 200, which is another configuration of the system 10 of the present invention, is configured to allow the user 20 to select an expert 30, that is, a mentor, to perform a make-up diagnosis and a question-and-answer response. Therefore, the user 20 generates and provides a list of experts 30 and is configured to receive any one of them. For this purpose, the list generator 210, the list provider 220, and the mentor selector 230 are selected. ) Is included by default.

The list generation unit 210 inputs expert information through the expert subscription unit 120 of the present invention, and generates a mentor list listing a plurality of the experts 30 approved for subscription on the system 10. Do this. At this time, the mentor list is the basic to allow the user 20 to provide more detailed information by deriving the mentor's name or nickname and the mentor's image as well as the biography of the mentor. In addition, it is natural that the cost information for the expert 30 should also be exposed if the cost of question and answer and diagnosis for each expert 30 is different.

The list providing unit 220 performs a function of providing the generated mentor list to the user 20, and outputs the mentor list to a display of a PC, a smartphone, a tablet PC, etc. possessed by the user 20. 20) to check the mentor list and to determine which experts 30 are included in the mentor list.

The mentor selector 230 selects at least one of the experts 30 included in the mentor list from the user 20, and performs the solution for the user 20 to diagnose his makeup and his question. It can be said to be a configuration to select a mentor to do. In this case, the user 20 may select one expert 30, but may of course select several experts 30. In this case, generally, the user 20 is selected by the user 30 by clicking on any one of the experts 30 included in the mentor list or inputting the name / nickname of the expert 30. . The expert 30 selected from the user 20 is matched with the user 20, and the expert 30 is authorized to perform a makeup diagnosis and a response to the makeup question of the user 20. .

The question input module 300, which is another component included in the system 10 of the present invention, basically includes a question input unit 320 to perform a function of receiving a photographed image and a related question from the user 20. In this case, the photographed image and the related question may be processed through a user's smartphone, PC, tablet PC, etc., and the photographed image and the user's 20 are connected to the system 10 of the present invention. Allows you to enter related questions.

The photographing image is an image of the face part of the user 20, and the entire face part is basically photographed as much as possible to make detailed diagnosis on the face part, and further, each part of the face part ( Eyes, nose, mouth, cheeks, chin, etc.) may be uploaded separately. Therefore, the photographed image should be at least one image including at least one front image taken with the entire face part. In addition, it is also possible to upload an image file photographing the face of the person when uploading the image, or the application or program implemented by the system 10 can be processed in conjunction with the user's smart phone, etc. on the system 10 Of course, the user may drive the smartphone camera and photograph the face to be uploaded as a photographed image.

The related question is a textual question of the user's 20 makeup problems or questions that the expert 30 wants to ask, and the user 20 may individually ask the related question, but the system 10 After the question list is generated in advance on the user 20, if the user 20 selects at least one of them, the user may process the input as a related question.

The question providing module 400, which is another basic configuration included in the system 10, photographs input from the user 20 to the expert 30 selected from the corresponding user 20 through the mentor selection unit 230. It serves to convey images and related questions. This can be provided by downloading the related question in text format and the photographed image in image file format to the expert 30, or displaying the related question and photographed image on the monitor, smartphone, or tablet PC of the expert 30. Through the expert 30 may provide a photographing image and related questions of the user 20.

The mentoring performing module 500, which is the last basic configuration, receives a comment from the expert 30 who has received the photographed image and the related question, and provides the comment to the user 20. For this, the comment input unit 510 and the comment are provided. It is configured to include a transmission unit 520 basically.

The comment input unit 510 receives a mentor comment from the expert 30 who has received the photographed image and the related question. In this case, the mentor comment includes the expert 30's answer to the makeup evaluation phrase and the related question given by the expert 30 based on the photographed image of the user 20. If the expert (30) evaluates that the eyebrows are too dark and the color combination is inappropriate in the image, it is better to draw a little lighter overall as a mentor comment. Trim the shapes a bit smoother. For hue, it is appropriate to place the point on either eye / lip only. If both the eye makeup and the lip makeup are dark, you can make an impression. ' Therefore, the mentor comment goes beyond simply answering the question of the user 20 and evaluates the makeup and evaluation of each part of the makeup of the user 20 determined through the photographing image, ie eyes, eyebrows, skin, and lips. It can be provided together. In addition, as a mentor comment, the makeup of the user 20 may be scored and input, for example, the expert 30 may receive an input from the expert 30 on how many points the makeup of the user 20 is out of 100 points.

The comment expert 30 performs the function of providing the user 20 with the mentor comment received through the comment input unit 510 described above, through which the user 20 immediately analyzes his / her makeup and At the same time to obtain an evaluation to be provided with the answer of the expert (30) about the make-up or beauty related concerns or questions that have been wondering.

Such a real-time makeup diagnosis and question and answer service providing system 10 can receive a makeup analysis, diagnosis, and evaluation of the expert 30 by simply uploading a makeup image of the user 20. It is effective to receive clear answers to the questions or questions that were asked from 30), as well as solutions and mentoring. In addition, since the expert 30 is based on being a makeup artist or an influencer, it is possible to attract more customers such as subscribers through his / her contents by providing makeup diagnosis and question and answer service of the present invention, thereby generating another profit. You can expect.

3 is a block diagram showing a detailed configuration of the real-time makeup diagnosis and question answering service providing system 10 of the present invention, Figure 4 is a conceptual diagram showing an example of the guide image of the present invention, Figure 5 is a makeup analysis example of the present invention A conceptual diagram showing the.

2 to 5 will be described with reference to Figures 3 to 5 further components that can be included in the system 10 for providing real-time makeup diagnosis and question answer service of the present invention.

First, in the present invention, the make-up diagnosis of the user 20 is made based on a photographed image uploaded by the user 20 by photographing his or her own facial part. At this time, when the user 20 does not see the makeup well, that is, when photographing the face part by uploading a photographed image at an angle at which the analysis of the makeup is difficult, there is a disadvantage in that make-up diagnosis becomes difficult. This is because in general, when people take a selfie, they tend not to take a self-camera for objective analysis, but to shoot at an angle or composition that looks pretty and handsome. ) Can provide a guide for the captured image.

This may be implemented through the guide providing module 600, which is an additional configuration that may be included in the system 10, the guide providing module 600 is to take a photographed image to be taken by the user 20, that is, to photograph the face of the person Provides a guide image for the captured image. An example of such a guide image is shown in FIG. 4, and the guide providing module 600 basically includes a guide generating unit 610 and a guide providing unit 620.

The guide generator 610 is a function of generating a guide image for a photographed image that is input from the user 20, that is, a photographed image for photographing the face of the user 20, which is illustrated in FIG. 4. If not, it can be implemented as a modeled image. Therefore, a photograph of a real person may be generated as a guide image, or a virtual image may be generated as a guide image as shown in FIG. 4. At this time, the generation of the guide image may be automatically generated on the system 10 (generated through 3D modeling), or the system 10 may allow a manager to photograph a specific person with an appropriate composition for makeup analysis and input it as a guide image. It is also possible. Such a guide image should basically include the front face part as shown in FIG. 4, and if the side image is included as shown in FIG. 4, more detailed analysis is possible, so the guide image may be at least one image including the face part front face. have. Obviously, the guide image should be an image having an appropriate composition for makeup analysis.

The guide provider 620 provides the guide image generated by the guide generator 610 to the user 20, and provides the guide image to the user 20 before the user 20 uploads the photographed image. In accordance with the composition of the guide image, the user 20 is to shoot the face of his own to upload as a photographed image. In this case, when the guide image is a total of three images including two front portions and two side portions at different angles as shown in FIG. 4, the user 20 guides the guide image based on the guide image provided through the question input module 300. Enter the same number of shots as the image. When the guide image as shown in FIG. 4 is provided, it means that the photographed image must also be a total of three image files including two front portions and two side portions at different angles.

By providing such a guide image, the user 20 can photograph his face at an angle that is easier for makeup analysis and upload it as a photographed image, thereby enabling a more objective analysis. Make-up analysis of 20) can be performed more easily.

However, even if such a guide image is provided, it is true that the user 20 may photograph the face part with a composition different from the guide image and upload it as a photographed image. Therefore, the system 10 of the present invention compares the composition of the guide image with the composition of the photographed image uploaded by the user 20, thereby providing a large gap between the composition of the photographed image input by the user 20 and the composition of the guide image. In this case, the user 20 may be requested to re-upload the captured image. For this purpose, the guide analysis module 700 may be included. The guide analysis module 700 includes a composition analyzer 710, a similarity analyzer 720, and an additional image requester 730.

The composition analyzer 710 analyzes the composition of the photographed image input from the user 20 and the guide image generated by the system 10, respectively. In this case, the composition in the present invention means the size (shooting distance) of the photographing object and the angle at which the object is photographed. Accordingly, by analyzing the composition of the photographed image and the composition of the guide image, it is determined to what degree the angle was taken based on the front face of the face, and also to determine the photographing distance between the camera and the object. For example, assume that a photograph of a, that is, a front face of FIG. 4 is given as a guide image. In this case, the composition of the guide image may be analyzed as having a shooting distance of 40cm and an angle of 0 °, and the composition of the photographing image may be analyzed as a shooting distance of 60cm and an angle of 40 °.

The similarity analyzer 720 analyzes the similarity between the analyzed composition of the photographed image and the composition of the guide image. In the case of the above-described example, the composition of the guide image was 40 cm at an angle of 0 ° and the composition of the photographed image was 60 cm at an angle of 40 °. At this time, since the shooting distance is 50% error, the similarity 50% and the angle 11%, so the similarity can be calculated as 89%. Therefore, the composition similarity can be calculated to be 66%. In this case, since the similarity calculation method is not limited, various known similarity calculation methods may be applied. Alternatively, an image analysis tool may be used to analyze compositional similarity between the guide image and the photographed image by comparing and analyzing the composition (shooting distance and angle).

The additional image requesting unit 730 performs a function of requesting the user 20 to re-enter the photographed image according to the similarity between the composition of the photographed image and the guide image being less than or equal to a preset reference similarity. This assumes that the reference similarity degree is set to 75% on the system 10, and if the similarity between the composition of the photographed image and the guide image is less than 75%, the user 20 requests re-entry of the photographed image. That is, when the composition similarity is calculated to 66% as described above, the user 20 is asked to retake the facial part. Therefore, the user 20 needs to re-enter the shot image, that is, retake the face part and re-enter it as the shot image. In this case, the reference similarity may be set on the server, and the reference similarity may be set by the administrator of the system 10.

Therefore, through the configuration of the guide analysis module 700 as described above, the user 20 can capture the facial part as close as possible to the guide image and input it as a photographed image, thereby inducing the user to upload a photographed image of the proper composition for makeup analysis. And, the expert 30 is to make it easier to perform the makeup analysis.

5 is a conceptual diagram showing an example of makeup analysis of the present invention.

3 and 5 will be described for the makeup automatic analysis configuration that can be added to the present invention. The system 10 of the present invention basically allows the expert 30 to analyze the photographed image of the face of the user 20 to analyze and diagnose the makeup, and furthermore, the system 10 automatically performs the diagnosis. Of course, it is also possible to analyze the makeup image to analyze the makeup of the user 20. To this end, the system 10 may include an automatic analysis module 800. In this case, the automatic analysis module 800 basically includes an area detector 810, an area analyzer 820, and an area calculator 830. ), The composite score calculation unit 840 may be configured.

The area detector 810 analyzes the photographed image and detects at least one makeup area from the photographed image. Since the photographed image is basically a photograph of the face of the user 20, the photographed image is one from the face part. This is to detect the above makeup area. For example, it may be possible to detect five makeup areas of the forehead / eyes / nose / lips / balls, and it is also possible to detect the makeup area even further.

The area analysis unit 820 performs a function of generating color information and application information for each of the makeup areas, wherein the application information is to determine whether the makeup product is applied. Basically, if the makeup product is recognized as not applied, the application information is recognized as a no makeup state, and if the makeup product is applied completely, it is recognized as fully applied. If a part of the makeup area is recognized as applying a makeup product, and a part of the makeup product is recognized as not applied, it may be recognized as a part applied. At this time, the distinction between no makeup and application is made by analyzing the uniformity of tone (the color and tone become more uniform when makeup) or the color difference (there is a big difference from the normal skin color of no makeup). Can be analyzed. Color information may be more easily determined by extracting RGB values of pixels included in the corresponding makeup area.

The area calculation unit 830 calculates the area score for each of the makeup areas according to the color information and the application information. That is, the combination of the tones is calculated through the color information determined through the RGB code, and the area score for each makeup area is calculated by determining whether the makeup product is uniformly applied through the application information. If the hue information (RGB code) judged in the make-up area differs only in brightness while having a similar saturation distribution, it can be judged that the gradation is well done. It may be determined that the makeup state is not uniform when it is recognized as applied and some parts are recognized as not applying the makeup product. Therefore, according to the color information and the application information, the score for each region can be calculated with a perfect score of 100 points or a perfect score of 10 points.

The comprehensive score calculation unit 840 calculates a comprehensive score for the photographed image based on the score for each area, and simply calculates the comprehensive score through an average value of the scores for each makeup area. Alternatively, among the makeup areas, the areas such as eyes / mouths may also calculate the importance of the scores for each area to reflect higher weights and calculate a comprehensive score based on the weights. That is, the comprehensive score calculation unit 840 calculates a comprehensive score for the photographed image, that is, the image in which the makeup of the user 20 is photographed by synthesizing the scores for each makeup area for each makeup area.

Therefore, if the area-specific scores and composite scores are automatically calculated on the system 10 as described above, the comment delivery unit 520 can provide the area 20 and the composite scores to the user 20 as well as the mentor comments. An example of this is provided in FIG. 5.

In addition, the makeup analysis of the present invention depends on the photographed image. That is, even if the photographed image is used with the same makeup product depending on the camera, or the brightness and color of the light in the photographed location, there may be a hue or distortion of the brightness.

Therefore, if there is a standard that can be corrected, the image can be corrected to obtain a more standardized and objective image, and analyze the makeup in the absence of light brightness or color, or color tone or brightness distortion according to the camera. You can do it. The additional configuration for this is described as follows.

First, the question input module 300 may provide a predetermined reference color image to the user 20 subscribed to the system 10 of the present invention through the reference color providing unit 310. At this time, the reference color image may be referred to as a color chip of a specific color, and may also be referred to as a small color paper. The reference color image is outputted with the reference color image output to the address input by the user 20, rather than provided as an image file. It is desirable to allow the user 20 to provide a fully controlled reference color image on the system 10 by causing it to be sent out. At this time, the reference color image is preset and manufactured / generated on the system 10, and the color is not limited thereto.

In this case, the question input unit 320 does not simply upload the photographed image of only the face of the user 20 in uploading the photographed image. Instead, the question input unit 320 captures the reference color image and the photographed image photographed together with the face of the user 20. You will be uploaded. That is, when the user 20 photographs the face part, the reference color image is photographed together with the face part by taking a reference color image in a hand and the like, and thus inputs the captured image. Therefore, in the photographed image, the color included in the reference color image, that is, the reference color is taken together.

Therefore, when analyzing the reference color and correcting the photographed image based on this, it is possible to obtain a photographed image without distortion by lighting or camera, so that even if the face part is photographed with different lights or different cameras, the photographed image with standardized brightness or color can be obtained. Will be.

To this end, the automatic analysis module 800 corrects the photographed image based on the reference color image, including the image compensator 850. In this case, the image compensator 850 is a reference analysis part 851, a comparative analysis part ( 852) and a correction processing part 853.

The reference analysis part 851 performs a function of analyzing the reference color information which is the color information of the reference color image provided to the user 20. For example, the reference color information of the reference color image is RGB (255, 97, 97). Since this can be easily understood by using a color analysis tool in an image program such as Photoshop, the detailed description will be omitted. At this time, the color analysis tool analyzes a specific position of the reference color image, that is, a specific pixel.

The comparison analysis part 852 performs a function of analyzing input color information which is color information of the reference color image included in the photographed image, which is a reference color photographed together with the face of the user 20 on the photographed image. The image can be easily analyzed by analyzing the image through a color analysis tool of an image program such as Photoshop. For example, a reference color image was taken with the photographed image, and this part was analyzed with Photoshop's color analysis tool, and the value of RGB (254, 96, 92) came out. In that case, the values of RGB (254, 96, 92) become input color information. In addition, at this time, when analyzing the input color information through the color analysis tool, analyzing the same position (pixel as the reference color image reference) that analyzed the color in the above-described reference analysis part 851 will enable more accurate analysis.

The correction processing part 853 performs a function of correcting the photographed image so that the input color information and the reference color information coincide. For example, as described above, the reference color information is RGB (255, 97, 97), and when the input color information is RGB (254, 96, 92), the photographed image is in a state in which all of the R, G, and B values are lower than the reference color information. Therefore, the image is corrected by increasing the R, G, and B values of the captured image so that the input color information becomes RGB (255, 97, 97).

The image compensator 850 as described above takes the reference color information of the reference color image provided to the user 20 as a standard value, and corrects the input color information contained in the photographed image by being distorted by a camera or lighting based on the reference color information. By doing so, it is possible to standardize color or brightness that may differ from image to image.

In addition, when the image analysis unit 850 is included in the automatic analysis module 800, the area detector 810 may analyze at least one makeup area by analyzing the corrected photographed image.

Furthermore, in the above-described configuration, the guide image may be provided to the user 20, and the photographed image corresponding to the composition of the guide image may be provided. At this time, if the guide image includes all of the bottom, top, front, left, and right sides of the face rather than the simple front or side of the face, at least five images including the bottom, top, front, left, and right sides of the face are also included. Can be more than one image. Combining this, it is also possible to obtain a three-dimensional image of the face portion of the user 20 three-dimensional.

To this end, the guide generation unit 610 may generate a plurality of guide images including a bottom guide image, a top guide image, a front guide image, a left side guide image, a right side guide image for the face part, and a guide providing module. 600 may provide the user 20 with a plurality of guide images for the bottom, top, front, left and right sides of the face part.

In this case, the user 20 uploads at least five or more photographed images by photographing the bottom, top, front, left, and right sides of the face unit based on the guide image through the question input unit 320.

In this case, if the combination of generating a three-dimensional image by combining the photographed images taken at various angles, the makeup analysis can also be performed based on the three-dimensional image. Therefore, for this purpose, the system 10 may further include a three-dimensional image generation module 900 for generating a three-dimensional image by combining the plurality of input images received. In this case, the 3D image generating module 900 basically includes a noise removing unit 910, an object extracting unit 920, a size correction unit 930, an angle correction unit 940, and a rendering unit 950. .

The noise removing unit 910 removes noise by preprocessing the plurality of photographed images received from the user 20. In this case, the noise removal is generally referred to as a bitmap operation, a luminance conversion operation, and a background color removal operation. In more detail, each of the photographed images is bitmaped to determine coordinates and color information for each pixel, and the edges of the object (the photographed object, which means the face of the user 20) included in the photographed image are sharper. By vectorizing the image and removing the background color, unnecessary noise included in the captured image is removed.

The object extractor 920 extracts only an image of an object, that is, a face part of the user 20, from the plurality of photographed images from which noise is removed. Since the background color is already removed through the noise removing unit 910, the noise and the background color are removed from the photographed image, and only the object (the face of the user 20) whose boundary is clear is extracted.

The size correction unit 930 corrects a shooting distance error with respect to the extracted object of each of the photographed images to correct the sizes of the plurality of objects, and captures each photographed image captured by the user 20. The distance may vary. Therefore, the size of the object is corrected so that the size of the object photographed in each direction is corrected by correcting the photographing distance error.

The angle correction unit 940 performs a function of correcting a photographing angle error with respect to each extracted object of the photographed image so as to form a plurality of objects at a right angle, which is also captured by the user 20. The shooting angle is different for each shot image. Therefore, the angle is corrected so that the face part (object) of the user 20 photographed from the front, left, right, bottom, and top surfaces forms a right angle, respectively. Therefore, the edges of the object (the face of the user 20) photographed in each of the front, left, right, bottom, and top surfaces may be coincident with each other through the size correction unit 930 and the angle correction unit 940. The pixel coordinates of the face part (object) of the user 20 may correspond to each direction.

The rendering unit 950 renders a plurality of objects by generating a plurality of objects based on the vertical axis coordinates and depth information of the object for each direction. The size correction unit 930 and the angle correction unit 940 are provided. After matching the edge and pixel coordinates through), the depth information is mapped by grasping the number of pixels to which color information is given for each vertical axis coordinate of the object photographing the left and right sides, and angles the plurality of objects based on this. 3D image is generated by arranging and rendering each other. Through this, the face of the user 20 can be generated as a three-dimensional image.

Therefore, when the three-dimensional image is generated in this way and the face portion of the user 20 is provided to the system 10 as the three-dimensional image, the area detection unit 810 analyzes the three-dimensional image and at least from the three-dimensional image. One or more makeup areas are detected, and the comprehensive score calculation unit 840 calculates a comprehensive score for the 3D image based on the score for each area.

Furthermore, when the 3D image is generated through the 3D image generating module 900 as described above, when there is residual noise that cannot be removed on the 3D image, the capacity of the image file is increased, so the processing speed is slowed and the quality of the 3D image is reduced. There is also a fear of deterioration. Accordingly, the three-dimensional image generation module 900 may further include an additional correction unit 960 for detecting and removing residual noise from the generated three-dimensional image to perform further correction on the three-dimensional image. Not only can the image be generated as a higher quality image, but also the capacity can be optimized. In this case, the additional correction unit 960 includes the edge detection part 961, the noise detection part 962, the noise removal part 963, Complementary pixel operation part 964, the image may be configured to include a complementary part (965).

The edge detection part 961 detects an edge from the 3D image through a Sobel operator, and the Sobel operator is a representative boundary detection algorithm for detecting a boundary region of an image corresponding to a high frequency component. Since the technology is a detailed description thereof will be omitted. In this case, the Sobel operator can detect edges with less noise than other boundary detection algorithms (Roberts operator, Prewitt operator, etc.).

The noise detection part 962 detects a pixel of an edge having a pixel value smaller than a predetermined reference value as noise. The noise detection part 962 determines that an edge having a pixel value smaller than a reference pixel value is ambiguously expressed as noise. Provide the function to detect. At this time, the reference value may be set through the system 10 administrator, or the reference value may be automatically set by the system 10 itself. In this way, no limitation is imposed on setting the reference value, and various methods may be applied.

The noise removing part 963 performs the function of removing the noise detected through the noise detecting part 962. In detail, the noise removing part 963 calculates a correction pixel value based on an average and a variance of pixel values of an edge. By removing the noise by replacing with the corrected pixel value, the image quality of the 3D image may be improved and capacity optimization may be performed.

The complementary pixel operation part 964 converts the frame of the 3D image into a binary frame and calculates a complementary pixel value of the binary frame through a recent interpolation method. It can be calculated through Equation 1.

Equation 1,

는 이진수 형태의 보완화소값,

는 2ⁿ배 축소시킨 후 다시 2ⁿ배 확대시킨 이진수 프레임,

는 비트 OR 연산)(here,

Is the complementary pixel value in binary form,

Is 2 ⁿ times smaller and then 2 ⁿ times larger

Is a bitwise OR operation)

Equation 1 is a formula for calculating a complementary pixel value from a bit OR operation on a pixel of a binary frame, through which a noise that has not been removed may be additionally removed through a noise removing process. That is, the image quality of the 3D image may be improved by additionally removing noise of a frame included in the 3D image.

의 블록으로 확대시켜 각 블록의 화소가 1의 값을 갖도록 함으로써 3차원이미지의 에지를 보완할 수 있게 된다.(구체적으로, 에지의 연결 흐름이 끊겨 보이는 느낌을 개선하는 효과를 준다.)Here, the bitwise OR operation means a process of comparing each digit of two binary digits and calculating one of them as 1 if there is 1, or 0, and the value of 1 in the binary frame reduced by such bit operation. Having the pixel back

By enlarging the block to the pixel of each block having a value of 1, the edge of the three-dimensional image can be compensated for (specifically, the effect of improving the discontinuity of the connection flow of the edge is improved).

The image complementary part 965 generates a complementary frame from the binary frame based on the complementary pixel value, and then replaces the frame of the 3D image with the supplementary frame to compensate for the edge of the 3D image. As a result, the image quality of the 3D image is further improved.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

As described so far, the configuration and operation of the real-time makeup diagnosis and question-answer service providing system 10 according to the present invention have been represented in the above description and the drawings, but this is merely described by way of example, and the spirit of the present invention is described above. The present invention is not limited to the drawings, and various changes and modifications are possible without departing from the technical spirit of the present invention.

10: system 20: user
30: expert 40: main server
100: subscription processing module 110: user registration unit
120: expert member 200: mentor selection module
210: list generation unit 220: list providing unit
230: mentor selector 300: question input module
310: reference color providing unit 320: question input unit
400: question provision module 500: mentoring performance module
510: comment input unit 520: comment delivery unit
600: guide providing module 610: guide generation unit
620: guide provider 700: guide analysis module
710: composition analysis unit 720: similarity analysis unit
730: additional image request unit 800: automatic analysis module
810: area detection unit 820: area analysis unit
830: calculation unit by area 840: comprehensive score calculation unit
850: Image Compensation 851: Standard Analysis Part
852: Comparative analysis part 853: Correction processing part
900: 3D image generation module 910: noise removing unit
920: object extraction unit 930: size correction
940: angle correction 950: rendering unit
960: additional correction 961: edge detection part
962: noise detection part 963: noise removal part
964: Complementary pixel calculation part 965: Image supplement part

Claims (14)

Real-time makeup diagnosis and question and answer service providing system,
A subscription processing module including a user subscription unit for receiving user information from a user and an expert subscription unit for receiving expert information from an expert;
A list generator for generating a mentor list listing a plurality of the experts, a list provider for providing the mentor list to the user, and a mentor selected by the user from at least one of the experts belonging to the mentor list A mentor selection module including a selection unit;
A question input module for receiving at least one photographed image and a related question photographing the face part of the user from the user;
A question providing module for providing a photographed image of the user and the related question to the expert selected from the user;
An area detector which detects at least one makeup area from the photographed image by analyzing the photographed image, an area analyzer which generates color information and application information for each of the makeup areas, and the color information and the application information An automatic analysis module including an area calculation unit for calculating an area score for each of the makeup areas, and a comprehensive score calculation unit for calculating a comprehensive score for the photographed image based on the area points;
And a mentoring execution module including a comment input unit for receiving a mentor comment from the expert, and a comment delivery unit for providing the user with the mentor comment, the score for each area, and the total score.
The question input module,
A reference color providing unit configured to provide a preset reference color image to the user, and a question input unit configured to receive at least one photographed image and the related question from which the reference color image and the user's face are photographed together from the user; ,
The automatic analysis module,
A reference analysis part for analyzing reference color information which is color information of the reference color image provided to the user, a comparison analysis part for analyzing input color information which is color information of the reference color image included in the photographed image, and the input color And an image correction unit including a correction processing part for correcting the photographed image so that the information and the reference color information coincide.
The area detection unit,
And detecting at least one makeup area by analyzing the corrected photographed image. The method of claim 1,
The system,
Guide generation unit for generating a guide image for the photographed image,
And a guide providing module including a guide providing unit for providing the guide image to the user.
The question input module,
And receiving at least one photographed image of the face part of the user and a related question from the user who has received the guide image. The method of claim 2,
The system,
A composition analyzer for analyzing each composition of the photographed image and the guide image;
A similarity analyzer for analyzing the similarity between the analyzed composition of the photographed image and the composition of the guide image;
And a guide analysis module including an additional image requesting unit requesting the user to re-enter the photographed image according to the similarity between the photographed image and the composition of the guide image being less than or equal to a preset reference similarity. Real-time makeup diagnosis and question and answer service providing system. delete delete The method of claim 1,
The system,
A guide generator for generating a plurality of guide images for the photographed image;
And a guide providing module including a guide providing unit providing a plurality of the guide images to the user.
The question input module,
Receiving a plurality of photographed images and related questions of photographing the face part of the user from the user provided with the guide image,
The system,
And a three-dimensional image generation module for generating a three-dimensional image by combining the plurality of input images.
The area detection unit,
Analyzing the three-dimensional image to detect at least one makeup area from the three-dimensional image,
The comprehensive score calculation unit,
And a comprehensive score for the 3D image is calculated based on the score for each region. The method of claim 6,
The guide image,
It includes the bottom guide image, the top guide image, the front guide image, the left side guide image, the right side guide image for the face part,
The question input module,
Receiving at least five photographed images and related questions including the bottom, top, front, left and right sides of the face part from the user provided with the guide image,
The 3D image generating module,
A noise removing unit for pre-processing a plurality of the captured images to remove noise;
An object extracting unit extracting an object from the plurality of photographed images from which noise is removed;
A size correction unit for correcting a size of a plurality of the objects by correcting a shooting distance error with respect to the object of each of the extracted images;
An angle correction unit for correcting a photographing angle error with respect to the extracted object of each photographed image so that a plurality of the objects form a right angle;
And a rendering unit for generating a 3D image by rendering the plurality of objects based on the vertical axis coordinates and depth information of the object for each direction. The method of claim 6,
The 3D image generating module,
And an additional correction unit configured to detect and remove residual noise from the generated 3D image to perform further correction on the 3D image. The method of claim 8,
The supplementary supplement is
An edge detection part for detecting an edge from the 3D image through a Sobel operator,
A noise detection part for detecting, as noise, a pixel at an edge having a pixel value smaller than the set reference value;
A noise removing part for removing the detected noise;
A complementary pixel operation part for converting a frame of the 3D image into a binary frame and calculating a complementary pixel value of the binary frame through a recent interpolation method;
After generating a complementary frame from the binary frame on the basis of the complementary pixel value, characterized in that it comprises an image complementary part for replacing the frame of the three-dimensional image by the complementary frame to complement the edge of the three-dimensional image, Real-time makeup diagnosis and question and answer service providing system. The method of claim 9,
The complementary pixel value is,
A system for providing a real-time makeup diagnosis and question and answer service, which is calculated through Equation 1 below.
Equation 1,

(here,

Is the complementary pixel value in binary form,

Is 2 ⁿ times smaller and then 2 ⁿ times larger

Is a bitwise OR operation) delete delete delete delete

Images