JP2002111998A - Image composition method and information processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image composition method capable of compositing an image of a decentralized processing type in which a portable communication terminal with small process capability can also be particitaped; and an information processor therefore. SOLUTION: In a second terminal connected to one or a plurality of first terminals so as to mutually communicate with each other, a composite image obtained by compositing a second image obtained at its own terminal with an object image extracted from among a first image transmitted from the first image is generated, and this composite image is transmitted to the first terminal to display, thereby decreasing a process load of the first terminal. Thus, an image composition of a decentralized processing type is carried out between a portable communication terminal with small processing capability and the second terminal.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a composite image generation method and an information processing apparatus for generating a composite image while transmitting the image via, for example, the Internet.

[0002]

2. Description of the Related Art With the spread of the Internet in recent years, a user can enjoy a video while downloading image data from a remote server using a personal computer, or photograph his / her own face using a camera connected to the personal computer. There is an increasing opportunity to enjoy the so-called videophone function of sending each other. In addition, the captured image is transmitted as it is,
There has been an increasing demand for creating various virtual spaces by cutting out an object such as a face, and displaying it by combining it with another image instead of displaying it.

[0003] The inventors of the present invention have described a document "Real-time object extraction using self-similar model" (Ida,
Hori, Sanbonsugi, Proceedings of the 6th Image Sensing Symposium, pp. 161-166, 2000)
We proposed a face image transmission system that combines the extracted user's face image with another background image. According to this system, distant persons can enjoy a conversation while seeing each other's faces by sharing the composite image.

FIG. 9 shows a schematic configuration of the system described in the above document. A captured image 2 captured by the camera 1 is input to the extraction unit 3. The cutout unit 3 automatically detects a face area from the captured image 2 and cuts out the head of the person along the contour. Here, to extract specifically means to generate an alpha mask of an object. The alpha mask is a binary image having the same size as the original image, in which “0” pixels are arranged in the background area and “1” pixels are arranged in the object area. Here, image data with an alpha mask is referred to as a cut-out image. The cut-out image 4 is sent to a compression unit 5 and converted into compressed data 6. The compression unit 5 encodes both the image data and the alpha mask using, for example, a shape mode of MPEG-4. The processing up to this point is performed by the terminal 7.

[0005] The compressed data 6 is transmitted to the terminal 8 and input to the reproducing unit 9 thereof. In the reproducing unit 9, the compressed data 6
Is converted into a cutout image 10 and sent to the synthesizing unit 11.
On the other hand, the background image 13 recorded in the image storage unit 12 such as a hard disk is also sent to the synthesizing unit 11.

The synthesizing unit 11 generates a synthesized image obtained by synthesizing the cut-out image 10 of the face with the background image 13. In the combining process, specifically, the pixel value of the alpha mask is “0” for the cut-out image while referring to the alpha mask.
Is replaced with the background image. The image synthesized in this way is called a synthesized image.

[0007] The composite image 14 is sent to a compression unit 15 and converted into compressed data 16. Since the composite image 14 is a normal image without an alpha mask, the compression unit 15 encodes only the image data. The compressed data 16 is sent back to the terminal 7.

In the terminal 7, the compressed data 16 is reproduced
, And converted into a composite image 18. Synthetic image 18 is C
It is sent to a display unit 19 such as an RT monitor and displayed.

In the above document, there is disclosed a system in which a plurality of face images are combined into one image by inputting another cut-out image to the terminal 8, and the faces of each other can be seen.

[0010] In the system described in the above-mentioned document, object extraction and synthesis processing can be distributed to terminals. However, if the amount of computation that can be processed per unit time of the terminal 7 is smaller than that of the terminal 8, that is, if the processing capacity is low, the processing at the terminal 7 becomes a bottleneck, and the overall operation speed decreases, As a result, the frame rate of the image may decrease.

[0011]

As described above, conventionally,
When generating a composite image by distributed processing between multiple terminals,
The power consumption and circuit scale are small, and the processing capacity is low.
For example, a portable communication terminal such as a mobile phone and a PDA has a problem that a processing load is large and it is not easy to participate.

In view of the above problems, the present invention provides an image synthesizing method and an information processing apparatus for enabling distributed processing type image synthesizing in which a portable communication terminal having a small processing capability can participate. The purpose is to:

[0013]

According to the image synthesizing method of the present invention, in a second terminal communicably connected to one or a plurality of first terminals, a second image acquired by the own terminal and the second image are stored. A first image is generated by synthesizing an object image extracted from the first image transmitted from the first terminal and transmitting the synthesized image to the first terminal for display. Terminal simply provides the first image,
Since it only receives a synthesized image synthesized by using it, the processing load on the first terminal can be reduced, and thus the first terminal can be reduced.
Even if the terminal is a portable communication terminal having a small processing capability, distributed processing-type image synthesis can be performed with the second terminal.

An information processing apparatus according to the present invention includes a means for receiving a first image from a partner apparatus, a means for extracting a predetermined object image from the first image, and a second image acquired by the own apparatus. Means for generating a combined image combining the object image and the object image, and means for transmitting and displaying the combined image to the partner device, the partner device simply provides a first image, Since it only receives a synthesized image synthesized by using it, even if the partner device is a portable communication terminal having a small processing capability, it is possible to perform distributed processing type image combining with the partner device.

Preferably, instruction information for a second image to be combined with the first image is added to the first image, and the composite image is generated based on the instruction information. Is transmitted together with the instruction information for the image, by generating a new composite image based on the instruction information, it is possible to generate a composite image according to the preference of the provider of the first image.

[0016]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows a system configuration for implementing the image synthesizing method according to the present embodiment. The first terminal 201 and the second terminal 206 are wired or connected so that they can communicate with each other. Connected by a wireless communication line, the second terminal 206 extracts an object image from the first image sent from the first terminal and converts it into a second image acquired by the second terminal. By combining the images, a synthesized image is created, and the synthesized image is transmitted to the first terminal 201. Note that there may be a plurality of first terminals, and in this case, the second terminal only needs to be communicably connected to the plurality of first terminals.

The first terminal 201 has a small power consumption and a small circuit size, and has a low processing capability, for example, a mobile phone, a P
It is a portable communication terminal such as DA.

The second terminal 206 is preferably the first terminal 201, that is, a personal computer having a higher processing capability than a portable communication terminal.

In the first terminal 201 of FIG. 1, first,
The first image 203 captured by the camera 202 is a compression unit 2
04 is input. In the compression unit 204, the first image 20
3 is converted to compressed data 205 and transmitted to the second terminal 206.

In the second terminal 206, the compressed data 205
Is input to the reproducing unit 207, and is converted into the first image 208. The first image 208 is input to the cutout unit 209,
Then, the object image is cut out. This cut-out object image is called a cut-out image. The cut-out image 210 is generated by the combining unit 21
Sent to 1. On the other hand, the second image 213 stored in the image storage unit 212 composed of a recording medium such as a hard disk and a memory is read out, and is also read out by the combining unit 211.
Sent to

The combining section 211 combines the cut-out image 210 with the second image 213 as a background image to generate a combined image 214. The composite image 214 is sent to the compression unit 215 and converted into compressed data 216. Compressed data 2
16 is sent to the first terminal 201.

In the first terminal 201, the compressed data 216
Is input to the reproducing unit 217, and is converted into a composite image 218.
The composite image 218 is sent to and displayed on a display unit 219 such as a CRT monitor or a liquid crystal display.

With the above configuration, the first terminal 20
No. 1 does not require the clipping process and the encoding of the alpha mask in the compression unit 204, so that the processing amount can be minimized and the power consumption and the circuit scale can be reduced.

FIG. 2 is a flowchart for explaining the processing operation of the system of FIG.

At the second terminal 206, an object image is cut out and the image is synthesized. In the first terminal 201, a first image from which an object is cut out is acquired by, for example, photographing it with an attached camera 202, or a storage unit (FIG. 1) configured in advance from a recording medium such as a hard disk or a memory. The first image is acquired by reading out the information from the first image.

Step S1: The first terminal 201 acquires a first image. For example, the camera 202 captures an image including a face image of the user of the first terminal 201. FIG. 7 shows an example of the first image thus obtained.

Step S2: Copy the first image to the second terminal 2.
06. At this time, if the capacity of the communication line is small, the first
Then, after the data is received by the second terminal 206, the data is reproduced by the reproducing unit 207.

Step S3: The second terminal 206 extracts an object image from the received first image. For this processing, the method described in the above-mentioned document “Real-time object extraction using self-similar mapping” may be used, and the processing operation in that case will be described with reference to the flowchart shown in FIG.

Step S11: The position of the human face is detected from the first image using the degree of separation and the like.

Step S12: An outline shape such as an octagon is set at the detected position, and the outline of the outline is extracted by making the outline of the outline shape coincide with the head of the person using a fractal method or the like. I do.

Step S13: The extracted head is cut out. That is, an alpha mask in which “1” is arranged inside the contour of the head and “0” is arranged outside the contour is generated.

Returning to the description of FIG.

Step S4: The second terminal 206 reads the image stored in the image storage unit 212 and sets it as the second image. The second image does not need to be an image read from the image storage unit 212. For example, the second image is captured and captured by a camera (not shown in FIG. 1) connected to the second terminal 206. For example, a landscape image at that time may be used.

Step S5: The second image is used as a background image, and the image of the object extracted earlier is combined with the background image to generate a combined image.

Step S6: The generated composite image is transmitted to the first terminal 201. Also at this time, when the capacity of the communication line is small, the data compression is performed by the compression unit 215 before transmission at the second terminal 206, and the data is reproduced by the reproduction unit 217 after reception at the first terminal 201.

Step S7: The first terminal 201 displays the received composite image on a display unit 219 such as a liquid crystal display or a CRT monitor.

As described above, in the first terminal 201, the first terminal
It is not necessary to perform the process of extracting the object from the image of (1), and the processing amount can be minimized.

In the above description, the case where the first image and the second image are only one has been described. However, when both or one of them is a temporally continuous image (moving image), By repeating the processing shown in FIG. 2, a composite image can be created for a moving image.

Further, for example, the second terminal 206 is a server device having a high processing capability, and is particularly effective for an application in which various unspecified plural client terminals (corresponding to the first terminal 201) access. is there. This is because there is a high probability that some client terminals have very low processing capabilities.

(Second Embodiment) Next, a case where the system described above is used for selling goods via the Internet will be described. That is, in such a case,
For example, a first image as shown in FIG. 7 is transmitted from a terminal on the purchaser side (first terminal) to a terminal on the store side (second terminal) as a server device for providing a product sales service. In the second terminal, a composite image generated based on the composite image as shown in FIG. 8 is sent back to the terminal on the purchaser side.

In the second terminal, a face image is cut out from the first image shown in FIG. 7, and as shown in FIG. 8, this cut out image H1 is combined with a background image as a second image, and In the upper center, a face image H2 of the salesperson, an image M1 of the hat requested by the purchaser, images M2 and M3 of recommended hats in addition to the hat, and the selling prices of the hats are also displayed.

The salesperson is at the terminal that performs image synthesis, that is, at the second terminal, and the salesperson's face image and hat image are synthesized at step S5 and the synthesis unit 211 in FIG. Audio is also transmitted separately. As a result, face-to-face sales can be performed even if the salesperson is away from the purchaser, and an improvement in sales of the product can be expected.

FIG. 4 shows a system configuration for implementing the image synthesizing method according to the second embodiment. The same parts as those in FIG. 1 are denoted by the same reference numerals, and different parts will be described. That is, the second terminal 2 in FIG.
The configuration around the synthesizing unit 211 is different. Note that there may be a plurality of first terminals, and in this case, the second terminal only needs to be communicably connected to the plurality of first terminals.

The first parameter 621 used for cutting is sent from the operating unit 620 to the cutting unit 209.
The first parameter 621 is for improving the accuracy of cutting out.

The operation unit 620 is used when the contour is shifted due to insufficient cutting accuracy in the cutting unit 209.
This is for the operator of the second terminal 206 to perform an auxiliary operation for cutting out. For example, the first image shown in FIG. 7 is displayed on the display unit of the second terminal 206, and the operator, that is, the salesperson, uses the operation unit 620 to draw a rough outline of the person using a mouse or the like. Draw or simply click on the center of the face. By using these coordinates as the first parameter 621, the accuracy of contour line extraction and the accuracy of face position detection are improved. In the above example of merchandise sales, the accuracy of clipping can be improved without bothering the buyer.

The third image 6 obtained by photographing the operator of the second terminal 206, that is, the salesperson, with the camera 622
23 is sent to the extracting section 624. The cutout unit 624 cuts out the salesperson's face image 625 and sends it to the synthesizing unit 211.

In the information storage unit 612, in addition to the second image to be used as a background image, an image of a product (for example, a cutout image of a hat (which may be simply called an image of a hat) in this case) and the like. The stored images 626 are also sent to the synthesizing unit 211. Then, the salesperson operates the operation unit 620.
Is operated, the second parameter 627 for composition is sent to the composition unit 211.

The second parameter 627 is, for example,
It is used to indicate which cutout image is to be combined with which position in the background image, to indicate the size of the cutout image, or to switch the background image. For example, it is input by instructing selection of a desired cut-out image and an arrangement position.

In the synthesizing section 211, the second parameter 62
7, the image cut out by the cutout unit 209, the face image of the salesperson cut out by the cutout unit 624, and the image of the product read from the information storage unit 612 in the second image as the background image At the arrangement position specified by the second parameter 627.

At the time of synthesizing by the synthesizing unit 211, for example, a face image of a purchaser, a facial image of a sales clerk, a hat, etc., based on the second parameter 627 input by the salesperson operating the operation unit 620 It is possible to change the arrangement and size of the image of the product and to switch the background image to another image. By switching the background image, it is possible to excite the atmosphere and amplify the purchase will. Further, as shown in FIG. 8, it is also possible to combine the hats as if they were wearing a hat. Even if the purchaser moves his / her head, the position is detected by the cutout unit 209, so that the hat can be moved following the movement.

The operation section 620 may be operated by a person other than the salesperson.

In the above description, the case where the combination parameter is set at the second terminal, which is the terminal on the store side, has been described. However, the present invention is not limited to this case. It is also possible to set parameters.

FIG. 5 shows a system configuration in a case where the combination parameter is set at the first terminal which is the terminal on the purchaser side. The same parts as those in FIG. 1 are denoted by the same reference numerals, and different parts will be described. That is, a parameter setting unit 430 is newly provided in the first terminal 201 of FIG. 5, and the combining unit 211 of the second terminal 206 performs combining based on the parameters set by the parameter setting unit 430. Has become. Note that there may be a plurality of first terminals, and in this case, the second terminal only needs to be communicably connected to the plurality of first terminals.

In parameter setting section 430, parameters 434 for combination are set in accordance with the operation of the purchaser and sent to compression section 204. In the compression unit 204, the image captured by the camera 202, that is, the first image 20
3 is multiplexed with the parameter 434 set by the parameter setting unit 430 on the compressed data obtained by converting
05 is transmitted.

The reproducing unit 207 of the second terminal 206 separates the parameter 436 from the transmission data 205 and sends it to the synthesizing unit 211. The combining unit 211 combines the images according to the parameter 436.

FIG. 6 is a flowchart for explaining the processing operation of the system of FIG.

In FIG. 6, the same parts as those in FIG. 2 are denoted by the same reference numerals, and different parts will be described. That is, in FIG. 6, step S2 of FIG.
2 and step S23, and step S5 in FIG.
Is replaced with step S24.

In step S22, the parameter setting unit 4
In step 30, parameters for combination are set. For example, while looking at the composite image transmitted from the second terminal on the shop side as shown in FIG. 8 and displayed on the display unit 219, the purchaser does not use the hat image M1 but a different hat image. Click the image M3 with the mouse. Then, what indicates the product hat corresponding to the selected hat image M3 is the parameter set by the purchaser. When a list of products is provided from the second terminal 206 to the first terminal 201 in advance, a desired product (for example, a hat) is selected from the list using a mouse or the like. Then, what indicates the selected product is a parameter set by the purchaser. Various parameters other than those described above can be considered as parameters set by the customer. For example, it may be for designating a background image. In short, any method can be used as long as it informs the seller of the customer's desire / instruction for the second image or the combined image to be combined with the first image.

In step S23, as the first image,
For example, a composite image as shown in FIG. 8 or an image as shown in FIG. 7 and a parameter are transmitted.

For the first image from which no object image has been extracted as shown in FIG. 7, an object image is extracted in the same manner as described with reference to FIG. 2 (step S3), and the first image as a background image is extracted. (Step S4 to Step S4)
24).

Further, in step S24, the synthesizing unit 211, for example, converts the image M3 of the hat selected by the buyer into the head of the buyer according to the parameters sent together with the first image from the first terminal. To be synthesized.

In FIG. 6, the process returns to the first step immediately after the last step (step S7) is completed. This enables processing of a moving image.

Thus, it is possible to generate a composite image according to the purchaser's preference.

According to the present invention, for example, the second terminal 2
06, the first terminal 2 sent as the first image
It is also possible to extract a face image of the user No. 01 and synthesize a makeup image on the face image. That way, you can even try out different colored lipsticks and buy the one you like while staying in a remote location.

Also, an image of a famous talent is prepared in the second terminal 206 in advance, and an image of the user of the first terminal 201 extracted from the first image is combined with the image to make it as if the talent is It is also possible to immediately obtain an image as if it were taken together with the remote location.

In the case where the present system is used for selling goods, if a purchase button is arranged on the composite image, the purchaser can inform the seller of the purchase intention by clicking the purchase button. Alternatively, even if the purchase button is not arranged, there is a method of placing an order by double-clicking on the product, for example. In these cases, the operation of the buyer can be transmitted to the seller by using the embodiment described with reference to FIGS.

The method of the present invention described in the embodiments of the present invention can be executed by a computer as a program such as a magnetic disk (floppy (registered trademark) disk, hard disk, etc.), an optical disk (CD-RO).
M, DVD, etc.) and a storage medium such as a semiconductor memory.

It should be noted that the present invention is not limited to the above-described embodiment, and can be variously modified in an implementation stage without departing from the gist of the invention. Furthermore, the above embodiments include inventions at various stages, and various inventions can be extracted by appropriate combinations of a plurality of disclosed configuration requirements. For example, even if some components are deleted from all the components shown in the embodiment, (at least one of) the problems described in the column of the problem to be solved by the invention can be solved, and the effect of the invention can be solved. If at least one of the effects described in (1) is obtained, a configuration from which this component is deleted can be extracted as an invention.

[0070]

As described above, according to the present invention,
It enables distributed processing type image synthesis in which even a portable communication terminal having a small processing capacity can participate.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration example of a first terminal and a second terminal according to a first embodiment of the present invention.

FIG. 2 is a flowchart for explaining processing operations of a first terminal and a second terminal according to the first embodiment.

FIG. 3 is a flowchart for explaining an object image cutout process.

FIG. 4 is a diagram showing a configuration example of a first terminal and a second terminal according to a second embodiment of the present invention.

FIG. 5 is a diagram showing another configuration example of the first terminal and the second terminal according to the second embodiment.

FIG. 6 is a flowchart for explaining processing operations of a first terminal and a second terminal according to the second embodiment.

FIG. 7 is a view showing a specific example of a first image.

FIG. 8 is a diagram showing a specific example of a composite image.

FIG. 9 is a diagram showing a configuration example of each terminal for performing a conventional distributed processing type image synthesis.

[Explanation of symbols]

 201: first terminal 202: camera 203, 208 ... first image 204 ... compression unit 205, 216 ... compressed data 206 ... second terminal 207 ... reproduction unit 209 ... cutout unit 210 ... cutout image 211 ... synthesis unit 212 ... second image storage unit 213 ... second images 214 and 218 ... composite image 215 ... compression unit 217 ... reproduction unit 219 ... display unit

Claims (8)

[Claims] A second terminal communicably connected to one or more first terminals and a second image acquired by the first terminal and a first image transmitted from the first terminal.
An image combining method comprising: generating a combined image by combining an object image extracted from an image of a first terminal; and transmitting the combined image to the first terminal for display. 2. The first terminal transmitted from the first terminal.
The information of the user of the first terminal with respect to the second image to be combined with the first image is added to the image of the first image, and the second terminal is added to the received first image. 2. The image synthesizing method according to claim 1, wherein the synthetic image is generated based on the instruction information. 3. The first terminal adds the instruction information of the user of the first terminal to the composite image to the composite image and transmits the composite image to the second terminal, and the second terminal receives the composite image. The method according to claim 1, wherein a new composite image is generated based on the instruction information added to the composite image and the composite image. 4. A means for receiving a first image from a partner device, a means for extracting a predetermined object image from the first image, and a second image and an object image acquired by the own device. An information processing apparatus comprising: a unit configured to generate a combined image that has been combined; and a unit configured to transmit the combined image to the partner device and display the combined image. 5. When the received first image is added with instruction information of a user of the counterpart device for a second image to be combined with the first image, the combined image is displayed based on the instruction information. 5. The method according to claim 4, wherein
An information processing apparatus according to claim 1. 6. A means for receiving, from a partner device, the composite image to which instruction information of the user of the partner terminal for the composite image has been added, based on the composite image and the instruction information added to the composite image. The information processing apparatus according to claim 4, further comprising: means for generating a new composite image. 7. A machine-readable recording medium on which a program for causing a computer to execute a process of generating a composite image using a first image transmitted from a partner device is recorded. A process for receiving a first image, a process for extracting a predetermined object image from the first image, a second image and the object image acquired separately from the first image A recording medium storing a program for causing a computer to execute a process for generating a combined image obtained by combining the above and a process for transmitting and displaying the combined image to the partner device. 8. A program product for causing a computer to execute a process of generating a composite image using a first image transmitted from a partner device, wherein the program product receives a first image from the partner device. And a process for extracting a predetermined object image from the first image; and generating a combined image obtained by combining the second image acquired separately from the first image and the object image. And a process for transmitting the composite image to the partner device and displaying the composite image.