JP5963921B2 - Digital camera and composite image display method of camera - Google Patents

Description

  The present invention relates to a digital camera having a function of reproducing a moving image using an external device and a composite image display method of the camera.

  In recent years, it has become possible to shoot moving images in various portable devices such as digital cameras and mobile phones. However, it is difficult for a general user to shoot a moving image that accurately captures the movement of a subject that does not perform a fixed movement. For this reason, ordinary users can only capture monotonous moving images. On the other hand, in a television program or the like created by an expert, it is possible to shoot while frequently changing the composition with a plurality of cameras, so that it is possible to shoot a moving image that is not monotonous. However, it is very difficult for a general user to take such a photographing method.

  Even if it is a monotonous moving image at the time of shooting, it can withstand viewing if it has undergone appropriate editing work. In recent years, various techniques for enabling moving image editing on a moving image reproduced on a television screen have been proposed (see, for example, Patent Document 1).

Japanese Patent No. 2822187

  In the conventional moving image editing technology such as Patent Document 1, no consideration is given to the viewer who appreciates the moving image obtained after editing, and the moving image obtained after editing is not necessarily interested in the viewer. It is not always a moving image. For this reason, the viewer of the moving image may be bored.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a digital camera and a composite image display method of a camera capable of obtaining a moving image that is easily interested by a viewer who appreciates the moving image. To do.

  In order to achieve the above object, a digital camera according to a first aspect of the present invention includes an imaging unit that acquires a moving image, a recording unit that records a moving image, and an external device having a display unit. A communication interface unit for transmitting a recorded moving image, and a moving image obtained by operating the imaging unit when a viewer is watching a moving image reproduced on the display unit of the external device A face detection unit that detects a face image in the image, a determination unit that determines whether or not the facial expression detected by the face detection unit has changed, and the moving image transmitted to the external device includes the face A synthesizing unit for synthesizing images, and the imaging unit captures an image when receiving a signal indicating a facial image synthesis command from the external device, and the synthesizing unit detects the facial expression detected by the face detecting unit. Sent at the timing of the change Characterized by combining the image and the face image.

  In order to achieve the above object, the composite image display method of the camera according to the second aspect of the present invention captures a moving image at a timing when a signal indicating a face image compositing command is received from an external device, The captured moving image is transmitted to the external device having the display unit, and the face in the moving image obtained by capturing the image when the viewer is watching the moving image reproduced on the display unit of the external device. When an image is detected, it is determined whether the expression of the detected face image has changed, and when it is determined that the expression of the face image has changed, the expression of the face image has changed The moving image transmitted to the external device at a timing and the face image are synthesized.

  According to the present invention, it is possible to provide a digital camera and a combined image display method of a camera that can obtain a moving image that is easily interested by a viewer who appreciates the moving image.

It is a figure which shows the structure of the moving image reproduction system which has a digital camera which concerns on one Embodiment of this invention. It is a figure which shows the mode at the time of use of a moving image reproduction system. It is FIG. 1 of the flowchart which shows the main operation | movement of a digital camera. It is a figure which shows the example of a display of the display part at the time of the reproduction | regeneration mode of a digital camera. It is FIG. 2 of the flowchart which shows the main operation | movement of a digital camera. It is a figure shown about the example of a synthesis | combination of a face image. It is a flowchart shown about a position change process. It is FIG. 3 of the flowchart which shows the main operation | movement of a digital camera. It is a figure shown about the example of a remote control operation display image. It is a flowchart shown about an external reproduction end process. It is a figure shown about the example of a display of a confirmation screen.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing a configuration of a moving image reproduction system having a digital camera according to an embodiment of the present invention. As shown in FIG. 1, the moving image playback system is configured by a digital camera 100 and a television receiver 200 that are communicably connected via a communication cable 300.

  The digital camera 100 according to the present embodiment includes an imaging unit 101, a face detection unit 102, a display unit 103, a sound collection unit 104, a recording unit 105, a clock unit 106, a control unit 107, and an operation unit 108. And a communication interface unit 109. Note that the digital camera 100 according to the present embodiment has, as operation modes, a moving image shooting mode for shooting a moving image, a moving image playback mode for playing back a moving image, and communication for performing data communication with the TV 200. And at least a mode.

  The imaging unit 101 includes a lens for condensing a subject image (not shown), an imaging element for converting the subject image condensed by the lens into an electrical signal, and the like. The imaging unit 101 captures a subject and acquires a still image of the subject. A moving image can also be acquired by continuously performing imaging by the imaging unit 101.

  The face detection unit 102 detects the face of the subject in the image obtained by imaging by the imaging unit 101. The face information detected by the face detection unit 102 can be used for, for example, control for adjusting the focus of the lens constituting the imaging unit 101 to the face position, or control for appropriately exposing the face in the image. . Note that the face detection in the face detection unit 102 is performed, for example, by detecting the feature amount of the face in the image (the shape and positional relationship of eyes and nose). In addition, the face detection unit 102 according to the present embodiment can detect facial expressions (especially smiles).

  The display unit 103 displays a still image or a moving image acquired by the imaging unit 101. The display unit 103 is a liquid crystal display disposed on the back surface of the digital camera 100, for example.

  The sound collection unit 104 acquires sound around the digital camera 100 at the time of moving image shooting, and converts the acquired sound signal into an electric signal.

  The recording unit 105 records a still image or a moving image acquired by the imaging unit 101. The recording unit 105 is, for example, a semiconductor memory card that is detachable from the digital camera 100. The clock unit 106 measures the date and time when still image shooting or moving image shooting was performed in the imaging unit 101. The shooting date and time counted by the clock unit 106 is recorded in the recording unit 105 in association with the still image or the moving image acquired by the imaging unit 101.

  The control unit 107 controls each block in the digital camera 100 including control of the imaging operation of the imaging unit 101, control of the display operation of the display unit 103, operation control of the sound collection unit 104, and the like. In addition, the control unit 107 also performs various data processing such as image processing on the image obtained by the imaging unit 101 and sound processing on the sound obtained by the sound collection unit 104. Here, the control unit 107 in this embodiment also has a function as a synthesis unit. That is, the control unit 107 combines the image of the viewer who is viewing the moving image with the moving image recorded in the recording unit 105 and associates the audio of the viewer with the moving image recorded in the recording unit 105. Processing is also performed. Further, the control unit 107 also has a function as a trimming unit, and also trims and extracts a face image (face image) in the image obtained by the imaging unit 101.

  The operation unit 108 is an operation unit for the user to perform various operations on the digital camera 100. After identifying the operation of the operation unit 108, the control unit 107 executes processing corresponding to the operation of the operation unit 108. As the operation unit 108, for example, an operation unit for setting the operation mode of the digital camera 100, an operation unit for instructing execution / termination of shooting by the digital camera 100, and an operation unit for performing various selection operations Etc. are included.

  The communication interface unit 109 is an interface for the digital camera 100 to perform data communication with a television receiver (hereinafter abbreviated as TV) 200. In the present embodiment, HDMI (High Definition Multimedia Interface) (registered trademark) is used as the communication interface unit 109. However, the communication interface unit 109 is not particularly limited as long as it is an interface suitable for moving image transfer. In addition, although a wired interface is shown in FIG. 1, a wireless interface corresponding to infrared communication, radio wave communication, or the like may be used.

  The TV 200 as an example of the external device of the present embodiment includes a broadcast receiving unit 201, a display unit 202, a network communication unit 203, an information superimposing unit 204, a control unit 205, a remote control signal receiving unit 206, and a communication interface. Part 207.

  The broadcast receiving unit 201 includes an antenna or the like for receiving a television broadcast from a television station (not shown) or the like. The display unit 202 displays a moving image based on the television broadcast received via the broadcast receiving unit 201. The display unit 202 in the present embodiment is also capable of displaying a moving image transferred as digital data from the digital camera 100.

  The network communication unit 203 is an interface for the TV 200 to communicate with a server or the like (not shown) via the Internet. The information superimposing unit 204 superimposes various data obtained from a server or the like (not shown) via the network communication unit 203 on the moving image displayed on the display unit 202.

  The control unit 205 controls each block in the TV 200 including control of the display operation of the display unit 202, operation control of the information superimposing unit 204, and the like. The remote control signal receiving unit 206 receives a remote control signal from the remote controller (remote controller) 400 of the TV 200. Recognizing the remote control signal received by the remote control signal receiving unit 206, the control unit 205 executes processing corresponding to the operation of the remote control 400.

  The communication interface unit 207 is an interface for the TV 200 to perform data communication with the digital camera 100. The communication interface unit 207 is also HDMI, for example.

  FIG. 2 is a diagram showing a state in use of the moving image reproduction system according to the present embodiment. As shown in FIG. 2, the digital camera 100 is connected to the TV 200 via a communication cable 300. In such a state, the viewer 500 can view the moving image 202a displayed on the display unit 202 of the TV 200. In addition, the viewer 500 can remotely control the TV 200 by operating the remote controller 400.

  Here, in the present embodiment, an image of the face of the viewer 500 is acquired by the imaging unit 101 of the digital camera 100 during playback of the moving image 202a transferred from the digital camera 100 via the communication cable 300, and the viewer 500 face images 202 b can be combined with a moving image transferred from the digital camera 100 and displayed on the display unit 202 of the TV 200. By performing such display, for example, as shown in FIG. 2, when two viewers 500 are watching the moving image 202a, the mutual reaction can be confirmed on the display unit 202, and thereby the moving image 202a. It is possible to enliven In this way, even if the moving image to be reproduced is not edited in advance, the moving image is given pleasure not only to view the moving image. That is, in the present embodiment, an image having the same effect as that of viewing the edited image is displayed on the display unit 202 of the TV 200.

  The details of the operation of the moving image reproduction system as shown in FIG. 2 will be described below with reference to FIGS. Note that the TV 200 in the present embodiment displays the moving image transmitted from the digital camera 100 and the operation of transmitting an operation signal indicating the operation content of the remote control 400 to the digital camera 100 as an external signal when the remote control 400 is operated. Just do. Therefore, in the following description, the operation on the digital camera 100 side will be particularly described. In addition, the digital camera 100 according to the present embodiment can capture and reproduce a still image. In the following description, operations relating to moving image shooting and moving image reproduction will be particularly described.

  In FIG. 3, the control unit 107 of the digital camera 100 first determines whether or not the operation mode of the digital camera 100 is the moving image shooting mode (step S101). If it is determined in step S101 that the operation mode of the digital camera 100 is the moving image shooting mode, the control unit 107 continuously operates the imaging unit 101 to capture images (step S102), and the continuous operation of the imaging unit 101 is performed. Is displayed on the display unit 103 in real time (step S103). The photographer can determine the start timing, composition, and the like while viewing the moving image displayed on the display unit 103. After the moving image is displayed on the display unit 103, the control unit 107 determines whether an instruction to start capturing a moving image is given by the operation of the operation unit 108 by the photographer (step S104). If it is determined in step S104 that an instruction to start capturing a moving image has not been given, the control unit 107 determines the operation mode in step S101 again. In the determination in step S104, when an instruction to start capturing a moving image is given, the control unit 107 records the moving image obtained via the imaging unit 101 and the sound obtained via the sound collecting unit 104. Recording on the unit 105 is started (step S105). In this embodiment, for example, the Motion JPEG method is used as a moving image recording method. The Motion JPEG method is used because it is easy to edit a moving image in units of frames (details will be described later on the synthesis of a face image). Of course, if the specification allows even predetermined conditions such as the timing of synthesis, etc. It may have an inter-frame compression process such as H.264. In addition, it is assumed that recording of a moving image via the imaging unit 101 and recording of sound via the sound collection unit 104 are performed simultaneously so as to be synchronized with each other.

  After the start of moving image shooting, the control unit 107 determines whether an instruction to end moving image shooting has been given by the operation of the operation unit 108 by the photographer (step S106). If it is determined in step S106 that an instruction to end moving image shooting has not been given, the process returns to step S105, and the control unit 107 continues to record the moving image. Also, in the determination in step S106, when an instruction to end the shooting of the moving image is given, the control unit 107 ends the recording of the moving image. At this time, the control unit 107 records the moving image recorded in the recording unit 105 in association with the shooting date and time recorded by the clock unit 106. After the recording of the moving image is completed, the control unit 107 performs the processing after step S101.

  In step S101, when the operation mode of the digital camera 100 is not the moving image shooting mode, the control unit 107 determines whether the operation mode of the digital camera 100 is the moving image playback mode (step S107). ). If it is determined in step S107 that the operation mode of the digital camera 100 is the moving image playback mode, the control unit 107 displays a thumbnail image indicating the moving image (sound is also associated) recorded in the recording unit 105. As shown in FIG. 4A, it is displayed on the display unit 103 (step S108). The image displayed as the thumbnail image is a representative image of each moving image. This representative image is, for example, an image of the first frame in a moving image or an image of a frame to which an index described later is attached.

  After displaying the thumbnail image, the control unit 107 determines whether or not a moving image has been selected by the operation of the operation unit 108 of the viewer (step S109). If it is determined in step S109 that no moving image has been selected, the control unit 107 continues displaying the thumbnail image. In addition, when a moving image is selected in the determination in step S109, the control unit 107 causes the display unit 103 to display a screen for selecting a moving image reproduction destination as illustrated in FIG. In the example of FIG. 4B, “1. Playback” indicates that the playback destination of the moving image is the display unit 103 of the digital camera 100, and “2. TV” is the playback destination of the moving image of the TV 200. The display unit 202 is displayed. After the screen shown in FIG. 4B is displayed, the control unit 107 determines whether the playback destination of the moving image is the display unit 103 (step S110). If the playback destination of the moving image is the display unit 103 in the determination in step S110, the control unit 107 reads the moving image currently selected by the viewer from the recording unit 105, as shown in FIG. Thus, the reproduction of the read moving image on the display unit 103 is started (step S111). At this time, as shown in FIG. 4C, a message indicating that the moving image is being reproduced may be displayed.

  After starting the reproduction of the moving image, the control unit 107 determines whether or not the moving image to be reproduced on the display unit 103 has been switched by the operation of the operation unit 108 by the user (step S112). When the moving image is switched in the determination in step S112, the control unit 107 switches the moving image to be reproduced after the reproduction of the moving image is completed (step S113). This switching is performed, for example, by switching to an old (or new) moving image with a shooting date and time with respect to the moving image being played each time the operation unit 108 is operated. After the moving image is switched, the control unit 107 performs the processing after step S109 on the moving image selected by the switching.

  Further, when the moving image is not switched in the determination of step S112, the control unit 107 determines whether or not the reproduction end of the moving image is instructed by the operation of the user operation unit 108 (step S114). . If the end of the moving image is not instructed in the determination in step S114, the control unit 107 performs the processing after step S109. On the other hand, in the determination in step S114, when a moving image end operation is performed, the control unit 107 performs the processing from step S101.

  In the determination in step S107, when the operation mode of the digital camera 100 is not the moving image playback mode, the control unit 107 executes the process of the flowchart shown in FIG. The process of FIG. 5 is a process corresponding to the communication mode. Although details will be described later, in the communication mode, a moving image is reproduced on the display unit 202 of the TV 200. The digital camera 100 operates in accordance with an external signal input from the TV 200 in accordance with the operation of the remote controller 400 by the viewer.

  During the communication mode, the control unit 107 determines whether an external signal is received from the TV 200 via the communication interface unit 109 (step S201). If it is determined in step 201 that no external signal is input from the TV 200, the process returns to step S101 in FIG. In the determination in step S201, when an external signal is input from the TV 200, the control unit 107 identifies the input external signal (step S202). Then, the control unit 107 determines whether or not the external signal is a signal indicating a compositing instruction for a face image (step S203). If it is determined in step S203 that the external signal is not a signal indicating a face image synthesis command, the control unit 107 stops the operation of the imaging unit 101 if the imaging unit 101 is currently operating (step S204). . As will be described in detail later, in the communication mode, the imaging unit 101 operates while a signal indicating a face image synthesis command is being input. After stopping the operation of the imaging unit 101, the control unit 107 determines whether or not the external signal is a signal indicating a moving image reproduction command (step S205). If it is determined in step S205 that the external signal is a signal indicating a moving image reproduction command, the control unit 107 transmits an image of a frame to be transmitted next to the TV 200 (step S206). On the other hand, if it is determined in step S206 that the external signal is not a signal indicating a moving image playback command, the control unit 107 performs an external playback end process after a predetermined time, for example (step S207). The external reproduction end process will be described later. After the processing in step S206 or step S207 is completed, the control unit 107 performs processing after step S101 in FIG.

  In the determination in step S203, when the external signal is a signal indicating a compositing command, the control unit 107 temporarily stops the reproduction of the moving image being reproduced, and then activates the imaging unit 101 to display the image of the viewer. Obtained in units of frames (step S208). Then, the control unit 107 detects the face portion in the viewer's image obtained via the imaging unit 101 by the face detection unit 102, trims the detected face image, and plays the moving image being reproduced on the display unit 202 of the TV 200. For example, the face image is temporarily stored in the recording unit 105 so that the face image of the viewer and the moving image currently being reproduced can be combined and stored later, if necessary. Recording is performed (step S209). In this way, it is possible to save the recording capacity of the recording unit 105 by trimming only the face image. In addition, by limiting the range to be combined to a narrow range, it is possible to make the viewer's facial expression easier to see without any extra background or the like being captured. At this time, the sound collection unit 104 is activated, and the voice is simultaneously captured so that the change in the voice can be monitored as necessary. You may make it determine the timing to synthesize | combine. Of course, either a face image or a sound may be designated.

  After the combined display process and provisional recording of the face image outside the reproduction screen (or the peripheral part of the moving image being reproduced), the control unit 107 performs a position change process (step S210). This position changing process will be described with reference to FIGS. In the position change process, as shown in FIG. 6A and FIG. 6B, the face image 202b of the viewer is synthesized at a predetermined position of the moving image 202a reproduced and displayed on the display unit 202 of the TV 200. This is a process of changing the position of the synthesized face image 202b. At this time, the size of the face may be changed.

  In FIG. 7, the control unit 107 determines whether or not the processing mode in the position change process is the automatic change mode (step S301).

  If it is determined in step 301 that the processing mode is the automatic change mode, the control unit 107 determines the facial expression of the viewer from the viewer's face image recorded in the recording unit 105 in step S209 of FIG. It is determined whether or not the facial expression detected by the detection unit 102 has changed to, for example, a smile (step S302). If it is determined in step S302 that the facial expression of the viewer changes to a smile, the control unit 107 causes the non-contrast portion (for example, an image as shown in FIG. 6B) in the frame image to be transmitted to the TV 200 next time. The face image being imaged is synthesized with the background portion in the middle (step S303). In order to determine the synthesis display timing of the face image in the moving image being reproduced, a change in the size of the laughing voice or the speaking voice may be used in addition to the change in the facial expression. This is because everyone is worried about the expression when they get excited. At the time of compositing the facial image into the moving image being played back, the control unit 107 records information indicating the timing at which the facial image is synthesized, the facial image at that time, and the position of the frame in the recording unit 105 as a synthesis history. Keep it. At this time, as described above, voices such as laughter and spoken voice may be recorded. This allows you to know later how the viewer has reacted. In this way, the face image is appropriately combined with the moving image being reproduced, so that the display can be changed. Even a video shot by an amateur who tends to fall into a monotonous state, a viewer's face image sometimes intrudes into the screen, resulting in a moving image with changes. Here, although only a smile is invaded into the moving image being played back, such a facial expression may be used based on the idea that a monotonous image will change if a boring face appears.

  In addition, when the face image always invades the moving image being played back, there is no inflection, so when the facial expression or the sound volume returns, or when there is an operation with the remote control 400, The invading face image is saved outside the screen being played or in the periphery. In the digital camera 100, in order to realize such a reproduction form, the moving image and the face image being reproduced are combined and displayed on the TV 200 (sent by HDMI). In this case, the combination information As (history), it is assumed that the composition is canceled.

  Also, if the face image is located outside or around the moving image frame being played, it is better to display the face image as inconspicuously as possible, so it is better to display the face according to the screen size of the display unit 202 of the TV 200. The display size of the image may be changed. If the TV 200 has a large screen, it can be confirmed that the face is reflected without using a wide range of the screen.

  Note that there may be a plurality of viewers of the moving image, and in this case, not all the facial expressions change to smiles. When there are a plurality of faces as an image of the viewer obtained through such an image pickup unit 101, for example, only a face image whose expression is changed to a smile is synthesized. Of course, all the face images may be combined. Such switching may be determined by the size of the voice. If the voice is loud, the faces of everyone are combined in the playback image. If the voice is small, only the smiling face is composited in the playback image. Although omitted for simplification of description, if the viewer's facial expression is no longer a smile, the composition in step S303 is canceled and the face image is saved outside the screen of the reproduced image or in the periphery. In addition, although the portion without contrast is written, it may be a portion with no time change. When a person is a subject, any other place may be used as long as the face is not hidden.

  If the processing mode is not the automatic change mode in the determination in step S301, or if the facial expression of the viewer has not changed to a smile in the determination in step S302, the control unit 107 controls the remote controller 400 by the viewer. It is determined whether or not an external signal indicating that the face image composition position is to be changed has been input by the operation (step S304). If it is determined in step S304 that an external signal indicating that the face image synthesis position is to be changed is input, the control unit 107 follows the operation of the remote controller 400 by the viewer in the frame image to be transmitted to the TV 200 next. The face image recorded in the recording unit 105 is synthesized at the designated position (step S305). After the synthesis of the face image, the control unit 107 records information indicating the position of the frame where the face image is synthesized in the recording unit 105 as a synthesis history.

  After combining in step S303 or step S305, the control unit 107 adds index information to the frame image transmitted next by the TV 200 (step S306). Such a device can assist the search when reviewing later. Thereafter, the control unit 107 transmits a frame image obtained by combining the face images to the TV 200 (step S307). Thereafter, the control unit 107 ends the process of FIG. Note that after the face image is synthesized, the control unit 107 records information indicating the position of the frame where the face image is synthesized in the recording unit 105 as a synthesis history.

  As described above, according to the present embodiment, by adding index information to a moving image automatically or manually, it is possible to automatically or manually specify a point that is considered to have risen during reproduction of the moving image. This makes it easy to identify the raised points in the moving image during the next appreciation. In particular, when a plurality of people are watching a moving image, the facial expressions of each viewer at that time can be seen, and a moving image that is more interesting to the viewer can be created.

  In the determination in step S304, if no external signal is input, the control unit 107, for example, after a predetermined time has elapsed, for example, a position other than the main display unit in the image of the frame to be transmitted to the TV 200 next (for example, The face image recorded in the recording unit 105 is synthesized on the outside of the moving image 202a as shown in FIG. 6A, and the frame image obtained by synthesizing the face image is transmitted to the TV 200 (step S308).

  Here, it returns to description of FIG. 3 again. In the determination in step S110 of FIG. 3, when the playback destination of the moving image is the display unit 202 of the TV 200, the control unit 107 performs a transition process to the communication mode shown in FIG. That is, the control unit 107 displays the representative image of the moving image currently selected by the viewer (for example, the image of the first frame or the image of the frame to which the index information is added) on the TV 200 via the communication interface unit 109. ) Is transmitted (step S401).

  After transmitting the representative image, the control unit 107 transmits a remote control operation display image to the TV 200 via the communication interface unit 109 (step S402). The remote control operation display image guides the operation of the remote control 400 when a moving image from the digital camera 100 is displayed on the display unit 202 of the TV 200 as shown in FIGS. 9A and 9B. It is a display image 202c for doing. By viewing such a remote control operation display image 202c, the viewer can easily understand what operation can be performed by operating which button of the remote control 400.

  Here, the remote control operation display image 202c shown in FIG. 9A is an image displayed on the display unit 202 of the TV 200 after the representative image is transmitted. When the playback button of the remote control 400 is pressed while the remote control operation display image 202 c shown in FIG. 9A is being made, a signal indicating a playback command is input from the TV 200 to the digital camera 100. On the other hand, when the imaging button of the remote controller 400 is pressed, a signal indicating a compositing command is input from the TV 200 to the digital camera 100. A remote control operation display image 202c shown in FIG. 9B is an image displayed on the display unit 202 of the TV 200 after the imaging button of the remote control 400 is pressed. When the playback button of the remote control 400 is pressed while the remote control operation display image 202c shown in FIG. 9B is being made, a signal indicating a playback command is input from the TV 200 to the digital camera 100. On the other hand, when the position change button of the remote controller 400 is pressed, the position of the face image 202b displayed on the display unit 202 is changed, and the composition position of the face image 202b from the TV 200 to the digital camera 100 is changed. A signal indicating the change is input.

  After transmitting the remote control operation display image 202c, the control unit 107 turns off the display on the display unit 103 of the digital camera 100 (step S403). After turning off the display of the display unit 103, the control unit 107 performs the processing after step S101 in FIG. That is, in the communication mode, since the moving image recorded in the digital camera 100 is displayed on the display unit 202 of the TV 200, it is considered that the viewer does not see the display unit 103 of the digital camera 100. Therefore, the display on the display unit 103 is turned off in order to save energy. After turning off the display of the display unit 103, the control unit 107 shifts the operation mode of the digital camera 100 to the communication mode (step S404). In addition, you may make it perform the display which shows that it is in communication mode as shown in FIG.4 (d), without turning off the display of the display part 103. FIG.

  Next, the external reproduction end process will be described with reference to FIGS. In FIG. 10, the control unit 107 determines whether or not a synthesis history corresponding to a moving image that has been reproduced is recorded in the recording unit 105 (step S501). In the determination in step S501, when the synthesis history is recorded in the recording unit 105, the control unit 107 confirms to the viewer whether or not to record the moving image obtained by synthesizing the face image in the recording unit 105. An image for displaying the screen is transmitted to the TV 200 (step S502). FIG. 11 is a diagram showing an example of a confirmation screen displayed on the display unit 202 of the TV 200. In the example of FIG. 11, a representative image 202a of a moving image that has been played back is displayed on the display unit 202, and a remote control operation display image 202c for guiding the operation of the remote controller 400 of the viewer during the external playback end process is displayed. Display.

  After transmitting the image for displaying the confirmation screen as shown in FIG. 11, the control unit 107 determines whether or not an instruction to save the moving image is given by the operation of the remote controller 400 by the viewer (step S503). In the determination in step S503, if saving is instructed, the frame that has been combined in the moving image is specified according to the combination history, and the face image recorded in the recording unit 105 is combined with the image of the specified frame. Then, it is recorded in the recording unit 105 (step S504). By re-recording a moving image obtained by synthesizing a face image, for example, other members can later relive the appearance of watching the moving image together. For example, when a state of an athletic meet is photographed with the digital camera 100, if the parent and child first view a moving image recording the state of the athletic meet and send the moving image at that time to the grandparents, You can experience the experience as if you were participating in a parent-child group. As described above, when there is no invasion process of the face image in the moving image being played back, the face image is outside the screen of the moving image being played back, or is in the peripheral part as shown in FIG. Is assumed to have no composition history. Only the face image intrusion scene as shown in FIG. In this case, the size of the face may be changed with the remote controller 400. In addition, if the face is displayed as it is, there is a person who does not like to see his / her face, so other face images such as an avatar may be displayed instead.

  As described above, in the present embodiment, since the face image synthesis history is recorded, it is possible to easily replace the recorded face image with another image.

  If the composite history is not recorded in the recording unit 105 in step S501, if saving of moving images is not instructed in step S503, or after step S504, the control unit 107 performs communication by operating the remote controller 400 by the viewer. It is determined whether or not an instruction to end reproduction of the moving image in the mode is given (step S505). When it is determined in step S505 that the end of moving image playback in the communication mode is instructed, the control unit 107 changes the operation mode of the digital camera 100 to the playback mode (step S506). Thereafter, the processing after step S101 is performed. On the other hand, when it is determined in step S505 that the end of moving image playback in the communication mode is not instructed, the control unit 107 determines whether or not replay is instructed by an operation of the remote controller 400 by the viewer ( Step S507). If it is determined in step S507 that replay is not instructed, the control unit 107 performs the processing after step S505. Further, when the replay is instructed in the determination in step S507, the control unit 107 executes the processing after step S201 in FIG.

  As described above, according to the present embodiment, when a moving image recorded in the recording unit 105 of the digital camera 100 is reproduced on the display unit 202 of the TV 200, a viewer who is viewing this moving image. Can also be played back together. Thereby, it is possible to obtain a moving image in which a viewer of the moving image is likely to be interested. In addition, index information is added to the frame where the facial expression of the viewer smiles and the facial image is synthesized, or to the frame where the facial image synthesis position is specified by the viewer. It is also possible to specify an important scene in the moving image.

  Furthermore, the moving image to be played back on the TV 200 is only the moving image selected by the digital camera 100, so that the private moving image recorded on the digital camera 100 is unexpectedly displayed on the large screen on the display unit 202 of the TV 200. There is nothing.

  Here, in the above-described embodiment, the viewer's face image is combined with the moving image reproduced on the display unit 202 of the TV 200. On the other hand, the present embodiment can also be applied to a case where a viewer's face image is combined with a moving image reproduced on the display unit 103 of the digital camera 100. Further, the TV 200 may be provided with an image capturing unit for capturing the face image of the viewer.

  In the above-described embodiment, since the face image is synthesized in units of frames of the moving image, it is possible to animate the face image by shifting the synthesis position of the face image for each frame. .

  Furthermore, in the above-described embodiment, the face image is synthesized with the moving image recorded in the digital camera 100 and reproduced on the TV 200. However, the present embodiment is not limited to this, and the method of the present embodiment has various video shooting functions. It can be applied to other portable devices.

  Although the description has been made on the assumption that the moving image is changed here, a face image obtained during playback or an image such as an avatar based on the face image information is synthesized while displaying a still image on the TV 200. Needless to say, it can be used for various purposes. As a result, when viewing a lot of still images, it is possible for other people to know which image was popular on the spot.

  Although the present invention has been described above based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications and applications are naturally possible within the scope of the gist of the present invention.

  Further, the above-described embodiments include various stages of the invention, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some configuration requirements are deleted from all the configuration requirements shown in the embodiment, the above-described problem can be solved, and this configuration requirement is deleted when the above-described effects can be obtained. The configuration can also be extracted as an invention.

  DESCRIPTION OF SYMBOLS 100 ... Digital camera, 101 ... Imaging part, 102 ... Face detection part, 103 ... Display part, 104 ... Sound collection part, 105 ... Recording part, 106 ... Clock part, 107 ... Control part, 108 ... Operation part, 109 ... Communication Interface unit 200 ... Television receiver (TV), 200 ... TV, 201 ... Broadcast receiving unit, 202 ... Display unit, 203 ... Network communication unit, 204 ... Information superimposing unit, 205 ... Control unit, 206 ... Remote control signal reception 207 ... Communication interface unit 300 ... Communication cable 400 ... Remote controller (remote controller)

Claims (6)

An imaging unit for acquiring a moving image;
A recording unit for recording moving images;
A communication interface unit for transmitting a moving image recorded in the recording unit to an external device having a display unit;
A face detection unit that detects a face image in a moving image obtained by operating the imaging unit when the viewer is watching the moving image reproduced on the display unit of the external device;
A determination unit that determines whether or not the expression of the face image detected by the face detection unit has changed;
A synthesizing unit that synthesizes the face image with the moving image transmitted to the external device;
Comprising
The imaging unit captures an image at a timing when a signal indicating a face image synthesis command is received from the external device,
The digital camera characterized in that the synthesizing unit synthesizes the transmitted moving image and the face image at the timing of the change in facial expression detected by the face detection unit.   The digital camera according to claim 1, wherein the change timing of the expression of the face image determined by the determination unit is a timing when the expression changes to a smile. An index information adding unit that gives a timing at which a change in facial expression of the face image occurs as index information to the transmitted moving image when the determination unit determines that the facial image has changed in facial expression; ,
The digital camera according to claim 1, wherein when index information is given to the moving image, a representative image of the moving image is transmitted to the external device by the communication interface unit.   The digital camera according to claim 1, wherein the synthesizing unit synthesizes the face image with a portion having no contrast or a portion having no time change in the moving image transmitted to the external device.   5. The digital camera according to claim 4, wherein, when a plurality of faces are detected by the face detection unit, the synthesis unit synthesizes only a face image whose expression changes to a smile. Capture a moving image at the timing of receiving a signal indicating a face image synthesis command from an external device,
Send the captured moving image to an external device having a display unit,
When the viewer is watching the moving image reproduced on the display unit of the external device, the face image in the moving image obtained by capturing is detected,
Determine whether the expression of the detected face image has changed,
A camera characterized in that, when it is determined that there is a change in facial expression of the face image, the moving image transmitted to the external device and the face image are synthesized at a timing when the facial expression of the face image has changed. Composite image display method.