JP2010279012A - Av system, program, recording device, and video recording method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a new technique allowing one's own action carried out while viewing a content during playback to be confirmed while the action is easily compared with a video of the content thereafter. <P>SOLUTION: An AV system includes: an image capturing apparatus 1 for output of a video signal; and a recording device 2 for recording a first video signal related to a first video obtained by the image capturing apparatus so that the first video can be played back as a video combined with or continued to a second video related to a second video signal later. The first video signal is acquired simultaneously with video display based on the second video signal. The temporal relationship between the first video part and the second video part in the combined video or the continued video is set identical to the temporal relationship between the first video and the second video in acquiring the first video. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an AV system having an imaging device, a program for causing a computer to function as a recording device in the system, a recording device suitable for use in the system, and a video recording method suitable for the system.

  Conventionally, for example, a videophone system is known as a method of projecting itself on a display device such as a television (see, for example, Patent Document 1). In this system, each speaker has a television camera and a display, and image information captured by the television camera can be displayed on the display of the other party. At that time, it is possible to control the shooting direction of its own television camera and the shooting direction of the other television camera.

  In addition, a game device is described that can replay an image of a virtual camera arranged in a game space in response to an event in the game space (see, for example, Patent Document 2). In the replay function of this apparatus, the player's operation input data and the position / posture data of the character in the game are stored as play data, and the game play is reproduced based on the stored play data.

  In addition, a technique for collecting and distributing replay data recorded in each game device is known (see, for example, Patent Document 3). In this technique, replay data is exchanged via a server on a network, and a replay video can be reproduced by the downloaded replay data.

JP-A-8-111863 JP 2007-31001 A JP 2006-6853 A

  On the other hand, there is a case where the user wants to see his / her own video that is operating while viewing the video content being played back by comparing with the operation of the video content later. For example, while watching a instructional video such as dance or yoga, after trying the same operation as the instructional video, compare your own operation with the operation of the instructional video, It would be advantageous if you could find out the correct parts and use them to improve dance and yoga. Also, in the case of a video game that moves the body, it is also possible to record the progress of the game screen and your movement at the same time, and then play back the record and compare both movements to help improve the game. It is convenient if possible.

  However, the above-described videophone system and TV camera are known as devices that project themselves. However, these devices categorize video content that they are viewing and their actions while viewing them. It does not make it possible to easily compare and study. In addition, there is no known simple apparatus that makes it possible to easily perform such comparative studies.

  Moreover, in the above-mentioned game device, the video image of the user who is playing the game cannot be replayed together. That is, the appearance of the user who is playing the game cannot be displayed on the screen of the television or the like together with the replay video based on the play data after the play is over. Therefore, even if you intend to do the same action as the instructional game while playing the instructional game such as dance or yoga or the battle game with a distant friend, to verify the action after that In addition, you cannot compare your actions with the actions of the instructional game or your friends.

  Therefore, an object of the present invention is to provide a new technique that enables the user to confirm his / her own operation while viewing the content being played back while easily comparing it with the video of the content.

  In order to achieve this object, a recording apparatus according to a first aspect of the present invention provides a first video signal related to a first video supplied from an imaging device outside the apparatus, and a second video related to a second video signal. And the first video signal is acquired simultaneously with the video display based on the second video signal, and the synthesized video is recorded so that it can be reproduced later as a synthesized or continuous video. Alternatively, the temporal relationship between the first video portion and the second video portion in the continuous video is the same as the temporal relationship between the first video and the second video when the first video is acquired. To do.

  According to a second aspect of the present invention, there is provided a recording apparatus according to the first aspect, wherein the recording of the first video signal is performed as recording of a signal synthesized with the second video signal.

  A recording device according to a third invention is the recording device according to the first invention, wherein the second video signal is based on a file recorded in a recording means in the recording device, and the recording of the first video signal is It is characterized by being recorded as a file separate from the file.

  According to a fourth aspect of the present invention, there is provided a recording apparatus according to the first aspect, wherein the second video signal is supplied from the outside of the recording apparatus, and the recording of the first video signal is performed on the second video signal. It is characterized by being performed simultaneously with recording in the recording apparatus and as a separate file.

  According to a fifth aspect of the present invention, there is provided a recording apparatus according to the third or fourth aspect of the present invention, wherein the reproduction of the video data relating to the corresponding second video signal is performed so that the subsequent reproduction can be performed when the first video signal is recorded. A starting address is recorded.

  According to a sixth aspect of the present invention, there is provided a recording apparatus according to the first aspect, wherein the recording apparatus displays an image based on predetermined data in response to a user operation input, and the second image signal is the display image. The first video signal is recorded simultaneously with the recording of the second video signal in the recording device and as a separate file.

  A recording device according to a seventh invention is the recording device according to the first invention, wherein the recording device displays video based on predetermined data in response to a user's operation input and records replay data for replay playback. The second video signal is a video signal of the display video, and the recording of the first video signal is performed simultaneously with the recording of the replay data file corresponding to the second video signal in the recording device. It is characterized by that.

  According to an eighth aspect of the present invention, in the sixth or seventh aspect, the recording of the one video signal is terminated when a predetermined event ends.

  According to a ninth aspect of the present invention, in any one of the sixth to eighth aspects, the recording of the one video signal is started when a predetermined event is started.

  According to a tenth aspect of the present invention, there is provided the recording apparatus according to any one of the third to eighth aspects, wherein the recording apparatus reproduces the file based on the first video signal file or the corresponding file related to the second video signal. When an instruction to perform is given, a video signal based on both files is output as a synthesized signal or as a separate signal so that the synthesized video or continuous video can be displayed. And

  According to an eleventh aspect of the present invention, there is provided a recording apparatus according to the tenth aspect, further comprising an AV device that displays the synthesized video or the continuous video based on a synthesized signal or a separate signal output from the recording device. .

  A recording apparatus according to a twelfth aspect of the present invention is the recording apparatus according to any one of the first to eleventh aspects, wherein the recording apparatus includes an input unit that accepts a setting related to the composite video or continuous video, and the setting includes: The setting includes whether to mirror-invert the portion of the first video or the second video.

  In a recording apparatus according to a thirteenth aspect based on the twelfth aspect, the setting includes setting whether the synthesized video is a picture-in-picture or a two-screen display, or the setting in the synthesized video. The setting of the aspect ratio of one video portion or the second video portion is included.

  According to a fourteenth aspect of the present invention, in the recording apparatus according to any one of the first to thirteenth aspects, the first video signal and the second video signal are accompanied by an audio signal, and the composite video or the continuous video is reproduced. In this case, the designated audio signal is reproduced from the accompanying audio signals.

  A recording apparatus according to a fifteenth aspect is the recording apparatus according to any one of the first to fourteenth aspects, wherein the recording of the first video signal is started on the condition that the imaging apparatus recognizes a user. To do.

  A program according to a sixteenth invention causes a computer to function as the recording device according to any one of the first to fifteenth inventions.

  An AV system according to a seventeenth aspect of the present invention is directed to an imaging device that outputs a video signal, a first video signal related to a first video obtained by the imaging device, and a second video related to a second video signal. A recording device for recording the composite video or a continuous video so that it can be reproduced later, and the first video signal is acquired simultaneously with the video display based on the second video signal, Alternatively, the temporal relationship between the first video portion and the second video portion in the continuous video is the same as the temporal relationship between the first video and the second video when the first video is acquired. To do.

  According to an eighteenth aspect of the present invention, there is provided an image recording method for outputting a video signal, a first video signal related to a first video obtained by the imaging step, a first video signal, and a second video signal. A recording step of recording the second video according to the second video signal so that it can be reproduced later as a synthesized or continuous video, and the first video signal is acquired simultaneously with the video display based on the second video signal And the temporal relationship between the first video portion and the second video portion in the composite video or the continuous video is the temporal relationship between the first video and the second video at the time of obtaining the first video. It is characterized by being the same.

  According to the present invention, it is possible to provide a new technique that allows the user to confirm his / her own operation while viewing the content being played back while easily comparing it with the video of the content.

It is a figure which shows the AV system which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of the imaging device in the system of FIG. It is a block diagram which shows the structure of the recording device in the system of FIG. 4 is a flowchart showing recording processing in the recording apparatus of FIG. 3. It is a flowchart which shows the detail of the one part process of FIG. It is a block diagram which shows the reproduction | regeneration processing in the recording device of FIG. It is a flowchart which shows the reproduction | regeneration processing corresponding to the file recorded in the reproduction | regeneration processing of FIG. 4 is a flowchart showing another recording process in the recording apparatus of FIG. 3. It is a figure which shows an example of the image | video reproduced | regenerated based on the file recorded by the process of FIG.3 and FIG.4. It is a figure which shows an example of the image | video reproduced | regenerated by the reproduction | regeneration processing of FIG. It is a figure which shows the AV system which concerns on other embodiment of this invention. It is a block diagram which shows the structure of the game device in the system of FIG. It is a flowchart which shows the recording process in the game device of the system of FIG. 12 is a flowchart showing another recording process in the game apparatus of the system of FIG. 12 is a flowchart showing still another recording process in the game apparatus of the system of FIG. It is a flowchart which shows the reproduction | regeneration processing in the game device of the system of FIG.

  FIG. 1 shows an AV system according to an embodiment of the present invention. As shown in the figure, this system includes an imaging device 1 used for shooting, a recorder device 2 connected to the imaging device 1, and an AV device 3 connected to the recorder device 2. The recorder device 2 has a function of recording a video signal and an audio signal of a photographed video supplied from the imaging device 1 and outputting a recorded video signal and an audio signal and a reproduction signal of other content to the AV device 3. . The AV apparatus 3 has a function of performing video display and audio output based on the video signal and audio signal from the recorder apparatus 2 and the content reproduction signal. For example, a video camera corresponds to the imaging device 1. Examples of the AV apparatus 3 include a television set and an AV personal computer. The content for which a playback signal is sent from the recorder device 2 to the AV device 3 includes broadcast content provided by wireless or wired broadcasting, content distributed via a network, content recorded on a DVD, and the like. .

  In FIG. 1, a situation is shown in which the user 4 is shooting with the imaging device 1 while watching the content being played back on the AV device 3 and exercising in accordance with the video and audio of the content. Yes. The recorder device 2 records the video signal and the audio signal sent from the image pickup device 1 along with the photographing while synthesizing or associating with the data of the reproduction content.

  FIG. 2 is a block diagram illustrating a configuration of the imaging apparatus 1. As shown in FIG. 1, the imaging apparatus 1 forms an image based on light from an object existing in the imaging range of the apparatus, converts the image into a video signal, and sounds around the apparatus as audio. Microphone unit 12 for converting to a signal, video signal input unit 13 for receiving an input of a video signal from an external source such as a television or a game machine, an audio signal input unit 14 for receiving an input of an audio signal from an external source, various operation inputs Based on an operation input unit 15 constituting a user interface for receiving, a recording unit 16 for storing recording data, a control unit 17 for controlling each unit of the apparatus, a display unit 18 for performing various displays, and an audio signal being recorded or reproduced. A speaker unit 19 for outputting audio, a video signal output unit 20 for outputting a video signal being recorded or reproduced to the outside, and an audio signal for outputting an audio signal being recorded or reproduced to the outside. An output unit 21.

  The various operation inputs accepted by the operation input unit 15 include, for example, shooting by the imaging unit 11, recording to the recording unit 16 based on signals from the video signal input unit 13 and the audio signal input unit 14, and recording contents in the recording unit 16. Based on the video signal output unit 20 and the audio signal output unit 21, the video signal output unit 20 without recording the video signal from the imaging unit 11 and the audio signal from the microphone unit 12 in the recording unit 16. Instructions relating to the process of outputting from the audio signal output unit 21, the process of reproducing recorded content in the recording unit 16, the editing process, and the like are applicable. Examples of signals recorded in the recording unit 16 include signals from the imaging unit 11, the microphone unit 12, the video signal input unit 13, and the audio signal input unit 14, and these signals based on instructions from the operation input unit 15. This is what is synthesized. Typically, a hard disk, a memory card, or the like can be used as the recording unit 16, but a disk medium such as a DVD or a BD (Blu-ray disc) may be used. The control unit 17 performs a recording process to the recording unit 16, a reproduction process based on the recording content of the recording unit 16, and various setting processes related to the shooting mode, according to the operation input from the operation input unit 15. The display unit 18 performs various types of display such as display of a playback video or video during recording, display of a setting screen, and the like in response to a command from the control unit 17 based on an operation input. The speaker unit 19 outputs reproduced sound and the like.

  FIG. 3 is a block diagram showing the configuration of the recorder device 2. As shown in the figure, the recorder device 2 includes a tuner unit 31 that performs broadcast wave reception processing via an antenna (not shown), a decoder 32 that decodes a received signal from the tuner unit 31, the imaging device 1 and the game device. A video signal input unit 33 and an audio signal input unit 34 for receiving input of video signals and audio signals from external devices, etc., an operation input unit 35 for receiving various operation inputs of a user, and recording and reproduction of various content data. 1 for recording / reproducing unit 36, control unit 37 for controlling each part of the apparatus, second recording / reproducing unit 38 for recording and reproducing various content data, and for outputting the reproduced audio signal and reproduced video signal of the content to the outside. Audio signal output unit 39 and video signal output unit 40. The imaging apparatus 1 is connected to the video signal input unit 33 and the audio signal input unit 34, and the video signal from the imaging unit 11 and the audio signal from the microphone unit 12 are input. The audio signal output unit 39 and the video signal output unit 40 are connected to the AV apparatus 3, and the AV apparatus 3 performs video display and audio output based on signals output from the audio signal output unit 39 and the video signal output unit 40. Is to be done.

  The decoder 32 also decodes the content data recorded in the first recording / playback unit 36 and the second recording unit 38. The control unit 37 accepts various instructions via the operation input unit 35 and performs necessary control and processing. As an instruction to accept, for example, recording reservation of a broadcast program, display of an EPG (electronic program guide), dubbing processing between the first recording / reproducing unit 36 and the second recording / reproducing unit 38, a video signal input unit 33 and an audio signal Recording process to the first or second recording / reproducing unit 36 or 38 for the video signal and the audio signal input through the input unit 34, and recorded in the first or second recording / reproducing unit 36 or 38. Instructions related to data reproduction processing and the like are applicable. The first recording / reproducing unit 36 records video signals and audio signals supplied via the video signal input unit 33 and the audio signal input unit 34, content data recorded in the second recording / reproducing unit 38, and the like. The recorded data is reproduced. Typically, the first recording / reproducing unit 36 corresponds to a hard disk device. The second recording / playback unit 38 records the content data supplied via the video signal input unit 33 and the audio signal input unit 34, the content data recorded in the first recording / playback unit 36, and the like. Play back the content data. Typically, the second recording / reproducing unit 38 corresponds to a disk medium such as a DVD or a BD (Blu-ray disc), a flash memory such as an SD card, or the like.

  FIG. 4 is a flowchart showing a recording process in the recorder device 2. This process is performed by the control unit 37 according to a predetermined program, and is started based on a predetermined recording instruction by a user operation input. In this recording process, a video signal and an audio signal from the imaging device 1 are recorded.

  When the recording process is started, the control unit 37 first, in step S41, a reproduced video signal from the first recording / reproducing unit 36 or the second recording / reproducing unit 38, or a video signal based on a broadcast wave (hereinafter referred to as these signals). Is called an “internal video signal”) and a video signal from the imaging apparatus 1 is simultaneously determined in the recording instruction. If it is determined that the request has been made, in step S42, it is determined whether or not the internal video signal includes a copy prohibition (copy protect) signal indicating that copying is prohibited. If it is determined that they are not included, in step S43, based on the video signal and audio signal from the video signal input unit 33 and the audio signal input unit 34, and the internal video signal and the audio signal associated therewith, these Recording is performed on the first or second recording / reproducing unit 36 or 38 while synthesizing the signals. Thereby, the process of FIG. 4 is completed. Note that the internal video signal to be synthesized and the accompanying audio signal relate to the content specified by the instruction included in the recording instruction.

  On the other hand, if it is determined in step S41 that there is no request for simultaneous recording, the process proceeds to step S44, and normal recording based on the video signal and audio signal from the video signal input unit 33 and audio signal input unit 34 is performed. 4 is terminated. If it is determined in step S42 that a copy prohibition signal is included, the process proceeds to step S45, a display indicating that copy is prohibited and a sound output are performed, and the processing in FIG. 4 is terminated.

  FIG. 5 is a flowchart showing the process of step S43 of FIG. In this process, in accordance with a synthesis method set by the user, the video signal and audio signal from the video signal input unit 33 and the audio signal input unit 34 are synthesized with the internal video signal and the audio signal associated therewith to generate one moving image. Record as a file.

  That is, when the process is started, first, in step S51, the setting of the synthesis method by the user's operation input is accepted. In this setting, it is set how to synthesize the signals from the video signal input unit 33 and the audio signal input unit 34 with the internal video signal and the audio signal associated therewith. The setting items include, for example, settings relating to the arrangement of the video by the video signal from the video signal input unit 33 and the video by the internal video signal, and the video signal from the video signal input unit 33 (video signal from the imaging unit 11) in a mirror. Setting whether or not to perform image conversion processing (hereinafter referred to as inversion processing or mirror image conversion processing), and recording of audio signals from the audio signal input unit 34 and audio signals accompanying the internal video signal The setting relating to the audio signal to be recorded and the recording form thereof corresponds.

  Examples of video layout settings include screen layout format settings corresponding to conventional display formats such as picture-in-picture and two-screen display, and which video is assigned to which screen in that case. This corresponds to the setting of the aspect ratio of each screen. Examples of settings relating to the audio signal to be recorded and the recording format thereof include, for example, whether to record only one or both of the main and sub audio signals accompanying the internal video signal, and the audio signal from the audio signal input unit 34 This includes settings relating to whether or not to perform recording, whether to record each audio signal to be recorded as main / sub audio, and which audio signal to record as an audio signal of which channel. For each setting item, when the setting by the user is not made, the default setting or the previous setting is adopted.

  Next, in step S52, it is determined whether it is necessary to change the aspect ratio of the video signal from the video signal input unit 33 and / or the internal video signal so as to match the set synthesis method. If it is determined that the change is required, the aspect ratio is changed in step S53, display based on the video signal and / or the internal video signal from the changed video signal input unit 33 is performed, and the process proceeds to step S54. If it is determined that it is not necessary to change the aspect ratio, the process directly proceeds to step S54.

  In step S54, it is determined whether or not to invert the video signal from the video signal input unit 33 so as to match the set synthesis method. When it is determined that the inversion process is to be performed, in step S55, the inversion process is started for the video signal from the video signal input unit 33, the video based on the inverted video signal is displayed, and the process proceeds to step S56. If it is determined that the inversion process is not performed, the process proceeds to step S56 as it is.

  In step S56, the video signal and the internal video signal from the video signal input unit 33 are displayed based on a signal synthesized by a set synthesis method, and the synthesized signal is received after a recording instruction from the user. In accordance with the setting contents related to the audio, the process of recording as one file together with the audio signal from the audio signal input unit 34 and / or the audio signal accompanying the internal video signal is started, and the recording ends according to a user instruction. Thereby, the process of FIG. 5 is completed. During the recording of the composite signal, the internal video signal and the accompanying audio signal are output to the AV device 3, and the AV device 3 displays video and outputs audio based on these signals.

  Note that for the synthesis of the video signal, for example, a predetermined memory area is used, and for each video frame at each time point of both video signals, the memory area is written based on the data of both frames in accordance with a set synthesis method. It can be performed by reading out from the memory area as a video signal of one frame.

  FIG. 9 shows an example of a video that is played back based on a file that is synthesized and recorded by the processes of FIGS. 4 and 5. In the figure, 91 is a video portion corresponding to the video signal from the video signal input unit 33, and 92 is a video portion corresponding to the internal video signal. That is, in FIG. 9, the playback is performed for the file in which the screen layout format is set to a format corresponding to the two-screen display and the video signal from the video signal input unit 33 is set to be mirror-inverted and recorded. The picture is shown. The video portion 91 is subjected to trimming processing and enlargement processing at the time of synthesis, and is mirror-inverted. The video portion 92 is reduced according to the default aspect ratio. The video portion 91 allows the user to view his / her video in the same direction as when he / she is seen in the mirror.

  According to the present embodiment, the video signal and audio signal (the signal from the imaging device 1) from the video signal input unit 33 and the audio signal input unit 34, the internal video signal generated in the recorder device 2 and the accompanying video signal and audio signal. Since the audio signal can be synthesized and recorded, along with the contents of the video and audio displayed and output by the AV apparatus 3 based on the internal video signal supplied from the recorder device 2 and the audio signal associated therewith, The situation where the user is viewing it can be recorded together. For example, a video or audio of a user who is performing an action corresponding to a dance, yoga or game content presented by the AV device 3 is synthesized with a video signal (internal video signal) and an audio signal of the content. Can be recorded. Therefore, the user can confirm his / her own operation while viewing the content being reproduced afterwards while easily comparing it with the video of the content.

  FIG. 6 is a flowchart showing a reproduction process in the recorder device 2 as another example in which the recorder device 2 records a signal from the imaging device 1. This process is performed by the control unit 37 according to a predetermined program. 4 and 5, the signal from the imaging device 1, the internal video signal, and the audio signal associated therewith are synthesized and recorded, whereas in the process of FIG. Only the signal from the imaging device 1 is recorded. At the time of the recording, a reproduction signal of the content instructed by the user is supplied to the AV apparatus 3 and used for video display and audio output in the AV apparatus 3. The content file is a corresponding file corresponding to a file newly generated by the recording, and corresponding file specifying information for specifying the file is recorded as accompanying information for the newly generated file.

  The reproduction process in FIG. 6 is started in response to a predetermined reproduction start instruction from the user. This reproduction start instruction includes designation of a content to be reproduced among the contents recorded in the first recording / reproducing unit 36 or the second recording / reproducing unit 38. When the reproduction process is started, first, in step S61, a request to record the video imaged by the imaging apparatus 1 and the audio signal associated therewith in association with the reproduction of the content to be reproduced is associated with the content. Detect if it exists. This detection can be performed, for example, by checking whether the request is included in the above-described reproduction start instruction.

  If it is detected that the request exists, the process waits for a recording start instruction from the user in step S62, and in response to the instruction, in step S63, the reproduction of the content to be reproduced and the video signal input unit 33 and recording of a video signal and an audio signal input from the imaging apparatus 1 via the audio signal input unit 34 are started. This recording is performed on the first recording / reproducing unit 36 or the second recording / reproducing unit 38 different from the first recording / reproducing unit 36 or the second recording / reproducing unit 38 in which the content is recorded. For example, when the content to be reproduced is recorded on the BD or SD card of the second recording / reproducing unit 38, this recording is performed on the hard disk, which is the first recording / reproducing unit 36, and conversely If it is recorded on the hard disk, it is performed on the BD or SD card.

  Next, in step S64, in response to the recording end request from the user, the recording of the signal from the imaging device 1 is ended, and the file of the content to be reproduced is associated with the file generated by the recording. Corresponding file specifying information specified as a file is recorded. The corresponding file specifying information includes information such as the file name and address position of the corresponding file. Thereby, the process of FIG. 6 is completed.

  On the other hand, if it is detected in step S61 that there is no request to record in association with the content to be played back, the content of the content to be played back is played in step S65 as in the case of normal content playback. Only reproduction is performed, and the processing of FIG. 6 is terminated. In this example, the corresponding file specifying information is recorded as information accompanying the file, but instead, it may be recorded in a table or the like separate from the file.

  FIG. 7 is a flowchart showing another example of the reproduction process in the recorder device 2. This reproduction process is performed by the control unit 37 according to a predetermined program. In this reproduction process, it is assumed that the file recorded by the process of FIG. 6 can exist in the first recording / reproducing unit 36 or the second recording / reproducing unit 38. This reproduction process is started in response to a reproduction request from the user who specified the file to be reproduced.

  When the reproduction process is started, first, in step S71, it is determined whether or not the corresponding file specifying information is recorded as accompanying information in the file to be reproduced. If it is determined that the file is recorded, in step S72, it is determined whether or not the file specified by the corresponding file specifying information actually exists in the first recording / playback unit 36 or the second recording / playback unit 38. judge.

  As a case where the corresponding file does not exist, for example, the corresponding file exists on the recording medium such as the SD card set in the recorder device 2 at the time of recording the file to be played back. This corresponds to the case where the medium does not exist in the recorder device 2 or the case where the corresponding file has already been deleted.

  If it is determined in step S72 that the corresponding file exists, it is determined in step S73 whether there is a user instruction for simultaneously reproducing the corresponding file. If it is determined that the instruction has been given, in step S74, the setting of the synthesis method for simultaneously reproducing the file to be reproduced and the corresponding file is accepted. The setting items include, for example, settings related to the display position of video based on each file, whether to perform inversion processing (mirror or mirror image conversion processing) on the video signal based on the file to be played back, and audio of each file The setting related to which of the signals is output corresponds.

  The settings related to the display position of the video include, for example, the conventional display format settings such as picture-in-picture and two-screen display, the setting of which video is assigned to which screen in that case, the aspect of each screen The ratio setting is applicable. The settings relating to the audio signal to be output include, for example, whether to output only or one of the main and sub audio signals in the corresponding file, or whether to output the audio signal of the file to be played back. The setting is applicable. For each setting item, when the setting by the user is not made, the default setting or the previous setting is adopted.

  Next, in step S75, it is determined based on the setting contents in step S74 whether or not to perform video inversion processing (mirror or mirror image conversion processing) based on the file to be reproduced. If it is determined that the inversion process is to be performed, in step S76, the video based on the file to be reproduced is set to be inverted, and the process proceeds to step S77. If it is determined that the inversion process is not performed, the process proceeds to step S77 as it is.

  In step S77, based on the file to be reproduced and the corresponding file, the synthesized video signal and the audio signal to be output are output via the video signal output unit 40 and the audio signal output unit 39 according to the settings in step S74. Output to. Thereby, the process of FIG. 7 is completed. The video signal and audio signal output to the AV device 3 are used for video and audio output in the AV output device 3.

  The synthesis of the video signal based on the file to be reproduced and the corresponding file in step S77 is performed using, for example, a predetermined memory area and according to a synthesis method set for each video frame at each time point of both video signals. This can be done by writing to the memory area based on the frame data and reading out from the memory area as a video signal of one frame. At this time, since it is necessary to synchronize the timing of both video signals, when recording the corresponding file specifying information in the file to be reproduced (step S64 in FIG. 6), information for adjusting the timing in the corresponding file specifying information The timing is adjusted based on this information. As information for matching the timing, for example, a time stamp attached to the video frame of the corresponding file corresponding to the first video frame of the video signal of the file to be reproduced can be used. In synthesizing both video signals, the timing of both video signals is determined by associating the video frame of the corresponding file at the time indicated by the time stamp with the first video frame of the file to be reproduced, and starting synthesis. Can be combined.

  On the other hand, if it is determined in step S71 that the corresponding file specifying information is not recorded, and if it is determined in step S73 that there is no user instruction for simultaneously reproducing the corresponding file, the process proceeds to step S79. If it is determined in step S72 that the corresponding file does not exist, the fact is displayed in step S78, and the process proceeds to step S79. In step S79, the video signal and the audio signal are output to the AV output device 3 via the video signal output unit 40 and the audio signal output unit 39 based on the file to be reproduced. The output video signal and audio signal are used for video and audio output in the AV output device 3. In this case, only the result captured by the imaging device 1 is presented in the AV output device 3. Thereby, the process of FIG. 7 is completed. If it is determined in step S73 that there is no user instruction to play back the corresponding file at the same time, the corresponding file may be played back after the playback of the file to be played back ends.

  In this reproduction process, each video signal based on the file to be reproduced and the corresponding file is synthesized and supplied to the AV apparatus 3. In the case of being compatible with picture display or two-screen display, each video signal is supplied to the AV apparatus 3 as it is without being synthesized, and the AV apparatus 3 displays the synthesized video according to the synthesis method by the user. You may make it perform according to a setting. However, also in this case, it is necessary to supply each video signal in accordance with the timing based on the information for adjusting the timing as described above. In this example, the corresponding file specifying information is recorded as information accompanying the file, but instead, it may be recorded in a table or the like separate from the file.

  FIG. 8 is a flowchart showing still another recording process in the recorder device 2. This recording process is performed by the control unit 37 according to a predetermined program. In this recording process, broadcast content or content supplied from outside the imaging device 1 is recorded in one of the first recording / playback unit 36 or the second recording / playback unit 38, and on the other hand, the imaging device 1. Video signals and audio signals are recorded. This recording process is started based on a predetermined recording instruction by a user operation input.

  When the recording process is started, first, in step S81, the control unit 37 determines whether there is a request to associate and record the video signal and the audio signal from the imaging apparatus 1 in conjunction with the recording of the broadcast or external content. To detect. This detection can be performed, for example, by checking whether the request is included in the above-described recording start instruction. If it is detected that the request exists, in step S82, the first recording / reproducing unit 36 or the second recording / reproducing unit records the broadcast or external contents and the video signal and the audio signal from the imaging device 1 respectively. The user instruction regarding which of the sections 38 is used is accepted. For example, the video signal and the audio signal from the imaging device 1 are performed on the hard disk as the first recording / reproducing unit 36, and the recording of the broadcast or external content is performed on the SD memory in the second recording / reproducing unit 38. Accept.

  Next, in step S83, recording of the broadcast or external content and the video signal and the audio signal from the imaging device 1 is started in the designated first recording / reproducing unit 36 or second recording / reproducing unit 38. At the same time, presentation of playback video and audio of broadcast or external content is started in the AV apparatus 3. Next, in step S84, in response to a recording end request from the user, the recording to the first recording / reproducing unit 36 and the second recording / reproducing unit 38 is terminated, and for each file generated by each recording, Corresponding file identification information for identifying the file as a corresponding file is recorded as accompanying information. Thereby, the process of FIG. 8 is completed. The corresponding file specifying information includes information such as the file name and storage location of the corresponding file. Further, if necessary for reproducing both files in association with each other, information for adjusting the above timing may be included. Each file can be reproduced in the same manner as in FIG. When each file is recorded, the time stamp can be used for synchronizing each video frame by reproducing the time stamp for each video frame of each file.

  On the other hand, if it is detected in step S81 that there is no request to record in association, only the video signal and the audio signal from the imaging device 1 are recorded in step S85, and the processing in FIG. 8 ends. In this example, the corresponding file specifying information is recorded as information accompanying the file, but instead, it may be recorded in a table or the like separate from the file.

  FIG. 10 shows an example of a video recorded by the processing of FIG. 6 or FIG. 8 and played back by the playback processing of FIG. Reference numeral 101 in the figure denotes a video portion based on a file that is captured by the imaging apparatus 1 and recorded by the recorder apparatus 2 and is a reproduction target in the processing of FIG. 7, and 102 is a file corresponding to the file, that is, the imaging apparatus. 1 is a video portion based on a file of content presented in the AV apparatus 3 at the time of shooting by 1. FIG. 10 shows a reproduced video when the display format is set to two-screen display and the video signal based on the file to be reproduced is set to be inverted. Video portions 101 and 102 are reduced according to the default aspect ratio. Since the video portion 101 is mirror-inverted, the user can see his / her video in the video portion 101 in the same direction as when he / she is looking at himself / herself reflected in the mirror.

  According to the embodiment of FIGS. 6 to 8, the video signal and the audio signal from the imaging device 1 are associated with the content already recorded in the recorder device 2 or the content (including broadcast content) supplied from the outside. Since it is recorded in the device 2 and can be presented in combination with the related content when the recording is played back, the state of the user who operates while viewing the related content with the AV device 3 is recorded in the recorder device 2 At the same time, the recording and the related content that was viewed can be combined and played back to check the state of the operation. For example, a video or audio of a user who is performing an action corresponding to a content of a dance, yoga, or game while watching the content of the AV device 3 is recorded in association with the data of the content, and then the composition method is set. It can be combined with content and played back. Therefore, the user can confirm his / her own operation while viewing the content being reproduced afterwards while easily comparing it with the video of the content.

  FIG. 11 shows an AV system according to another embodiment of the present invention. As shown in the figure, this system includes an imaging device 41 used for shooting, a game device 42 connected to the imaging device 41, and an AV device 43 connected to the game device 42. In addition to the game function inherent to the game device, the game device 42 records the video signal and audio signal of the captured video supplied from the imaging device 41 as user record information, and also records the recorded user record information and other content playback signals. Is output to the AV apparatus 43. The game device 42 can be connected to a game device 45 of another user via the Internet or a dedicated line 44. The imaging device 41 and the AV device 43 are the same as the imaging device 1 and the AV device 3 in the system of FIG. The imaging device 41 also includes a known face recognition means, a human sensor, and the like. When the user 46 is present within a predetermined shooting range of the imaging device 41, the imaging device 41 has a function of detecting that and notifying the game device 42. Prepare.

  In FIG. 11, a state in which the user 46 is playing a game related to a game function executed by the game device 42 based on the game content and provided via the screen of the AV device 43 is photographed by the imaging device 41. The situation is shown. The game device 42 records video signals and audio signals sent from the image pickup device 41 along with shooting as user record information.

  FIG. 12 is a block diagram showing a configuration of the game apparatus 42. As shown in the figure, the game device 42 includes a game recording medium 51 on which game data is recorded, a video signal input unit 52 and an audio signal input unit 53 that receive input of a video signal and an audio signal from the imaging device 41, respectively. Video signal output unit 54 and audio signal output unit 55 for outputting video signals and audio signals of content to the AV device 43 and the like, a first recording unit 56 for recording game record information such as the degree of progress of the game, and video A second recording unit 57 that records video and audio signals input via the signal input unit 52 and the audio signal input unit 53 as user record information, a control unit 58 that controls each unit of the apparatus, and a first recording unit 56 and the second recording unit 57, a reproduction unit 59 that performs reproduction based on the data recorded in the second recording unit 57, an operation input unit 60 that accepts various operation inputs by the user, an Internet DOO or via a dedicated line 44 and a communication unit 61 for enabling connection with the game apparatus 45 of another user.

  The video signal input unit 52 and the audio signal input unit 53 accept an input of a video signal and an audio signal regarding the operation of the user 46 to be photographed as a video signal and an audio signal from the imaging device 41. The operation input unit 60 constitutes a user interface and receives input of various operations from the user. Typically, it is a game controller, but it may be a TV remote control. Specifically, instructions related to various functions used in game play, or various functions of game machines, such as a playback function of a recording medium such as a DVD, or a playback function of a photograph, a movie, or a song recorded on a USB memory or an SD card It is used to perform. Further, in the present embodiment, in order to record the video of the user 46 who is playing the game, to reproduce the recorded video by a predetermined operation, or to reproduce it according to the replay image of the game. Used for. Based on the game data recorded on the game recording medium 51, the reproduction unit 59 reproduces the game screen according to the user instruction and the progress of the game event. Further, in accordance with a user instruction, reproduction based on a file of user record information recorded in the second recording unit 57 is performed.

  Game data distributed as game software is recorded on the game recording medium 51. The video signal output unit 54 and the audio signal output unit 55 send the video data and audio signal based on the game data read from the game recording medium 51 and the file recorded in the second recording unit 57 to the AV device 43. Output. The audio signal may be output to a speaker device or the like separate from the AV device 43. The control unit 58 controls each unit in accordance with a user operation or the like, and records the above-described game recording information and user recording information in the first recording unit 56 and the second recording unit 57, and playback based on these recordings. Or display a game screen and perform necessary processing as game processing in accordance with game data of the game recording medium 51 and user operation. The communication unit 61 includes a communication module for connecting to a wired LAN, a wireless LAN, Bluetooth, a mobile phone network, and the like, and records games with other users' game devices 45 through the Internet or a dedicated line 44. Information and user record information are sent and received.

  The game record information is information for reproducing the play contents of the game after the game is played. As game record information, replay data and video and audio data can be considered. The replay data is data that is recorded in order to completely reproduce the game play contents later. The replay data includes, for example, data on the change amount of the parameter data of the character in the game, the progress of the game event, and the like. The video and audio data is the video signal and audio signal data itself sent to the AV device 43 by the game process when executing the game function.

  FIG. 13 is a flowchart showing recording processing in the game apparatus 42. This recording process is performed by the control unit 58 according to a predetermined program. This recording process is started based on a recording instruction from the user after starting the game process. In this recording process, user record information (signal from the imaging device) and game record information are recorded.

  When the recording process is started, the control unit 58 first waits in step S301 to receive a notification from the imaging device 41 that the user 46 has been recognized within the imaging range. When the notification is received, recording of the video signal and audio signal of the user 46 transmitted from the imaging device 41 is started in step S302. Thereby, the state of the user who operates according to the game content presented in the AV device 43 by the game processing by the control unit 58 is recorded as the user record information. Recording is performed in the second recording unit 57. Next, in step S303, recording of game record information is started. The recording of the game record information is performed on the first recording unit 56.

  Next, in step S304, an instruction to end shooting from the user is awaited. An instruction to end shooting is typically given via the operation input unit 60. If there is an instruction to end the shooting, the process proceeds to step S305, and the recording of the video signal and the audio signal from the imaging device 41 is ended. As a result, a file of user record information is generated. At this time, video thumbnail data corresponding to the generated file is generated, and specific terms related to the file, the recording date and time of the file, and the like are recorded. In step S306, the recording of the game record information is terminated. At the end of recording, processing such as recording of replay data from the start to the end of recording or recording of video and audio signals of a game into one file is performed. Further, information indicating that the generated user record information and game record information files are mutually related is attached to both files and recorded as corresponding file specifying information. Thereby, the recording process of FIG. 13 is completed. In this example, the corresponding file specifying information is recorded as information accompanying the file, but instead, it may be recorded in a table or the like separate from the file.

  FIG. 14 is a flowchart showing another recording process in the game apparatus 42. This recording process is performed by the control unit 58 according to a predetermined program. The recording process is started based on a predetermined recording instruction by a user operation input after the game process is started. In this recording process, user record information (signal from the imaging device) and game record information are recorded. The processes in steps S401 to S403, S405, and S406 in the figure are the same as the processes in steps S301 to S303, S305, and S306 in FIG.

  The recording process of FIG. 14 is started, and after the processes of steps S401 to S403 are performed in the same manner as steps S301 to S303 in FIG. 13, in step S404, the process waits for a predetermined event in the game to end. When a predetermined event ends, for example, when a certain stage in the game is cleared, or when one unit in the instructional game content, for example, one yoga pose ends, a scoring scene in a sports game such as baseball This applies when there is a home run or soccer goal scene.

  When it is detected in step 404 that the predetermined event has ended, in steps S405 and S406, as in the case of steps S305 and S306 in FIG. 13, recording of signals from the imaging device 41 (recording of user record information) and The recording of the game record information is terminated. Thereby, the recording process of FIG. 14 is completed.

  In this recording process, if the end of the predetermined event waiting in step S404 is a scoring scene, only the game recording information and user recording information for a part that goes back to a predetermined period or part from the end of recording. May be deleted, and previous records may be deleted. Further, the user may be able to select whether or not to delete, or whether or not to delete either or both of the game record information and the user record information.

  FIG. 15 is a flowchart showing still another recording process in the game apparatus 42. This recording process is performed by the control unit 58 according to a predetermined program. This recording process is started based on a recording instruction from the user after starting the game process. In this recording process, user recording information (signal from the imaging device) and game recording information are recorded. The processes in steps S501 and S503 to S507 in the figure are the same as the processes in steps S401 to S406 in FIG.

  When the recording process is started and the notification that the presence of the user 46 is recognized from the imaging device 41 is received in step S501 in the same manner as in step S401 in FIG. 14, the control unit 58 performs game processing in step S502. Wait for a predetermined game event to start. As the start of a predetermined game event, for example, the start of a certain stage in the game, the start of one unit in the instructional game content, the start of the game in the sports game, the start of the first half or the second half, the attack / defense of baseball For example, the change of

  When the start of the predetermined game event is detected, in steps S503 to S507, the game record information and the user record information are recorded until the predetermined game event ends in the same manner as in steps S402 to S406 in FIG. . When this recording is completed, thumbnail data of the video corresponding to the file of the user record information generated by the recording is generated, and specific terms related to the file, recording date and time of the file, and the like are recorded. In step S508, it is determined whether an instruction to end the game has been issued by the user. Specifically, it is determined whether or not there has been a power operation, a press of a reset button, selection of another game, or the like in the game device 42. The occurrence of a game over event may be regarded as the end of the game. If it is determined that an instruction to end the game has been given, the recording process of FIG. 15 ends. If it is determined that the end instruction has not been given, the process returns to step S502 to wait for the start of the next game event.

  FIG. 16 is a flowchart showing the playback process in the game apparatus 42. This reproduction process is performed by the control unit 58 according to a predetermined program.

  When the reproduction process is started, first, in step S601, the control unit 58 accepts the selection of a desired one of the user record information files recorded in the second recording unit 57 in accordance with a user instruction. . Specifically, it accepts selection with thumbnail display of recorded video in each file, and accepts selection with sorting or searching using a specific term or recording date and time as a key.

  Next, in step S602, it is determined whether there is a file of game recording information related to the selected file. This determination can be made based on corresponding file specifying information accompanying each file. The case where the related file does not exist corresponds to the case where the file of the game recording information recorded in the first recording unit 56 is deleted in the recording process, for example. Further, when the game record information is recorded replay data, the corresponding game data itself does not exist in the game recording medium 51, for example, the game recording medium 51 is replaced.

  If it is determined that there is a recording file of related game recording information, the setting of the reproduction method is accepted in step S603. As a setting of the playback method, for example, a game video based on the game record information file and a user video based on the user record information file are displayed side by side on one screen at the same time or after one video is played back This corresponds to the setting of whether to play the other video. Settings such as the picture-in-picture, video allocation, and the presence / absence of inversion processing described above may be accepted. By displaying the game video and the user video simultaneously or sequentially, it is possible to compare the game video and the user video.

  Next, in step S604, the game image and the user image are reproduced and displayed by the set reproduction method, and the reproduction process of FIG. 15 is terminated. When the game video and the user video are displayed at the same time, a time stamp may be used as described above in order to match the time relationship between the two during playback with the time relationship during recording. If it is determined in step S602 that the related game record information file does not exist, in step S605, the fact that there is no game video is displayed, and only the user video is played back and displayed, and the playback process of FIG. finish. In this example, the corresponding file specifying information is recorded as information accompanying the file, but instead, it may be recorded in a table or the like separate from the file.

  According to the embodiment of FIGS. 11 to 16, the state of the user who is playing the game in the AV system and the game record information are recorded in parallel, and the user video and the game video related to each other are simultaneously or continuously recorded. Therefore, after the game is played, the user who is playing the game can be confirmed together with the video of the game. In that case, according to the form of FIG.14 and FIG.15, since a user image | video and game recording information were recorded for every game event, it can reproduce | regenerate for every game event. In addition, when replay data is recorded as game record information, the storage capacity of the first recording unit can be reduced as compared with the case where a video signal and an audio signal are recorded as game record information.

  Note that the present invention is not limited to the above-described embodiment, and can be appropriately modified and implemented. For example, in the above description, the mirror image inversion setting is performed on a video signal (video signal from the imaging unit 1) from the video signal input unit 33 or a file to be reproduced (a video file shot by the imaging device 1). On the contrary, the video signal based on the internal video signal or the file corresponding to the reproduction target file may be set as the target. In this case, the user's video is not inverted, and the synthesized video can be displayed by inverting the video of the content being viewed by the user, so that it may be easy to compare and check both videos.

  In the above description, the video signal from the video signal input unit 33 (the video signal from the imaging unit 1 and the internal video signal are synthesized in the same temporal relationship, or the video signal from the video signal input unit 33 is recorded. However, instead of this, the video signal from the video signal input unit 33 is used instead of the video signal and the playback signal of the playback content or the broadcast or external content. You may make it carry out a synthesis | combination or a synthetic | combination display a little later.

  In the above description, the corresponding file specifying information is recorded as information accompanying the file, but instead, it may be recorded in a table or the like separate from the file.

  1: imaging device, 2: recording device, 3: AV device, 4: user, 11: imaging unit, 12: microphone unit, 13: video signal input unit, 14: audio signal input unit, 15: operation input unit, 16 : Recording unit, 17: Control unit, 18: Display unit, 19: Speaker unit, 20: Video signal output unit, 21: Audio signal output unit, 31: Tuner unit, 32: Decoder, 33: Video signal input unit, 34 : Audio signal input unit, 35: operation input unit, 36: first recording / playback unit, 37: control unit, 38: second recording / playback unit, 39: audio signal output unit, 40: video signal output unit, 41 : Imaging device, 42: Game device, 43: AV device, 44: Internet or dedicated line, 45: Game device, 46: User, 51: Game recording medium, 52: Video signal input unit, 53: Audio signal input unit, 54: Video signal output unit, 55: Audio No. output unit, 56: first recording unit, 57: second recording unit, 58: control unit, 59: playback unit, 60: operation input unit, 61: communication unit, 91, 92, 101, 102: video portion.

Claims (18)

Recording that records the first video signal related to the first video supplied from the imaging device outside the device so that it can be reproduced later as a synthesized or continuous video with the second video related to the second video signal A device,
The first video signal is acquired simultaneously with video display based on the second video signal,
The temporal relationship between the first video portion and the second video portion in the composite video or continuous video is the same as the temporal relationship between the first video and the second video at the time of acquiring the first video. A recording apparatus.   The recording apparatus according to claim 1, wherein the recording of the first video signal is performed as a recording of a signal synthesized with the second video signal. The second video signal is based on a file recorded in a recording means in the recording device;
The recording apparatus according to claim 1, wherein the recording of the first video signal is performed as a recording in a file different from the file. The second video signal is supplied from outside the recording apparatus;
The recording apparatus according to claim 1, wherein the recording of the first video signal is performed simultaneously with recording of the second video signal in the recording apparatus and as a separate file.   5. The reproduction start address in the video data related to the corresponding second video signal is recorded so that the subsequent reproduction can be performed when the first video signal is recorded. Recording device. The recording device displays video based on predetermined data in response to a user operation input,
The second video signal is a video signal of the display video;
The recording apparatus according to claim 1, wherein the recording of the first video signal is performed simultaneously with the recording of the second video signal in the recording apparatus and as a separate file. The recording device displays video based on predetermined data in response to a user operation input, and records replay data for performing replay playback,
The second video signal is a video signal of the display video;
The recording apparatus according to claim 1, wherein the recording of the first video signal is performed simultaneously with the recording of the replay data file corresponding to the second video signal in the recording apparatus.   The recording apparatus according to claim 6 or 7, wherein the recording of the one video signal is ended when a predetermined event ends.   9. The recording apparatus according to claim 6, wherein recording of the one video signal is started when a predetermined event is started.   When the recording device is instructed to perform playback based on the file of the first video signal or the file related to the second video signal corresponding thereto, the video signal based on both files is converted to the synthesized video. Alternatively, the recording apparatus according to any one of claims 3 to 9, wherein the recording apparatus outputs a synthesized signal or a separate signal so that a continuous video can be displayed.   The recording apparatus according to claim 10, further comprising an AV apparatus that displays the synthesized video or the continuous video based on a synthesized signal or a separate signal output from the recording apparatus. The recording device comprises input means for accepting settings relating to the composite video or continuous video;
The recording apparatus according to claim 1, wherein the setting includes setting whether to mirror-invert a portion of the first video or the second video in the synthesized video. .   The setting includes setting whether the synthesized video is a picture-in-picture or a two-screen video, or setting the aspect ratio of the first video portion or the second video portion in the synthesized video. The recording apparatus according to claim 12, wherein the recording apparatus is included. The first video signal and the second video signal are accompanied by an audio signal,
The recording apparatus according to any one of claims 1 to 13, wherein when the synthesized video or continuous video is reproduced, an instructed audio signal is reproduced among the accompanying audio signals.   The recording apparatus according to claim 1, wherein the recording of the first video signal is started on condition that the imaging apparatus recognizes a user.   A program for causing a computer to function as the recording device according to claim 1. An imaging device that outputs a video signal;
A recording device for recording the first video signal related to the first video obtained by the imaging device so that the first video signal related to the second video signal is combined with or continuous with the second video related to the second video signal. Equipped,
The first video signal is acquired simultaneously with video display based on the second video signal,
The temporal relationship between the first video portion and the second video portion in the composite video or continuous video is the same as the temporal relationship between the first video and the second video at the time of acquiring the first video. AV system characterized by An imaging process for outputting a video signal;
The first video signal related to the first video obtained by the imaging step can be reproduced later as a video that is synthesized or continuous with the second video related to the second video signal. A recording process for recording,
The first video signal is acquired simultaneously with video display based on the second video signal,
The temporal relationship between the first video portion and the second video portion in the composite video or continuous video is the same as the temporal relationship between the first video and the second video at the time of acquiring the first video. A video recording method characterized by the above.

Images