JP2017112455A - Information processing unit - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique which allows for selection of a desired frame with high accuracy.SOLUTION: An information processing unit includes first selection means for selecting a selection frame that is any one of a plurality of first frames constituting a first moving image, determination means for determining a specific section that is a partial section of a second moving image, based on the frame rate of the first moving image, the frame rate of a second moving image corresponding to the first moving image and having a frame rate higher than that of the first moving image, and the selection frame, and acquisition means for acquiring the second frame included in the specific section, out of a plurality of second frames constituting the second moving image, from an external device for storing the second moving image. The determination means determines a section including a second frame corresponding to the selection frame, out of the plurality of second frames, as the specific section.SELECTED DRAWING: Figure 7

Description

  The present invention relates to an information processing apparatus.

  There has been proposed a technique for generating both a high-resolution and high-frame-rate moving image (main body moving image) and a moving image having the same content as the main body moving image, a low-resolution and low-frame rate (proxy moving image). By using a proxy video, it is possible to play and edit the video more comfortably than using the main video. In addition, a technique for reflecting the result of editing performed on a proxy moving image on a mobile terminal such as a smartphone or a tablet device on a main body moving image recorded on a server or the like has been proposed (Patent Document 1). Patent Document 1 describes designation of a scene cut out from a moving image as editing of the moving image.

  However, in the conventional technique, as a process of reflecting the result of proxy editing in the main body moving image, two main body frames corresponding to the two proxy frames selected as the start point and the end point of the scene are considered in consideration of the frame rate. Only the process of selecting is performed. Proxy editing is editing performed using a proxy moving image, the proxy frame is a frame of the proxy moving image, and the main body frame is a frame of the main body moving image. And since the frame rate of a proxy moving image differs from the frame rate of a main body moving image (because the frame rate of a proxy moving image is lower than the frame rate of a main body moving image), a desired frame does not necessarily exist in a proxy moving image. Therefore, in the conventional technique, in proxy editing, the user cannot confirm a desired frame, or a different frame from the desired frame is erroneously selected.

JP 2010-246008 A

  An object of the present invention is to provide a technique that enables a desired frame to be selected with high accuracy.

The first aspect of the present invention is:
First selection means for selecting a selection frame that is one of a plurality of first frames constituting the first moving image;
Based on the frame rate of the first moving image, the frame rate of the second moving image having a higher frame rate than the first moving image corresponding to the first moving image, and a portion of the second moving image based on the selected frame Determining means for determining a specific section of
Obtaining means for obtaining a second frame included in the specific section from a plurality of second frames constituting the second moving image from an external device storing the second moving image;
Have
The determination unit is an information processing apparatus that determines, as the specific section, a section including a second frame corresponding to the selected frame among the plurality of second frames.

The second aspect of the present invention is:
A selection step of selecting a selection frame that is one of a plurality of first frames constituting the first moving image;
Based on the frame rate of the first moving image, the frame rate of the second moving image having a higher frame rate than the first moving image corresponding to the first moving image, and a portion of the second moving image based on the selected frame A determining step for determining a specific section of
Obtaining a second frame included in the specific section among a plurality of second frames constituting the second moving image from an external device storing the second moving image;
Have
In the determining step, the section including the second frame corresponding to the selected frame among the plurality of second frames is determined as the specific section.

  A third aspect of the present invention is a program that causes a computer to function as each unit of the information processing apparatus described above.

  According to the present invention, a desired frame can be selected with high accuracy.

The block diagram which shows the structural example of the reproducing | regenerating apparatus concerning this embodiment. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to the present embodiment. The figure which shows an example of the main body animation and proxy animation concerning this embodiment The figure which shows the file structural example of the moving image which concerns on this embodiment The figure which shows an example of the data communication which concerns on this embodiment The figure which shows an example of the difference of the main body moving image and proxy moving image which concern on this embodiment The flowchart which shows an example of the processing flow of the system which concerns on this embodiment The figure which shows an example of the GUI image which concerns on this embodiment The flowchart which shows an example of the processing flow of the reproducing | regenerating apparatus concerning this embodiment. The figure which shows an example of the GUI image which concerns on this embodiment The flowchart which shows an example of the processing flow of the system which concerns on this embodiment The figure which shows an example of the notification process which concerns on this embodiment

  Hereinafter, embodiments of the present invention will be described. FIG. 1 is a block diagram illustrating a configuration example of a playback apparatus 1000 that is an information processing apparatus according to the present embodiment. The playback apparatus 1000 includes a control unit 1001, a first storage unit 1002, a second storage unit 1003, a display unit 1004, an operation unit 1005, a communication unit 1006, an audio output unit 1007, and the like. Note that the information processing apparatus according to the present embodiment is not limited to a playback apparatus. For example, as an information processing apparatus according to the present embodiment, a PC (personal computer), a portable terminal (smart phone, tablet device, portable media player, etc.), an image display apparatus (liquid crystal display apparatus, organic EL display apparatus, plasma display) Device, MEMS shutter type display device, etc.), server, etc. may be used.

  The control unit 1001 controls various operations and processes of each functional unit of the playback device 1000 according to a signal input to the control unit 1001, a program described later, and the like. As the control unit 1001, for example, a central processing unit (CPU) can be used. Note that one piece of hardware may be used as the control unit 1001, or a plurality of pieces of hardware may be used. The processing of the playback apparatus 1000 may be controlled by a plurality of hardware sharing and executing the processing.

  The first storage unit 1002 is a storage unit that temporarily stores various data, and is used for processing (operation) of the control unit 1001. For example, a volatile memory can be used as the first storage unit 1002. The second storage unit 1003 is a storage unit that can electrically store various data. The second storage unit 1003 stores, for example, various programs (including firmware) used in the playback apparatus 1000, various setting information, and the like. As the second storage unit 1003, for example, a nonvolatile memory can be used.

  In the present embodiment, data acquired from an external device by the playback device 1000 (communication unit 1006) is recorded in the second storage unit 1003, but is not limited thereto. For example, data acquired from an external device by the playback device 1000 may be recorded in another storage unit (a storage unit that is detachable from the playback device 1000 or another storage unit that is built in the playback device 1000). Good. As the other storage unit, a semiconductor memory (memory card), a magnetic disk (hard disk), an optical disk (CD, DVD, Blu-ray disc), or the like can be used. In that case, the playback apparatus 1000 only needs to have a function of accessing the other storage unit and reading / writing data from / to the other storage unit, deleting data stored in the other storage unit, and the like. .

  The display unit 1004 displays an image on the screen. The display unit 1004 displays an image based on image data, displays a graphic image (characters, icons, etc.) for interactive operation, and the like. As the display unit 1004, for example, a liquid crystal display panel, an organic EL display panel, a plasma display panel, a MEMS shutter system display panel, or the like can be used. Note that an image display device separate from the playback device 1000 may be used as the display unit 1004. The playback apparatus 1000 only needs to have a function of controlling display on the display unit 1004.

  The operation unit 1005 accepts user operations on the playback apparatus 1000. The operation unit 1005 outputs an operation signal corresponding to the user operation to the control unit 1001. Then, the control unit 1001 performs processing according to the operation signal. That is, the control unit 1001 performs processing according to a user operation on the playback apparatus 1000. As the operation unit 1005, for example, an input device such as a physical button, a touch panel, or a keyboard can be used. Further, as the operation unit 1005, an input device separate from the playback apparatus 1000 can be used. As an input device separate from the playback apparatus 1000, for example, a keyboard, a mouse, a remote controller, and the like can be used. The playback apparatus 1000 only needs to have a function of receiving an electrical signal corresponding to a user operation using the input device.

  A communication unit 1006 connects the playback device 1000 to an external device, and transmits and receives data (control commands, image data, etc.) between the playback device 1000 and the external device. As the communication protocol, for example, PTP (Picture Transfer Protocol), MTP (Media Transfer Protocol), or the like can be used. Note that the communication unit 1006 may connect the playback device 1000 to an external device using a cable such as a USB (Universal Serial Bus) cable. Data transmission / reception between the playback apparatus 1000 and the external apparatus may be performed using a cable. The communication unit 1006 may wirelessly connect the playback device 1000 to an external device using a wireless LAN or the like. The playback device may be directly connected to the external device, or may be connected to the external device via a network such as the Internet, a server, or the like.

  The audio output unit 1007 outputs audio based on audio data. As the audio output unit 1007, for example, a speaker can be used. Note that an audio output device that is separate from the playback device 1000 may be used as the audio output unit 1007. The playback apparatus 1000 only needs to have a function of controlling audio output by the audio output unit 1007.

  FIG. 2 is a block diagram illustrating a configuration example of the imaging device 2000 that is an external device of the playback device 1000. The imaging apparatus 2000 includes a first storage unit 2001, a storage unit 2002, a control unit 2003, an operation unit 2004, an imaging unit 2005, a display unit 2006, an optical member 2007, an audio input unit 2008, a communication unit 2009, and a second storage unit. 2010, etc. Note that the external device of the playback device 1000 is not limited to the imaging device. For example, a PC, a portable terminal, an image display device, a playback device, a server, or the like may be used as an external device of the playback device 1000.

  The control unit 2003 controls various calculations and processing of each functional unit of the imaging apparatus 2000 according to a signal input to the control unit 2003, a program described later, and the like. As the control unit 2003, for example, a CPU can be used. Note that one piece of hardware may be used as the control unit 2003, or a plurality of pieces of hardware may be used. The processing of the imaging apparatus 2000 may be controlled by a plurality of hardware sharing and executing the processing.

  The first storage unit 2001 is a storage unit that temporarily stores various data, and is used for processing (work) of the control unit 2003. As the first storage unit 2001, for example, a volatile memory (DRAM or the like) can be used. The second storage unit 2010 is a storage unit that can electrically store various data. For example, the second storage unit 2010 stores various programs (including firmware) used in the imaging apparatus 2000, various setting information, and the like. As the second storage unit 2010, for example, a non-volatile memory (flash memory or the like) can be used.

  The optical member 2007 includes a lens, a shutter, a diaphragm, and the like. The optical member 2007 forms an image of light from the subject on the imaging unit 2005. Specifically, the optical member 2007 controls the amount of light from the subject to an appropriate amount at an appropriate timing, and forms an image of the light after the amount of light is controlled on the imaging unit 2005.

  The imaging unit 2005 generates image data that is digital data by converting light (subject light) imaged by the optical member 2007 into an electrical signal (imaging; photographing). The imaging unit 2005 outputs the image data (photographed image data) generated by photographing. As the imaging unit 2005, for example, an imaging device such as a CMOS sensor or a CCD sensor can be used.

  The storage unit 2002 is a storage unit that can store various data. For example, captured image data is recorded in the storage unit 2002. As the storage unit 2002, a semiconductor memory (memory card), a magnetic disk (hard disk), an optical disk (CD, DVD, Blu-ray disc), or the like can be used. Note that the storage unit 2002 may be a storage unit that can be attached to and detached from the imaging device 2000, or may be a storage unit that is built in the imaging device 2000. The imaging apparatus 2000 only needs to have a function of accessing the storage unit 2002 and reading / writing data from / to the storage unit 2002, deleting data stored in the storage unit 2002, and the like.

  The operation unit 2004 receives a user operation on the imaging apparatus 2000. The operation unit 2004 outputs an operation signal corresponding to the user operation to the control unit 2003. Then, the control unit 2003 performs processing according to the operation signal. That is, the control unit 2003 performs a process according to a user operation on the imaging apparatus 2000. As the operation unit 2004, for example, an input device such as a physical button, a touch panel, or a keyboard can be used. Further, as the operation unit 2004, an input device separate from the imaging device 2000 can be used. As an input device separate from the imaging apparatus 2000, for example, a keyboard, a mouse, a remote controller, and the like can be used. The imaging apparatus 2000 only needs to have a function of receiving an electrical signal corresponding to a user operation using the input device.

  The display unit 2006 displays an image on the screen. A display unit 2006 displays an image based on the captured image data, displays a graphic image (characters, icons, etc.) for interactive operation, and the like. In addition, the display unit 2006 may display an image representing an imaging target at the time of imaging. That is, the display unit 2006 may be used as an electronic viewfinder. Note that a display device separate from the imaging device 2000 may be used as the display unit 2006. The imaging device 2000 only needs to have a function of controlling display on the display unit 2006.

  The voice input unit 2008 generates voice data (voice signal) based on the voice input to the voice input unit 2008. As the voice input unit 2008, for example, a microphone can be used. Note that a voice input device separate from the imaging device 2000 may be used as the voice input unit 2008. The imaging apparatus 2000 only needs to have a function of acquiring audio data generated by the audio input unit 2008.

  A communication unit 2009 connects the imaging device 2000 to an external device, and transmits and receives data (control commands, image data, etc.) between the imaging device 2000 and the external device.

  FIG. 3 is a diagram illustrating an example of a captured image (captured image data) obtained by the imaging apparatus 2000. In FIG. 3, reference numeral 3001 indicates the imaging device 2000 of FIG. 2. The imaging device 3001 can perform moving image shooting to obtain a moving image as a captured image. Then, the imaging apparatus 3001 can generate the first moving image 3003 and the second moving image 3002 by one moving image shooting. Therefore, the first moving image 3003 is a moving image having the same content as the second moving image 3002. The second moving image 3002 corresponds to the first moving image 3003 and is a moving image having a higher frame rate than the first moving image 3003. That is, the first moving image 3003 corresponds to the second moving image 3002 and has a lower frame rate than the second moving image 3002. Therefore, by using the first moving image 3003, the moving image can be reproduced and edited more comfortably than using the second moving image 3002.

  The image resolution of the first moving image 3003 and the image resolution of the second moving image 3002 are not particularly limited, but in the example of FIG. 3, the image resolution of the first moving image 3003 is lower than the image resolution of the second moving image 3002. If the image resolution of the first movie 3003 is lower than the image resolution of the second movie 3002, the movie (first movie 3003) can be reproduced and edited more comfortably. Hereinafter, the first moving image is referred to as “proxy moving image”, and the second moving image is referred to as “main body moving image”.

  The imaging device 3001 can also perform still image shooting that obtains a still image as a captured image. Specifically, the imaging apparatus 3001 can also perform various types of still image shooting for obtaining various types of still images, various types of moving image shooting for acquiring various types of moving images, and the like.

  The generation of the main video 3002 and the generation of the proxy video 3003 may be performed in parallel, or the proxy video 3003 may be generated by performing image processing on the main video 3002 after the main video 3002 is generated. . The process of generating the proxy video 3003 from the main body video 3002 may be performed by a device different from the imaging device 3001.

  In the present embodiment, an example in which the first moving image and the second moving image are moving images obtained by shooting will be described, but the first moving image and the second moving image are not particularly limited. For example, the first video and the second video may be a video of broadcast content (broadcast program, CM (commercial message), etc.), a CG (computer graphic) video, or an animation. It may be a video.

FIG. 4 is a diagram illustrating a file configuration example of the main body moving image. The main body moving image file (moving image file) 400 includes attribute information 410, related attribute information 420, frame attribute information 430, and
Frame data 440 is included. The file structure of the proxy moving image is the same as the file structure of the main body moving image. Specifically, the proxy moving image has a file structure in which “main body moving image” in the following description is replaced with “proxy moving image”.

  The attribute information 410 is attribute information of the main body moving image (main body moving image corresponding to the moving image file 400). For example, the attribute information 410 includes information (shooting date / time information) indicating the shooting date / time of the main body moving image, the model name (camera model name) of the imaging device used for shooting the main body moving image, the file size of the moving image file 400, the main body moving image's Image resolution, start time code (time code of the first frame of the main video), end time code (time code of the last frame of the main video), duration of the main video, frame rate of the main video, main video (video file) 400), an ID for uniquely determining, and other additional information. As the ID, for example, a UMID (Unique Material Entity) defined by SMPTE (Society of Motion Picture and Television Engineers) is used. UMID is often used as a moving image identifier.

  The related attribute information 420 is attribute information of a proxy video related to the main video (video file 400). For example, the related attribute information 420 includes the duration of the proxy movie, the frame rate of the proxy movie, the UMID of the proxy movie, and other additional information. The attribute information 410 and the related attribute information 420 are described in the header part of the moving image file 400.

  The frame attribute information 430 is frame attribute information of the main body moving image. For example, the frame attribute information 430 includes a frame time code, shooting parameters used for shooting the frame (shutter speed, aperture value, ISO sensitivity, shooting mode, etc.), other additional information, and the like. Frame data 440 is image data of a frame of the main body moving image. The frame attribute information 430 is described in the header part of the data area related to the frame, and the frame data 440 is described in the image data part of the data area related to the frame. As shown in FIG. 4, the moving image file 400 includes frame attribute information 430 and frame data 440 for each frame of the main body moving image.

  Note that the data format of the frame data 440 of the main body moving image and the data format of the frame data 440 of the proxy moving image are not particularly limited. For example, RAW format image data is used as the frame data 440 of the main body moving image, and H.264 is used as the frame data 440 of the proxy moving image. Image data compressed in the H.264 format is used.

  FIG. 5 is a diagram illustrating an example of data transmission / reception between the imaging apparatus and the reproduction apparatus. In FIG. 5, reference numeral 5001 indicates the imaging apparatus 2000 in FIG. 2, and reference numeral 5002 indicates the reproduction apparatus 1000 in FIG. For example, data is transmitted and received between the imaging device 5001 and the playback device 5002 by wireless communication using Wi-Fi, Bluetooth, or the like. FIG. 5 shows a state in which the proxy moving image 5003 is transmitted from the imaging device 5001 to the playback device 5002. Note that data transmission / reception may be performed between the imaging device 5001 and the playback device 5002 by wired communication. In addition, the imaging apparatus 5001 may record data in an external storage such as an SD card, the external storage may be detached from the imaging apparatus 5001 and attached to the playback apparatus 5002, and the playback apparatus may read data from the external storage.

FIG. 6 is a diagram illustrating an example of a difference between the main body moving image and the proxy moving image. Hereinafter, each of a plurality of frames (second frame) constituting the main moving image is referred to as a “main body frame”, and each of a plurality of frames (first frame) constituting the proxy moving image is referred to as a “proxy frame”. In the example of FIG. 6, the main moving image 6001 has a frame rate of 120 fps, and the proxy moving image 6002 has a frame rate of 30 fps. Then, proxy video 6002
Is a moving image obtained by thinning out the main body frame from the main body moving image 6001. In the example of FIG. 6, three main body frames are thinned between two main body frames corresponding to two consecutive proxy frames. The thinning-out number M, which is the number of body frames thinned between two body frames corresponding to two consecutive proxy frames, can be calculated using the following equation 1. In Formula 1, “FR_O” is the frame rate of the main moving image, and “FR_P” is the frame rate of the proxy moving image. In the example of FIG. 6, the thinning-out number M = 120 ÷ 30−1 = 3 is obtained.

M = FR_O ÷ FR_P−1 (Expression 1)

  Note that the proxy video may not be a video in which the main body frame is thinned out from the main body video. For example, the plurality of proxy frames constituting the proxy moving image may include an interpolation frame generated from two or more frames of the main moving image. In that case, at least one of the body frames used for generating the interpolation frame can be interpreted as a body frame corresponding to the interpolation frame. The body frame corresponding to the interpolation frame can be detected based on information on the interpolation method, information indicating the correspondence between the proxy frame and the body frame, and the like. Such information is included in, for example, a moving image file. Further, the frame rate of the main body moving image and the frame rate of the proxy moving image are not particularly limited. The frame rate of the main moving image may be higher or lower than 120 fps, and the frame rate of the proxy moving image may be higher or lower than 30 fps.

  FIG. 7 is a flowchart illustrating an example of a processing flow of the system according to the present embodiment (the system including the reproduction apparatus 1000 in FIG. 1 and the imaging apparatus 2000 in FIG. 2). FIG. 7 shows a processing flow when the first selection process is performed by the playback apparatus 1000. In the first selection process, one of a plurality of proxy frames is selected as a candidate for a desired frame (such as a frame to be cut out). At the start of the processing flow in FIG. 7, it is assumed that the second storage unit 1003 of the playback device 1000 stores a proxy moving image, and the storage unit 2002 of the imaging device 2000 stores a main body moving image.

  First, in S7001, the control unit 1001 of the playback apparatus 1000 selects any one of a plurality of proxy frames constituting the proxy moving image (first selection process). Hereinafter, the first frame selected by the first selection process is referred to as a “selected frame”. Details of the first selection process will be described later. In step S <b> 7002, the control unit 1001 acquires the frame rate information of the proxy moving image from the proxy moving image file (attribute information 410 in FIG. 4).

  In step S <b> 7003, the control unit 1001 requests the image capturing apparatus 2000 for information on the frame rate of the main body moving image via the communication unit 1006 of the playback apparatus 1000. Next, in step S7004, the control unit 2003 of the imaging apparatus 2000 outputs information on the frame rate of the main body moving image in response to receiving the request for S7003 from the playback apparatus 1000 via the communication unit 2009 of the imaging apparatus 2000. To do. Specifically, the control unit 2003 acquires information on the frame rate of the main body moving image from the moving image file of the main body moving image (attribute information 410 in FIG. 4), and sends the acquired information to the playback apparatus 1000 via the communication unit 2009. Output. As a result, the control unit 1001 acquires information about the frame rate of the main body moving image from the imaging apparatus 2000 via the communication unit 1006.

For example, in step S7003, the control unit 1001 acquires the UMID of the main body video from the proxy video file (related attribute information 420 in FIG. 4), and outputs the acquired UMID to the imaging apparatus 2000. The control unit 2003 can uniquely determine the main body moving image from the UMID output from the playback apparatus 1000. In step S <b> 7004, the control unit 2003 outputs information on the frame rate of the main body moving image corresponding to the UMID output from the playback apparatus 1000 to the playback apparatus 1000.

  As shown in FIG. 4, the video file of the main video may include information about the frame rate of the proxy video, or the video file of the proxy video may include information about the frame rate of the main video. . When the proxy moving image file includes information on the frame rate of the main moving image, the control unit 1001 may acquire information on the frame rate of the main moving image in the moving image file of the proxy moving image. Then, the processes of S7003 and S7004 may be omitted. The processing of S7003 and S7004 is necessary, for example, when the proxy moving image file does not include the frame rate information of the main moving image.

  In steps S7005 and S7006, the control unit 1001 determines a specific section that is a section of the main body moving image based on the frame rate of the main body moving image, the frame rate of the proxy moving image, and the selected frame. For example, the specific section is a section to be referred to when a desired frame (such as a frame to be cut out) is selected from the sections of the main body moving image. The specific section is also a section that does not need to be referred to when selecting a desired frame among the sections of the main body moving image. In the present embodiment, a section including the main body frame corresponding to the selected frame among the plurality of main body frames constituting the main body moving image is determined as the specific section. This is because there is a high possibility that the desired frame exists around (before and after) the selected frame.

  Since the selected frame is a desired frame candidate, there is a high possibility that the desired frame is not included in the section before the proxy frame immediately before the selected frame. For this reason, it is considered that the main frame corresponding to the proxy frame immediately before the selected frame does not need to be referred to when selecting a desired frame. Therefore, it is preferable that a section that does not include the main body frame corresponding to the proxy frame immediately before the selected frame is determined as the specific section.

  In addition, since the selected frame is a candidate for a desired frame, there is a high possibility that the desired frame is not included in the section after the proxy frame immediately after the selected frame. Therefore, it is considered that there is no need to refer to the main body frame corresponding to the proxy frame immediately after the selected frame when selecting a desired frame. Therefore, it is preferable that a section that does not include the main body frame corresponding to the proxy frame immediately after the selected frame is determined as the specific section.

  In the present embodiment, from the main body frame immediately after the main body frame corresponding to the proxy frame immediately before the selected frame to the main body frame immediately before the main body frame corresponding to the proxy frame immediately after the selected frame. An example in which the section is determined as the specific section will be described. However, the specific section is not limited to this. For example, a section up to the main moving picture frame corresponding to the selected frame, a section up to the main moving picture frame m1 before or after the main moving picture frame corresponding to the selected frame, etc. may be determined as the specific section (m1 An integer of 1 or more). A section from the main moving picture frame corresponding to the selected frame, a section from the main moving picture frame m2 before or after the main moving picture frame corresponding to the selected frame, etc. may be determined as the specific section (m2 is 1 or more) Integer).

In addition, a section from the moving image frame corresponding to the proxy frame immediately before the selected frame may be determined as the specific section. A section from the main moving picture frame m3 after the main moving picture frame corresponding to the proxy frame immediately before the selected frame may be determined as the specific section (m3 is an integer of 2 or more). A section from the main moving picture frame m4 ahead of the main moving picture frame corresponding to the proxy frame immediately before the selected frame may be determined as the specific section (m4 is an integer of 1 or more). The specific section may be determined based on the proxy frame M1 before the selected frame, not the proxy frame immediately before the selected frame (M1 is an integer of 2 or more).

  A section up to the main moving picture frame corresponding to the proxy frame immediately after the selected frame may be determined as the specific section. A section up to m5 main moving picture frames before the main moving picture frame corresponding to the proxy frame immediately after the selected frame may be determined as a specific section (m5 is an integer of 2 or more). A section up to m6 main video frames after the main video frame corresponding to the proxy frame one after the selected frame may be determined as a specific section (m6 is an integer of 1 or more). The specific section may be determined on the basis of the proxy frame M2 after the selected frame, not the proxy frame immediately after the selected frame (M2 is an integer of 2 or more). The values of m1 to m6, M1, and M2 may be predetermined fixed values, values that can be changed by the user, or are automatically determined according to the type of moving image. It may be a value.

  In step S7005, the control unit 1001 uses the information acquired in steps S7002 and S7004 (the frame rate of the main moving image and the frame rate of the proxy moving image) as the number of target main body frames that are acquisition target main frames. Get the number M. As described above, the thinning-out number M is the number of body frames thinned between two body frames corresponding to two consecutive proxy frames, and can be calculated using Equation 1. In this embodiment, it is assumed that the frame rate FR_O = 120 fps of the main body moving image and the frame rate FR_P = 30 fps of the proxy moving image. Therefore, the number M = 3 is obtained.

  In step S7006, the control unit 1001 determines a target main body frame based on the selected frame selected in step S7001, the information acquired in steps S7002 and S7004, and the number M determined in step S7005. All body frames included in a specific section may be determined as target body frames, but when a desired frame is selected, the selected frame can be referred to, and therefore a body frame corresponding to the selected frame is not necessary There is. Therefore, in S7006, a body frame that is included in the specific section and does not correspond to the selected frame is determined as a target body frame. The process of S7006 will be described in detail below.

First, the control unit 1001 detects a main body frame corresponding to the selected frame based on the selected frame, the main body moving image frame rate FR_O, and the proxy moving image frame rate FR_P. If the first frame of the moving image is the 0th frame, the time position P_O of the main body frame corresponding to the time position P_P of the selected frame can be calculated using the following equation 2. In the present embodiment, it is assumed that the P_P = 5th proxy frame of the proxy moving image is selected as the selection frame. Therefore, P_O = 5 × (120 ÷ 30) = 20th body frame is detected as the body frame corresponding to the selected frame.

P_O = P_P × (FR_O ÷ FR_P) (Expression 2)

  Next, the control unit 1001 determines the target main body frame based on the time position P_O (the detection result of the main body frame corresponding to the selected frame) and the number M. Specifically, M body frames immediately before the P_Oth body frame and M body frames immediately after the P_Oth body frame are determined as target body frames. In the present embodiment, the time position P_O = 20 and the number M = 3. Therefore, the three main body frames immediately before the 20th main body frame and the three main body frames immediately after the twentieth main body frame are determined as target main body frames. That is, the 17, 18, 19, 21, 22, and 23rd frames of the main body moving image are determined as the target main body frames.

  Note that since the section composed of the main body frame corresponding to the selected frame and the target main body frame is the specific section, the process of determining the target main body frame can also be referred to as “a process of determining the specific section”. Further, since the definition of the specific section is not particularly limited as described above, the determination method of the specific section (target body frame) is not particularly limited. For example, the specific section may be determined without acquiring the number M. In addition, when information indicating the correspondence relationship between the proxy frame and the main body frame is prepared in advance, the main body frame corresponding to the selected frame is detected based on the information prepared in advance and the selected frame. Good.

  In step S <b> 7007, the control unit 1001 requests the image capturing apparatus 2000 for a target main body frame via the communication unit 1006. In step S <b> 7008, the control unit 2003 outputs the target main body frame to the reproduction apparatus 1000 via the communication unit 2009 in response to receiving the request of step S <b> 7007 from the reproduction apparatus 1000 via the communication unit 2009. As a result, the control unit 1001 acquires the target main body frame from the imaging device 2000 via the communication unit 1006 (S7009).

  Note that the target main body frame may or may not be output from the imaging apparatus 2000 to the playback apparatus 1000 without being subjected to image processing such as a change in image resolution. For example, the control unit 2003 may reduce the image resolution of the target main body frame. In the present embodiment, the image resolution of the main body frame is higher than the image resolution of the proxy frame. In such a case, the control unit 2003 may reduce the image resolution of the target main body frame to approximately the same image resolution as that of the proxy frame. Then, the control unit 2003 may output the target main body frame after the image resolution is reduced to the playback apparatus 1000. In the present embodiment, “substantially the same” includes “perfect match”. Further, only the thumbnail image of the target main body frame may be output from the imaging device 2000 to the playback device 1000. As a result, the playback apparatus 1000 reduces the transmission amount and transmission time of the target body frame from the imaging apparatus 2000 to the playback apparatus 1000, and performs processing using the target body frame (display, deletion, etc. of the target body frame). The processing load at the time can be reduced.

  In the present embodiment, the main body frame corresponding to the selected frame is not determined as the target main body frame. Therefore, the total number of target main body frames can be reduced, and the transmission amount and transmission time of the target main body frame from the imaging apparatus 2000 to the playback apparatus 1000 can be reduced. However, a high image resolution may be required for the selected frame. In such a case, it is preferable that the main body frame corresponding to the selected frame is further determined as the target main body frame.

  FIG. 8 is a diagram illustrating an example of a GUI (Graphical User Interface) image displayed on the display unit 1004 of the playback apparatus 1000 when the first selection process of S7001 is performed. The control unit 1001 performs display control to display the GUI image shown in FIG. 8 on the display unit 1004 when the first selection process is performed. Reference numerals 8000 and 8100 indicate a method of the first selection process. In the GUI image of FIG. 8, one or more proxy frames (one or more thumbnail images respectively corresponding to one or more proxy frames) are arranged. When a swipe operation for tracing the screen as indicated by an arrow 8101 is performed, the control unit 1001 switches the proxy frame (thumbnail image) displayed on the display unit 1004 to another proxy frame by scrolling.

In the method 8000, the control unit 1001 selects the designated proxy frame as a selection frame in response to a user operation that designates one of a plurality of proxy frames. Specifically, as indicated by reference numeral 8001, when a user operation for designating a thumbnail image of a proxy frame is performed, as indicated by reference numeral 8002, the control unit 1001 displays a proxy frame ( Non-thumbnail image). Thereafter, when a touch operation is performed on the determination button 8003, the control unit 1001 determines the designated proxy frame as a selection frame.

  In the method 8100, the control unit 1001 selects a selected frame based on the proxy frame (thumbnail image) displayed on the display unit 1004. Specifically, as indicated by reference numeral 8102, the control unit 1001 displays a proxy frame N frames after the proxy frame in which the thumbnail image is displayed at a predetermined position (center, right end, left end, etc.) of the display unit 1004. Is selected as the selected frame. N is an integer greater than or equal to 1, for example 50. Note that the value of N may be a predetermined fixed value, a value that can be changed by the user, or a value that is automatically determined according to the type of moving image or the like. Good. Further, the value of N may be determined in consideration of at least the scroll speed at which the proxy frame (thumbnail image) displayed on the display unit 1004 is switched to another proxy frame by scrolling. For example, a larger value may be determined as the value N as the scroll speed is higher. The value of N may be determined in consideration of at least the transmission speed when the main body frame is transmitted from the imaging apparatus 2000 to the playback apparatus 1000. For example, a smaller value may be determined as the value of N as the transmission speed is slower. The value of N may be determined based on one of the scroll speed and the transmission speed, or the value of N may be determined based on both the scroll speed and the transmission speed.

  Note that the method of the first selection process is not limited to the above method. For example, a proxy frame having a specific image feature amount may be automatically selected as a selection frame based on the image feature amount of each proxy frame. As long as a specific proxy frame can be selected, the proxy frame may be selected by any method.

  In the present embodiment, the control unit 1001 performs display control to display at least one of a plurality of frames including the plurality of proxy frames and the acquired target main body frame on the display unit 1004 after acquiring the target main body frame. . For example, the control unit 1001 adds a thumbnail image of the target main body frame to the GUI image in FIG. Thereby, the user of the playback apparatus 1000 can browse not only the proxy frame but also the target main body frame. The thumbnail image of the target main body frame may be generated inside the playback apparatus 1000 or may be input to the playback apparatus 1000 from the outside.

  FIG. 9 is a flowchart illustrating an example of a processing flow of the playback apparatus 1000. FIG. 9 shows a process flow when the second selection process is performed by the playback apparatus 1000. In the second selection process, one of a plurality of frames including the plurality of proxy frames and the acquired target main body frame is selected as a desired frame (such as a frame to be cut out) according to a user operation. The process flow of FIG. 9 is performed after the process flow of FIG. 7 is performed.

  First, in S9001, the control unit 1001 performs display control to display at least one of a plurality of frames including the plurality of proxy frames and the target main body frame acquired in S7009 on the display unit 1004. Reference numeral 10000 in FIG. 10 indicates an example of a display form of the display unit 1004 after the processing of S9001 is performed. In S9001, display control is performed to display a GUI image in which the selected frame 10001 and the six target main body frames 10002 are arranged as in the display form 10000.

  In step S <b> 9002, the control unit 1001 waits for a user operation to specify one of the displayed seven frames. When a user operation is performed, the control unit 1001 selects the commanded frame as a desired frame.

In step S9003, the control unit 1001 automatically deletes the target main body frame (non-selected frame) that was not selected in step S9002. Specifically, in S7009 of FIG. 7, the control unit 1001 records the acquired target main body frame in the first storage unit 1002 or the second storage unit 1003. In step S9003, the control unit 1001 deletes the target main body frame not selected in step S9002 from the first storage unit 1002 or the second storage unit 1003. Note that the process of deleting the target main body frame may be a process of deleting the target main body frame from the screen instead of a process of deleting the target main body frame from the storage unit. Reference numeral 10100 in FIG. 10 indicates an example of a display form of the display unit 1004 after the processing of S9003 is performed. When the main body frame 10101 is selected in S9002 as in the display form 10100, the remaining five main body frames are erased from the screen.

  As described above, according to the present embodiment, in the playback device, the specific section is determined, and the main body frame included in the specific section is acquired from the external device. Thereby, when a desired frame is selected using the playback device, not only the proxy frame but also the main frame of a specific section can be used. As a result, a desired frame can be selected with high accuracy. For example, when a desired frame is a main body frame and a proxy frame corresponding to the desired frame does not exist, a proxy frame that is not a desired frame can be prevented from being erroneously selected as a desired frame. .

  Note that when the first selection process is performed, information (notification information) regarding the fact that the frame rate of the proxy moving image is lower than the frame rate of the main unit moving image may be notified to the user. FIG. 11 is a flowchart illustrating an example of a processing flow of the system according to the present embodiment. FIG. 11 shows a process flow when the notification process is performed by the playback apparatus 1000. In the notification process, the notification information described above is notified to the user. For example, information indicating that some main body frames are thinned out in the proxy moving image, information indicating the frame rate of the proxy moving image and the frame rate of the main moving image, and the like are notified as the notification information. Information indicating that a desired frame cannot be selected with high accuracy unless the main body frame is acquired, information indicating that processing for acquiring the main frame is performed after the first selection processing, and the like may be notified as notification information. .

  First, in step S11001, the control unit 1001 activates (enables) a frame selection function that is a function of selecting a desired frame in response to a user operation. Next, the processes of S11002 to S11005 are performed. Since the processing of S11002 to S11005 is the same as the processing of S7002 to S7005 in FIG. 7, the description of the processing of S11002 to S11005 is omitted. Following S11005, in S11006, the control unit 1001 performs the notification process described above. Specifically, as illustrated in FIG. 12, the control unit 1001 displays a thumbnail view image in which the thumbnail images of the proxy frames are arranged as notification processing, and the thumbnail images of the thinned main body frames. Instead, display control for displaying a broken line 12000 is performed. The number of thinned main frames and the time position can be determined from the thinning number N. The thumbnail view image is, for example, the GUI image in FIG. By performing such display control, when the user designates a selection frame, it is possible to easily grasp that some main body frames are thinned out from the main body moving image.

  Note that the processing of S11002 to S11005 is the same as the processing of S7002 to S7005 in FIG. 7, and thus the processing of S7002 to S7005 may be omitted. For example, after the processing of S11006, the processing of S7001, S7006 to S7009 of FIG. 7 may be performed, and then the processing flow of FIG. 9 may be performed.

The notification process is not limited to the display control described above. For example, as the graphic image showing the thinned main body frame, another graphic image (a predetermined icon or the like) other than the broken line 12000 may be used. As the notification process, display control for displaying a text image in which notification information is described may be performed. The notification information may be notified by a method other than image display. For example, the notification information may be notified by voice output, lamp lighting, or the like. The notification information may be notified to the user by any method as long as the user can grasp the notification information.

  Although the present invention has been described in detail based on the preferred embodiments, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. For example, the process of S9003 in FIG. 9 and the process of S11006 in FIG. 11 may be omitted. Even if the process P1 for selecting a desired frame from only a plurality of proxy frames and the process P2 for selecting a desired frame from a plurality of frames including a plurality of proxy frames and a target main body frame can be switched and executed. Good. For example, the playback apparatus may be able to set either an operation mode for executing the process P1 or an operation mode for executing the process P2. Then, the playback device may execute processing corresponding to the set operation mode. Further, the playback device may inquire the user whether to execute the process P1 or the process P2. Then, the playback device may execute processing designated by the user.

<Other embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  1000, 5002: Playback device 1001: Control unit 1006: Communication unit

Claims (17)

First selection means for selecting a selection frame that is one of a plurality of first frames constituting the first moving image;
Based on the frame rate of the first moving image, the frame rate of the second moving image having a higher frame rate than the first moving image corresponding to the first moving image, and a portion of the second moving image based on the selected frame Determining means for determining a specific section of
Obtaining means for obtaining a second frame included in the specific section from a plurality of second frames constituting the second moving image from an external device storing the second moving image;
Have
The information processing apparatus according to claim 1, wherein the determining unit determines, as the specific section, a section including a second frame corresponding to the selected frame among the plurality of second frames. The first video is a video obtained by thinning out the second frame from the second video,
The determining means includes
Based on the frame rate of the first moving image and the frame rate of the second moving image, the number of second frames thinned between two second frames corresponding to two consecutive first frames is obtained. And
The information processing apparatus according to claim 1, wherein the specific section is determined based on the acquired number, the frame rate of the first moving image, the frame rate of the second moving image, and the selected frame. . The determining means includes
Detecting the second frame corresponding to the selected frame based on the frame rate of the first moving image, the frame rate of the second moving image, and the selected frame;
The specific section is determined based on a frame rate of the first moving image, a frame rate of the second moving image, and a detection result of the second frame corresponding to the selected frame. 2. The information processing apparatus according to 2. When the first selection means selects the selected frame, the information processing apparatus further comprises notification means for notifying information relating to a frame rate of the first moving image being lower than a frame rate of the second moving image. The information processing apparatus according to any one of claims 1 to 3. The said 1st selection means selects the designated 1st frame as said selection frame according to the user operation which designates either of these 1st frames, The any one of Claims 1-4 characterized by the above-mentioned. The information processing apparatus according to claim 1. Control means for performing display control for displaying one or more first frames on the display unit;
The information processing apparatus according to claim 1, wherein the first selection unit selects the selection frame based on the first frame displayed on the display unit. The first selection means selects, as the selection frame, a first frame that is N frames (N is an integer of 1 or more) after the first frame displayed at a predetermined position of the display unit. The information processing apparatus according to claim 6. The first selection means is a scrolling speed when the first frame displayed on the display unit is switched to another frame by scrolling, and transmission when transmitting the second frame from the external device to the information processing device. The information processing apparatus according to claim 7, wherein the value of N is determined based on at least one of speed. The control means obtains at least one of a plurality of frames including the plurality of first frames and the second frame acquired by the acquisition means after the acquisition means acquires the second frame. The information processing apparatus according to claim 6, wherein display control for displaying on the display unit is performed. The image resolution of the second frame is higher than the image resolution of the first frame,
The said acquisition means acquires the 2nd frame after image resolution is reduced to the image resolution substantially the same as the image resolution of the said 1st frame, The any one of Claims 1-9 characterized by the above-mentioned. Information processing device. The information processing apparatus according to claim 1, wherein the acquisition unit acquires a second frame that is included in the specific section and does not correspond to the selected frame. Second selection means for selecting any of a plurality of frames including the plurality of first frames and the second frame acquired by the acquisition means in response to a user operation;
And a deletion unit that automatically deletes a non-selected frame that is the second frame acquired by the acquisition unit and has not been selected by the second selection unit. The information processing apparatus according to claim 1. The said determination means determines the area which does not contain the 2nd frame corresponding to the 1st frame immediately before the said selection frame as said specific area. The information processing apparatus described. The said determination means determines the area which does not contain the 2nd frame corresponding to the 1st frame after the said selection frame as said specific area, The any one of Claims 1-13 characterized by the above-mentioned. The information processing apparatus described. The determining means determines a second frame corresponding to the first frame immediately after the selected frame from the second frame immediately after the second frame corresponding to the first frame immediately before the selected frame. The information processing apparatus according to any one of claims 1 to 14, wherein a section up to the previous second frame is determined as the specific section. A selection step of selecting a selection frame that is one of a plurality of first frames constituting the first moving image;
Based on the frame rate of the first moving image, the frame rate of the second moving image having a higher frame rate than the first moving image corresponding to the first moving image, and a portion of the second moving image based on the selected frame A determining step for determining a specific section of
Obtaining a second frame included in the specific section among a plurality of second frames constituting the second moving image from an external device storing the second moving image;
Have
In the determining step, a section including a second frame corresponding to the selected frame among the plurality of second frames is determined as the specific section.   A program causing a computer to function as each unit of the information processing apparatus according to any one of claims 1 to 15.

Images