JP7389558B2 - Information processing device and its control method and program - Google Patents

Description

The present invention relates to an information processing device, a control method thereof, and a program.

2. Description of the Related Art In recent years, imaging devices such as digital still cameras have been known that are equipped with a shooting mode called a shutter speed priority mode. The shutter speed priority mode is a shooting mode in which, when a user sets a desired shutter speed, the imaging device automatically sets exposure settings other than the shutter speed, such as an aperture value and ISO sensitivity. For example, if the user sets a high shutter speed before shooting and shoots in shutter speed priority mode, it is possible to shoot an image with less motion blur.

When photographing in shutter speed priority mode, it becomes easier to set a desired shutter speed if you can visually check the roughness of blur that will occur in the photographed image. To address this problem, Patent Document 1 discloses a technology that allows the user to easily check the moving area during preparatory shooting by displaying an image with the moving area highlighted on the viewfinder during preparatory shooting. Disclosed. Note that preparatory photography is photography for adjusting the composition and setting photography conditions while looking at the electronic viewfinder or rear LCD of the imaging device.

Japanese Patent Application Publication No. 2008-172667

However, even if you set a high shutter speed while checking the image during preparatory shooting in order to take an image with less motion blur, and then take the actual shot in shutter speed priority mode, the image taken during the actual shooting may not be what you want. be. This is because the blur that occurs during the actual shooting and the preparatory shooting may be different, so even if the user sets the shutter speed that he or she considers optimal during the preparatory shooting, the subject may be blurred as a result of the actual shooting. May be photographed.

In addition, the user may be able to set a more optimal shutter speed by visually checking the image actually obtained by actual shooting. However, simply displaying the actual photographed image poses a problem in that blurring of the subject may not be visible unless the image is enlarged, and confirmation is time-consuming.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to realize a technology that makes it possible to easily confirm motion blur in the actually captured image when the actually captured image is played back and displayed.

In order to solve this problem, for example, the information processing device of the present invention has the following configuration. That is, it is an acquisition means for photographing a subject image and acquiring a first image and a second image that are sequentially output, and at least the second image is recorded on a recording medium as an image of the main photographing. an acquisition means, and converting motion information obtained using the first image and the second image into motion blur in the second image based on an exposure time associated with the second image. and a notifying means for notifying the motion blur of the image based on the converted motion blur in the second image when the second image is displayed. The exposure time associated with the image is an exposure time set after recording the second image.

According to the present invention, when the actually photographed image is played back and displayed, it becomes possible to easily check for motion blur in the actually photographed image.

A block diagram showing an example of the functional configuration of a digital camera as an example of the information processing device of this embodiment Flowchart showing a series of operations of photographing processing in the digital camera of this embodiment A diagram showing an example of a main photographed image and a motion vector according to the present embodiment Flowchart showing a series of operations in the motion vector calculation unit of this embodiment Diagram for explaining an example of a motion vector calculation method according to the present embodiment A block diagram showing an example of the functional configuration of the image generation unit of this embodiment Flowchart showing a series of processing operations in the image generation unit of this embodiment Diagram for explaining an example of motion blur converted to a motion vector Diagram for explaining an example of a motion blur notification method

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the claimed invention. Although a plurality of features are described in the embodiments, not all of these features are essential to the invention, and the plurality of features may be arbitrarily combined. Furthermore, in the accompanying drawings, the same or similar components are designated by the same reference numerals, and redundant description will be omitted.

Below, as an example of an information processing apparatus, an example will be described in which a digital camera capable of recording captured images is used. However, the present embodiment is applicable not only to digital cameras capable of photographing, but also to other devices capable of acquiring photographed images. These devices may include, for example, mobile phones including smartphones, personal computers, game consoles, tablet terminals, watch-shaped or glasses-shaped information terminals, medical devices, monitoring systems, in-vehicle system devices, and the like.

In this embodiment, an example will be described in which the digital camera 100 converts motion blur from motion information obtained from an actually captured image, and reports motion blur when playing back the captured image.

(Digital camera configuration)
FIG. 1 is a block diagram showing an example of a functional configuration of a digital camera 100 as an example of an information processing apparatus according to this embodiment. Note that one or more of the functional blocks shown in FIG. 1 may be realized by hardware such as an ASIC or a programmable logic array (PLA), or may be realized by a programmable processor such as a CPU or MPU executing software. You can. Alternatively, it may be realized by a combination of software and hardware.

The control unit 101 is, for example, a CPU, and reads a control program for each block included in the digital camera 100 from a ROM 102 (described later), expands it to a RAM 103 (described later), and executes it. Thereby, the control unit 101 controls the operation of each block included in the digital camera 100. Further, the control unit 101 performs display control to display a predetermined image on the display unit 109.

The ROM 102 is an electrically erasable/recordable nonvolatile memory such as an EEPROM, and stores operation programs for each block included in the digital camera 100 as well as parameters necessary for the operation of each block. The RAM 103 is a rewritable volatile memory, and is used to temporarily store programs executed by the control unit 101 and the like, and data generated by the operations of each block included in the digital camera 100.

The optical system 104 includes a lens group including a zoom lens and a focus lens, and forms a subject image that has passed through the optical system 104 on an imaging surface of an imaging unit 105, which will be described later. The imaging unit 105 includes an imaging device such as a CCD or a CMOS sensor, photoelectrically converts an optical image formed on the imaging surface of the imaging unit 105 by the optical system 104, and converts the obtained analog image signal into an A/D converter. It is output to the converter 106. The A/D converter 106 converts the input analog image signal into digital image data. Digital image data output from the A/D converter 106 is temporarily stored in the RAM 103.

The image processing unit 107 applies various image processing such as white balance adjustment, color interpolation, and gamma processing to the image data stored in the RAM 103. The image processing unit 107 also includes an image generation unit 600, which will be described later. The image generation unit 600 generates a motion blur notification image in which an image plane that facilitates confirmation of motion blur is superimposed on the image stored in the RAM 103. Further, the image processing unit 107 includes a motion vector calculation unit (not shown) described later, and calculates a motion vector between images acquired from the imaging unit 105 or the recording unit 108.

The recording unit 108 records data on a fixed recording medium such as a semiconductor memory, but the recording medium may be a removable memory card. The recording unit 108 records the image data processed by the image processing unit 107 as a recorded image. The display unit 109 is a display device such as an LCD, and displays images stored in the RAM 103 and images recorded in the recording unit 108. Furthermore, the display unit 109 can display a user interface for accepting instructions from the user. Further, the display unit 109 displays an image captured by the imaging unit 105 for composition adjustment and the like during preparatory shooting. Note that the display unit 109 does not necessarily need to be provided inside the digital camera 100, and may be configured such that the control unit 101 performs display control so as to output display data to the outside for display on an external monitor.

The instruction input unit 110 includes operation members such as a touch panel, switches, dials, and buttons, receives instructions from the user, and notifies the control unit 101 of the received instructions.

(Series of shooting processing operations)
Next, the photographing process in the digital camera 100 will be described with reference to FIG. 2(a). Note that this processing is realized by the control unit 101 loading a program stored in the ROM 102 into the work area of the RAM 103 and executing it, unless otherwise specified. Further, this process is started when the user issues a photography preparation instruction such as composition adjustment via the instruction input unit 110 after the digital camera 100 is started.

In S201, the control unit 101 starts preparatory photography. During this preparatory shooting period, the control section 101 controls each section of the digital camera 100 to capture continuous images and display them on the display section 109. The user of the digital camera 100 can adjust the composition while viewing the preparatory photographed image displayed on the display unit 109. Note that the processes of S202, 203, and 204, which will be described later, are executed during the preparatory shooting period.

In S202, the control unit 101 sets the exposure time for actual shooting. For example, the control unit 101 sets the exposure time during actual shooting based on the user's operation instruction received from the instruction input unit 110. In this embodiment, a case will be explained in which the control unit 101 sets the exposure time for the actual shooting based on the user's operation, but the control unit 101 sets the exposure time for the actual shooting based on the user's operation, for example. It may also be set automatically according to conditions. The control unit 101 causes the set exposure time to be held in the ROM 102 or the recording unit 108, for example.

In S<b>203 , the control unit 101 displays the preparation photographed image acquired by the imaging unit 105 on the display unit 109 . In S204, the control unit 101 determines whether the shutter button has been pressed based on the signal from the instruction input unit 110. That is, the user presses the shutter button at a photo opportunity while looking at the image displayed on the display unit 109, and when the control unit 101 detects the pressing of the shutter button, the process advances to S205. If the control unit 101 determines that the shutter button has not been pressed, the process returns to S203 and continues displaying the preparation image.

In S205, the control unit 101 controls the imaging unit 105 and the like to perform main photography, and records the image captured by the main photography (hereinafter referred to as the main photography image) in the recording unit 108. In S206, the control unit 101 determines whether the setting of the motion blur alert function is ON (that is, enabled) or OFF (disabled). The setting of ON or OFF of the motion blur notification function is set using, for example, the instruction input unit 110, and the set value is stored in the ROM 102 or the recording unit 108, for example. If the control unit 101 determines that the motion blur notification function is ON, the process proceeds to S207; otherwise, the process proceeds to S208 without calculating the motion vector.

In S207, the image processing unit 107 calculates a motion vector for the actual photographed image in accordance with an instruction from the control unit 101. Note that the motion vector calculation process will be described later with reference to FIG. 4. The control unit 101 may record information on the motion vector calculated by the image processing unit 107 in the recording unit 108 in association with the actual photographed image. In S208, the control unit 101 displays the main photographed image photographed in S206 on the display unit 109.

FIG. 3(a) shows an example of the actual photographed image photographed in S205. This example of the actual photographed image shows an image in which a scene of a dog 301 running to the left and a dog 302 standing still is photographed. In calculating the motion vector in S207, for example, as motion information, a motion vector between the preparatory photographed image immediately before the actual photographing and the main photographed image is calculated. If the actual shooting is performed continuously (that is, continuous shooting), the motion vector between images is calculated using the actual shooting image one before or one after the actual shooting image, not the preparation shooting image. You may. Note that the motion vector "between images" in the following description may refer to either a motion vector between the preparatory image and the actual image, or a motion vector between the two actual images. Further, a motion vector represents the amount of movement of the subject in the horizontal direction and the amount of movement in the vertical direction between two images as a vector.

Note that the motion between images obtained as a motion vector represents the motion between images that are sequentially acquired at predetermined time intervals such as 1/60 seconds, so it does not necessarily depend on the exposure time set by the user (for example, 1/60 seconds). 120 seconds). For this reason, in this embodiment, as will be described later, motion vectors are used so that the user can easily understand the magnitude of blurring that occurs during the exposure time used in shooting (or set by the user), for example. Processing is performed to convert the size of , using the exposure time.

Here, with reference to FIG. 4, a series of operations for motion vector calculation processing according to the present embodiment will be described. Note that the motion vector calculation process of this embodiment is executed by, for example, a motion vector calculation unit (not shown) included in the image processing unit 107. Further, in the following explanation, a case will be explained using a block matching method as an example of a motion vector calculation method, but the method is not limited to this example, and for example, a gradient method may be used.

In S401, the motion vector calculation unit acquires two temporally adjacent images that are sequentially output from the imaging unit 105, and uses the previously captured preparatory image or another main image as a reference frame. settings, and set the actual captured image as the reference frame. Note that the image set as the reference frame is also simply referred to as the standard captured image, and the image set as the reference frame is also simply referred to as the reference captured image. In S402, the motion vector calculation unit arranges a reference block 502 of N×N pixels in the reference captured image 501 of the reference frame, as shown in FIG.

In S402, as shown in FIG. 5, the motion vector calculation unit calculates, for the reference captured image 503 of the reference frame, the center coordinates of the reference block 502 of the reference captured image 501 and the periphery of the same coordinates 504 (N+n)×(N+n). A pixel is set as a search range 505.

In S404, the motion vector calculation unit performs a correlation calculation between the reference block 502 of the reference captured image 501 and the reference block 506 of N×N pixels at different coordinates existing within the search range 505 of the reference captured image 503, and calculates the correlation. Calculate the value. The correlation value is calculated based on the sum of absolute differences between frames for pixels of the standard block 502 and the reference block 506. In other words, the coordinate with the smallest inter-frame difference absolute value sum is the coordinate with the highest correlation value. Note that the method for calculating the correlation value is not limited to the method of calculating the sum of absolute values of inter-frame differences, and may be, for example, a method of calculating a correlation value based on the sum of squared inter-frame differences or a normal cross-correlation value. In the example shown in FIG. 5, it is assumed that the reference block 506 has the highest correlation.

In S405, the motion vector calculation unit calculates a motion vector based on the reference block coordinates indicating the highest correlation value determined in S404. For example, the motion vector calculation unit calculates a motion vector based on the coordinates 504 corresponding to the center coordinates of the reference block 502 of the reference captured image 501 and the center coordinates of the reference block 506 within the search range 505 of the reference captured image 503. . That is, the inter-coordinate distance and direction from the same coordinate 504 to the center coordinate of the reference block 506 are determined as a motion vector. The motion vector calculation unit records information on the determined motion vector in the recording unit 108 together with the moving image or music data or alone.

In S406, the motion vector calculation unit determines whether motion vectors have been calculated for all pixels of the reference captured image 501. If the motion vector calculation unit determines in S406 that the motion vectors of all pixels have not been calculated, the process returns to S402. Then, in S402, a reference block 502 of N×N pixels is placed in the reference captured image 501 described above centering on pixels for which motion vectors have not been calculated, and the processes of S402 to S405 are similarly executed. That is, the motion vector calculation unit calculates the motion vectors of all pixels of the reference captured image 501 by repeating the processes from S402 to S405 while moving the reference block 502 shown in FIG. 5 within the reference captured image. An example of a portion of the motion vectors calculated in this manner is shown in FIG. 3(b). FIG. 3(b) is a diagram showing an example of a motion vector of the preparatory photographed image of FIG. 3(a).

The preparatory photographed image in FIG. 3A shows an example in which the dog 301 is running to the left. An example of a motion vector when the subject is moving in this way is shown in FIG. 3(b). In the example of FIG. 3(b), the running dog 301 is represented by a leftward motion vector 303, and the other stationary dogs 302 and the fence in the background do not move, so their motion vectors are not shown. . Note that the motion vector calculation unit may calculate a motion vector for each predetermined pixel instead of calculating the motion vector for all pixels. Through the above processing, the motion vector calculation unit calculates motion vectors between temporally adjacent images.

The operation of the photographing process will be explained with reference to FIG. 2 again. In S209, the control unit 101 determines whether continuous shooting is being performed with the shutter button being pressed down. For example, if the shutter button is pressed based on a notification from the instruction input unit 110, the control unit 101 determines that continuous shooting is being performed. When the control unit 101 determines that continuous shooting is being performed, the control unit 101 returns to the main shooting process in S205 and repeats shooting. If the shutter button is not pressed, the control unit 101 determines that continuous shooting has ended, advances to S210, and waits to enter image playback mode. In this embodiment, it is assumed that after continuous shooting ends, the user uses the instruction input unit 110 to input an instruction to switch the digital camera 100 to the image playback mode.

In S210, the control unit 101 determines whether the digital camera 100 has been switched to image playback mode. For example, the control unit 101 determines whether the digital camera 100 has been switched to the image playback mode based on the signal from the instruction input unit 110, and if it is determined that the switch to the image playback mode has been performed, the control unit 101 proceeds to S211. move on. The control unit 101 repeats the process (that is, waits) until it determines that switching to the playback mode has been performed.

In S211, the control unit 101 determines whether the motion blur notification function is enabled. If the control unit 101 is enabled in S206, the control unit 101 reads the motion blur notification setting and proceeds to S212. If not, the process returns to S211 and repeats the process.

In S212, the image processing unit 107 generates an image (also simply referred to as a motion blur notification image) that shows an area with motion blur superimposed on the main photographed image to be reproduced, in response to an instruction from the control unit 101. Note that the motion blur notification image generation process will be described later.

In S213, the control unit 101 displays the motion blur notification image on the display unit 109 (that is, controls the display on the display unit 109). Note that if the motion blur notification is not enabled, the control unit 101 causes the display unit 109 to display the reproduced actual photographed image. After displaying the image on the display unit 109, the control unit 101 then ends a series of operations related to the photographing process.

Note that in S213, the user may be able to change the exposure time, which will be described later, while the motion blur notification image is being displayed. In this case, upon receiving a user operation to change the exposure time, the control unit 101 returns to S212 and generates a motion blur notification image according to the changed exposure time. Then, in S213, the displayed motion blur notification image is changed to the newly generated motion blur notification image. By doing so, the user can check the motion blur caused by different exposure time settings.

Next, in addition to the operation of playing back and displaying the actually shot image when switching to image playback mode as shown in Figure 2(a), the playback display of the actually shot image will be automatically displayed immediately after the actual shooting. It may also be done only for a predetermined period. The photographing process of the digital camera 100 in this case is, for example, as shown in FIG. 2(b). Note that similar operations and processes shown in FIG. 2(a) are given the same reference numerals and redundant explanations will be omitted.

In FIG. 2(b), the control unit 101 executes the processes of S201 to S205 and enters a state in which actual photographing is performed. In S206, the control unit 101 determines whether the motion blur notification function is enabled. Note that the setting as to whether or not the motion blur notification function is enabled can be set by the user using the instruction input unit 110, for example, and once set, the setting value is stored in the ROM 102 or the recording unit 108, for example. has been done.

In S251, the control unit 101 proceeds to S207 if the motion blur notification function is enabled, and otherwise proceeds to S208 without calculating a motion vector. In S207, the control unit 101 calculates a motion vector for the actual photographed image, and further in S252 generates a motion blur notification image. In S253, the control unit 101 causes the display unit 109 to display the motion blur notification image if a motion blur notification image has been generated, or to display the actual photographed image if a motion blur notification image has not been generated. When the image display is completed, the control unit 101 ends a series of operations related to the photographing process.

(Generation of motion blur notification image)
Next, a configuration and a series of operations for generating the above-mentioned motion blur notification image will be described with reference to FIGS. 6 and 7. FIG. 6 shows an example of a functional configuration of an image generation unit 600 included in the image processing unit 107. The image generation unit 600 includes a motion blur calculation unit 602, a motion blur notification plane generation unit 603, and an image superimposition unit 604. The image generation unit 600 acquires the shooting conditions described later, motion vector information from the motion vector calculation unit, and the actual captured image from the recording unit 108 or the RAM 103, and generates a motion blur notification image by the following generation process. do.

FIG. 7 shows a series of operations in which the image generation unit 600 generates a motion blur notification image. Note that the process of generating the motion blur notification image shown here is realized by the hardware that constitutes the image generation unit 600, or by software that realizes the functions of the image generation unit 600 being executed by one or more processors. be done.

In S701, the image generation unit 600 inputs information on a motion vector obtained when the main captured image is captured. When in the image reproduction mode, the image generation unit 600 inputs information on motion vectors recorded in the recording unit 108 together with the main photographed image. In S702, the motion blur calculation unit 602 acquires the exposure time of the actual shooting set in S202 as the shooting condition from, for example, the ROM 102. Further, the motion blur calculation unit 602 similarly acquires the time interval between images in the main shooting and the preparatory shooting immediately before the main shooting from, for example, the ROM 102. The time interval between images becomes the time interval between successive actual shootings during continuous shooting.

In S703, the motion blur calculation unit 602 converts the motion vector into motion blur for actual shooting. Specifically, the motion blur calculation unit 602 converts the motion vector for each pixel calculated in S207 into the motion blur of the actual imaging using the exposure time of the actual imaging and the time interval between images acquired in S702. Note that the motion vector calculated in S207 represents a motion vector between images sequentially acquired from the imaging unit 105 (for example, at an acquisition interval of 1/60 seconds or at an acquisition interval of consecutively shot images). The motion vector in the actual shooting and the motion blur in the actual shooting will be explained with reference to FIG. 8. FIG. 8 shows the shooting conditions in which the time interval between images in which motion vectors are detected is 1/60 seconds, and the exposure time for main shooting is 1/120 seconds and 1/30 seconds. ing. The motion blur calculation unit 602 converts the motion vector for each pixel into the motion blur of the actual shooting based on the conversion formulas shown in equations (1) and (2), for example.

CONV_GAIN=EXP_TIME/INT_TIME...(1)
CONV_BLUR=VEC_LEN×CONV_GAIN...(2)
Here, in equation (1), CONV_GAIN indicates the conversion gain for converting the motion vector of the preparatory shooting into the motion vector of the main shooting, EXP_TIME indicates the exposure time of the main shooting, and INT_TIME indicates the time interval between images. show. Furthermore, in equation (2), CONV_BLUR indicates the converted motion blur in the actual shooting, and VEC_LEN indicates the length of the motion vector in the actual shooting.

The motion blur calculation unit 602 calculates a conversion gain based on equation (1). That is, the conversion gain is calculated by dividing the exposure time of the actual shooting by the time interval between images. Then, the motion blur calculation unit 602 calculates the motion blur of the actual shooting based on equation (2). That is, the motion blur calculation unit 602 calculates the motion blur of the actual shooting by multiplying the magnitude of the motion vector by the conversion gain.

For example, if the size of the motion vector in the actual shooting is 10 pixels as shown in FIG. becomes. Further, for motion blur when the exposure time of the actual photographing is 1/30 seconds, the converted gain is doubled, so the number of pixels is 20.

Note that, as described above, when the user switches the exposure time setting in the image playback mode, EXP_TIME can be set to the exposure time that the user sets when playing back the image. By calculating the conversion gain using the exposure time set during playback of the actual shot image, the amount of motion blur when changing the exposure time can be displayed during image playback, allowing you to shoot with the desired amount of motion blur. You will be able to check the exposure time for

In S704, the motion blur notification plane creation unit 603 creates an image plane for reporting motion blur based on the motion blur for each pixel calculated in S703. In S705, the image superimposition unit 604 superimposes the motion blur notification plane created in S704 on the main photographed image to generate a motion blur notification image. Here, an example of a motion blur notification image will be described with reference to FIG. 9. FIGS. 9A to 9C show examples of three motion blur notification images in this embodiment. By displaying the motion blur notification image on the display unit 109 during image playback, the user can easily confirm motion blur in the actually captured image.

FIG. 9A shows an example in which motion blur is reported by adding an icon to the actually photographed image as a motion blur notification image. Here, an example will be described in which a motion blur notification image is generated by adding an icon display. In S704, the motion blur notification plane generation unit 603 calculates the ratio of the number of pixels in which the converted motion blur is equal to or greater than a predetermined value to the entire screen, among the motion blur for each pixel. The motion blur notification plane creation unit 603 generates an icon indicating the existence of motion blur exceeding a reference value, in response to the fact that the proportion of the entire image in which the converted motion blur has a number of pixels equal to or greater than a predetermined value is equal to or greater than a predetermined proportion. Create a motion blur notification plane. The icon indicating the presence of motion blur greater than the reference value may be, for example, a motion blur icon 901 as shown in FIG. 9(a). Then, the image superimposing unit 604 generates a motion blur notification image as shown in FIG. 9(a) by superimposing the motion blur notification plane on the main photographed image. On the other hand, the motion blur notification plane generation unit 603 does not create a motion blur notification plane if the proportion of the entire image having the number of pixels in which the converted motion blur is equal to or greater than a predetermined value is lower than the predetermined proportion.

FIG. 9B shows an example in which motion blur is reported by displaying a motion blur frame in the main photographed image as a motion blur notification image. Here, a method of generating a motion blur notification image by displaying a motion blur frame will be described. In S704, the motion blur notification plane generation unit 603 calculates the ratio of the number of pixels, of which the converted motion blur is equal to or greater than a predetermined value, to the entire divided region among the motion blur for each pixel in the divided region. The motion blur notification plane creation unit 603 determines whether a motion blur of a reference value or more exists for a divided region in which the ratio of the converted motion blur to the entire divided region of the number of pixels of which the converted motion blur is a predetermined value or more is a predetermined percentage or more. A frame indicating the area is created as a motion blur notification plane. The frame indicating the area where motion blur greater than the reference value exists may be, for example, a motion blur frame 902 shown in FIG. 9(b). The image superimposition unit 604 generates a motion blur notification image as shown in FIG. 9(b) by superimposing the motion blur notification plane on the main photographed image. On the other hand, the motion blur notification plane creation unit 603 does not create a motion blur notification plane if the proportion of the entire divided area with the number of pixels in which the converted motion blur is equal to or greater than the predetermined value is lower than the predetermined ratio.

FIG. 9C shows an example in which motion blur is reported by emphasizing an edge where motion blur has occurred on the main photographed image as a motion blur notification image. Here, a method for generating a motion blur notification image by emphasizing motion blur edges will be described. In S704, the motion blur notification plane generation unit 603 detects the edge strength within the actual captured image. Since a known method such as a Sobel filter can be used to calculate the edge strength, a description thereof will be omitted. The motion blur notification plane generation unit 603 extracts pixels for which the detected edge strength is greater than or equal to a predetermined value and the converted motion blur is greater than or equal to a predetermined value. The motion blur notification plane generation unit 603 creates a motion blur notification plane with a highlighted display indicating an edge where a motion blur of a reference value or more exists for the extracted pixel. The motion blur notification plane creation unit 603 creates a motion blur notification plane that highlights a motion blur edge 903 as shown in FIG. 9C, for example. The image superimposition unit 604 generates a motion blur notification image as shown in FIG. 9C by superimposing a motion blur notification plane in which the motion blur edge is highlighted on the main photographed image. The example of a motion blur edge 903 shown in FIG. 9C shows an example in which edges with motion blur are emphasized thickly. Another example of a highlighting method is to extract pixels whose edge strength is greater than a predetermined value and whose converted motion blur is greater than a predetermined value, and to add a predetermined color to the extracted pixels (for example, paint them red). An example of this is highlighting. When the image generation unit 600 outputs the motion blur notification image, it ends the series of operations of the motion blur notification image generation process.

In addition, in the above-mentioned example, an example was described in which a motion blur is notified when the converted motion blur is greater than or equal to a predetermined value. However, if the converted motion blur is less than or equal to a predetermined value, the motion blur may be notified. In this way, in the case of long exposure photography in which motion blur is desired to be expressed as a sense of motion, it becomes easier to confirm the lack of motion blur during image reproduction.

In addition, in the above example, an example was explained in which a motion blur notification image is displayed using one method of notifying motion blur, but the user can switch between the methods of notifying motion blur shown in FIGS. 9(a) to 9(c). It may be possible to do so. In this way, the user can confirm motion blur using a desired notification method according to the set exposure time.

In the above example, three examples of methods for notifying motion blur were described: display of a motion blur icon, display of a motion blur frame, and highlighted display of a motion blur edge. However, the method of reporting motion blur is not limited to this. For example, areas where motion blur occurs may be highlighted, including flat areas. Specifically, the motion blur notification plane generation unit 603 performs highlighted display such as painting pixels in which the converted motion blur for each pixel is equal to or greater than a predetermined value in red. In this way, by highlighting not only the edge area but also areas other than the edge area, the entire subject is highlighted, making it easier to confirm motion blur.

Furthermore, in the above example, as a method for notifying motion blur, an example was explained in which information for notifying motion blur is displayed on the display unit 109. However, the method for notifying motion blur is not limited to this. For example, motion blur may be reported using sound. Specifically, out of the motion blur converted for each pixel, if the number of pixels for which the converted motion blur is equal to or greater than a predetermined value occupies a predetermined percentage or more of the entire screen, a motion blur notification sound is generated. You can.

As explained above, in this embodiment, the image of the actual shooting and other images are acquired sequentially from the imaging unit 105, and the acquired images are used based on the exposure time associated with the actual shooting image. The obtained motion information is converted into motion blur during actual shooting. When displaying the actual photographed image, the motion blur of the image is notified based on the determined motion blur in the actual photographic image. By doing so, when the actually photographed image is played back and displayed, it becomes possible to easily check for motion blur in the actually photographed image. Particularly, when the user reproduces and displays the actually captured image, the user does not have to worry about enlarging the actually captured image to check the degree of blur. In addition, since you can check the movement blur according to the exposure time set when shooting the actual image or when playing back, it is possible to accurately and easily understand the blur in the actual image. becomes possible.

(Other embodiments)
The present invention provides a system or device with a program that implements one or more functions of the embodiments described above via a network or a storage medium, and one or more processors in a computer of the system or device reads and executes the program. This can also be achieved by processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the embodiments described above, and various changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, the following claims are hereby appended to disclose the scope of the invention.

DESCRIPTION OF SYMBOLS 101... Control part, 105... Imaging part, 109... Display part, 300... Motion blur notification image generation part, 302... Motion blur calculation part, 303... Motion blur notification plane creation part, 304... Image superimposition part

Claims (12)

Acquisition means for photographing a subject image and acquiring a first image and a second image that are sequentially output, wherein at least the second image is recorded on a recording medium as an image of main photography. and,
a conversion means for converting motion information obtained using the first image and the second image into motion blur in the second image based on an exposure time associated with the second image; ,
When displaying the second image, a notification means for notifying the motion blur of the image based on the converted movement blur in the second image,
An information processing apparatus characterized in that the exposure time associated with the second image is an exposure time set after recording the second image. The notifying means is configured to reduce the motion blur of the image by superimposing a display notifying the motion blur on the second image in response to the motion blur in the converted second image satisfying a predetermined condition. The information processing device according to claim 1, wherein the information processing device notifies the information processing device. 3. The information processing apparatus according to claim 2, wherein the display notifying the motion blur includes an icon indicating the presence of motion blur greater than a reference value. 3. The information processing apparatus according to claim 2, wherein the display notifying the motion blur includes a frame indicating an area where motion blur greater than a reference value exists. 3. The information processing apparatus according to claim 2, wherein the display notifying the motion blur includes a highlighted display indicating an edge where motion blur of a reference value or more exists. The notifying means notifies the motion blur of the image by generating a sound notifying the motion blur in response to the converted motion blur of the second image satisfying a predetermined condition. The information processing device according to any one of claims 1 to 5. The conversion means converts the motion information into the second image using a ratio of an exposure time associated with the second image to a time interval between the first image and the second image. 7. The information processing apparatus according to claim 1, wherein the information processing apparatus converts motion blur into motion blur. The motion information obtained using the first image and the second image is recorded on the recording medium,
The conversion means converts the motion information recorded on the recording medium into motion blur in the second image based on an exposure time associated with the second image. The information processing device according to any one of Items 1 to 7. The information processing apparatus according to any one of claims 1 to 8, wherein the first image and the second image are images of main photography output by continuous photography. The information processing apparatus according to any one of claims 1 to 9, wherein the first image is an image output by preparatory photography. an acquisition step in which the acquisition means photographs a subject image and acquires a first image and a second image that are sequentially output, and at least the second image is recorded on a recording medium as an image of the main photography. an acquisition process;
A conversion means converts motion information obtained using the first image and the second image into motion blur in the second image based on an exposure time associated with the second image. a conversion process to
a notification step in which the notification means, when displaying the second image, notifies the motion blur of the image based on the converted movement blur in the second image;
A method for controlling an information processing apparatus, wherein the exposure time associated with the second image is an exposure time set after recording the second image. A program for causing a computer to function as each means of the information processing apparatus according to claim 1.

Images