Nothing Special   »   [go: up one dir, main page]

WO2019188933A1 - Dispositif d'imagerie, procédé d'imagerie, et système de caméra - Google Patents

Dispositif d'imagerie, procédé d'imagerie, et système de caméra Download PDF

Info

Publication number
WO2019188933A1
WO2019188933A1 PCT/JP2019/012410 JP2019012410W WO2019188933A1 WO 2019188933 A1 WO2019188933 A1 WO 2019188933A1 JP 2019012410 W JP2019012410 W JP 2019012410W WO 2019188933 A1 WO2019188933 A1 WO 2019188933A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
mode
unit
imaging
exposure
Prior art date
Application number
PCT/JP2019/012410
Other languages
English (en)
Japanese (ja)
Inventor
祐樹 杉原
小林　潤
一樹 石田
真彦 宮田
Original Assignee
富士フイルム株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 富士フイルム株式会社 filed Critical 富士フイルム株式会社
Priority to JP2020510050A priority Critical patent/JP7015906B2/ja
Publication of WO2019188933A1 publication Critical patent/WO2019188933A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B15/00—Special procedures for taking photographs; Apparatus therefor
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B17/00—Details of cameras or camera bodies; Accessories therefor
    • G03B17/02—Bodies
    • G03B17/17—Bodies with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B5/00—Adjustment of optical system relative to image or object surface other than for focusing
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B7/00—Control of exposure by setting shutters, diaphragms or filters, separately or conjointly
    • G03B7/08—Control effected solely on the basis of the response, to the intensity of the light received by the camera, of a built-in light-sensitive device
    • G03B7/091—Digital circuits
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene

Definitions

  • the present invention relates to an image pickup apparatus, an image pickup method, and a camera system, and more particularly to an image pickup apparatus, an image pickup method, and a camera system that obtain an effect of shooting by long exposure.
  • Patent Document 1 describes switching between a first mode in which continuous exposure is performed and a second mode in which a plurality of images are acquired and combined according to a shutter speed (exposure time). In the first mode, image stabilization control (camera shake correction) is performed by driving the shift lens, and in the second mode, normal image stabilization control is performed.
  • the conventional technique does not set an appropriate shooting mode according to the shooting conditions, and an image having an effect of long exposure cannot be obtained with good image quality.
  • the present invention has been made in view of such circumstances, and an object of the present invention is to provide an imaging device, an imaging method, and a camera system that can obtain an image having a long-time exposure effect with good image quality.
  • an imaging apparatus includes an imaging unit that acquires an image of a subject using an imaging element on which an optical image of the subject is formed by a shooting lens, and an imaging unit that acquires the image.
  • a determination unit that determines whether or not whiteout occurs in an image to be performed, a first mode in which a plurality of images are intermittently acquired from the image sensor, and images are continuously acquired from the image sensor
  • a mode setting unit that sets one of the second mode and the first mode when it is determined that whiteout occurs, and when it is determined that whiteout does not occur, the first mode is set.
  • a mode setting unit that sets the mode of 2
  • a detection unit that detects movement of the subject in the image
  • a plurality of images acquired in the first mode are aligned according to the movement of the subject detected by the detection unit, Multiple aligned It comprises an image generator which synthesizes the images to generate a composite image.
  • the imaging device When it is determined that whiteout occurs in an acquired image, the imaging device according to the first aspect sets the first mode in which a plurality of images are intermittently acquired from the imaging element to the imaging unit and is intermittent When a plurality of images are obtained by exposure, a composite image is generated. On the other hand, if it is determined that whiteout does not occur, a second mode in which continuous exposure is performed is set in the imaging unit, and the image is displayed. get. In the first mode, when the determination unit determines that “overexposure occurs”, the occurrence of overexposure is prevented by setting the exposure time and the number of exposures for intermittent image acquisition in the first mode. Can do.
  • the image generation unit aligns the plurality of images according to the movement of the subject (for example, aligns the main subject by movement and / or rotation) and combines them, thereby effecting long exposure. It is possible to obtain a composite image that exhibits (background flow, subject trajectory, etc.). When generating the composite image, the image generation unit may perform other image processing such as blurring processing to enhance the effect of long exposure. On the other hand, when the determination unit determines that “the whiteout does not occur”, since the image is acquired by the continuous exposure in the second mode, an image having an effect corresponding to the exposure time can be obtained. As described above, according to the first aspect, it is possible to obtain the effect of long-time exposure with good image quality that does not cause whiteout.
  • “out-of-white” means a state in which the output of the image sensor is saturated and the luminance difference of the subject cannot be expressed at all or almost in an image (originally a portion having a difference in luminance) However, it can be said that all have the maximum luminance or a luminance close to the maximum luminance).
  • “intermittently acquiring a plurality of images” includes a mode in which an image sensor is intermittently exposed and a plurality of images corresponding to individual exposures are acquired, and an image sensor is continuously exposed. , (By an electronic shutter or the like) the image may be read intermittently from the image sensor, and a plurality of images corresponding to individual reading may be acquired.
  • the imaging apparatus according to the first aspect can be applied to the camera body of a lens-integrated camera, the body part of a lens interchangeable camera, the body part of a surveillance camera, and the like, but is not limited to these examples.
  • the imaging device further includes an exposure control unit that controls exposure with a preset exposure time in the first aspect, and the determination unit includes the first exposure time set by the exposure control unit. If it is greater than or equal to the threshold value, it is determined that whiteout occurs, and the mode setting unit is caused to set the first mode.
  • the second mode prescribes a specific example of the determination of overexposure, and the determination unit has a long exposure time (initial exposure time) set in advance (specifically, the initial exposure time ⁇ the first exposure time). In the case of the threshold value), it is determined that whiteout occurs, and the mode setting unit sets the first mode based on the determination result.
  • the first threshold value may be set according to the type of subject and / or the effect (content, degree) to be obtained. Further, the first threshold value may be set according to the user's designation, or may be set by the imaging apparatus regardless of the user's designation.
  • the determination unit makes a determination at the start of exposure.
  • the determination is made when the operation for shifting to the long-time exposure mode is performed.
  • the present invention is not limited to these modes. However, the present invention is not limited to these modes.
  • the imaging device is any one of the first to third aspects, wherein the imaging unit determines the number of images and / or the number of pixels acquired in the first mode according to the exposure time of the image.
  • the fourth aspect is for reducing the load on the imaging device when acquiring, processing, and synthesizing a plurality of images.
  • the imaging unit has an exposure time (initial exposure time) set to obtain a desired effect. ) Is long and the load is high, the number of images to be taken and combined and / or the number of pixels can be reduced. In addition to these measures, it is also preferable to set the exposure time (multiple exposure time) of each image acquired intermittently according to the initial exposure time.
  • the imaging device further includes an information presentation unit that presents information that prompts a change in the shooting direction according to the detection result, and the detection unit includes: The moving direction of the subject in the image is detected.
  • the user can easily perform the photographing direction changing operation (pan and / or tilt) with reference to the presented information.
  • Information can be presented by screen display, audio output, or the like.
  • the imaging device further includes a photographing direction changing unit that automatically changes the photographing direction following the movement of the subject
  • the mode setting unit includes: When the focal length of the photographing lens when obtaining the image is longer than the second threshold value, the image is obtained while changing the photographing direction by the photographing direction changing unit.
  • the shooting direction changing unit By automatically changing the shooting direction by the shooting direction changing unit, it is possible to reduce the burden of pan / tilt operation during exposure for the user. Since the effect (the degree of background flow, etc.) due to the change in the shooting direction becomes higher when the focal length is long, the shooting direction changing unit is used in the sixth mode (focal length> second threshold). I was going to do the shooting.
  • the effect of the pan / tilt operation during exposure can be obtained efficiently.
  • the configuration of the shooting direction changing unit include a mode of changing the direction of the lens and / or the image sensor, and a mode of changing the shooting direction by driving an optical member such as a mirror. It is not limited to.
  • the imaging device notifies the user which mode the mode setting unit has set between the first mode and the second mode. And a notification unit.
  • the user can recognize the set mode. Notification to the user can be performed by screen display, light emission, vibration, voice output, or the like.
  • the imaging device further includes a storage unit that stores the composite image generated by the image generation unit in any one of the first to seventh aspects.
  • the composite image stored in the storage unit may be output by screen display, communication, printing, or the like.
  • an imaging method provides imaging in an imaging apparatus including an imaging unit that acquires an image of a subject with an imaging element on which an optical image of the subject is formed by a photographing lens.
  • the ninth aspect it is possible to obtain an image having a long-time exposure effect with good image quality as in the first aspect.
  • the imaging method according to the ninth aspect may further have the same configuration as the second to eighth aspects.
  • a program that causes an imaging apparatus and / or camera system to execute the imaging method of these aspects, and a non-transitory recording medium that records a computer-readable code of such a program can also be cited as aspects of the present invention.
  • a camera system according to a tenth aspect of the present invention includes an imaging device according to any one of the first to eighth aspects, and a photographing lens. Since the camera system according to the tenth aspect includes the imaging device according to any one of the first to eighth aspects, an image that exhibits the effect of long exposure can be obtained with good image quality.
  • the camera system according to the tenth aspect can be applied to a lens integrated camera, a lens interchangeable camera, a surveillance camera, and the like, but is not limited to these specific examples.
  • an image that exhibits the effect of long exposure can be obtained with good image quality.
  • FIG. 1 is a diagram illustrating a configuration of a camera system according to the first embodiment.
  • FIG. 2 is a diagram illustrating a functional configuration of the image processing unit.
  • FIG. 3 is a diagram illustrating how the shooting direction is changed.
  • FIG. 4 is another diagram showing a state of changing the shooting direction.
  • FIG. 5 is an overall flowchart of the imaging method.
  • FIG. 6 is a table showing the relationship between shooting conditions and shooting modes.
  • FIG. 7 is an individual flowchart of the imaging method.
  • FIG. 8 is a diagram illustrating a display example of information that prompts the user to change the shooting direction.
  • FIG. 9 is a diagram illustrating how a composite image is generated.
  • FIG. 10 is another diagram showing how a composite image is generated.
  • FIG. 9 is a diagram illustrating how a composite image is generated.
  • FIG. 11 is still another view showing a state of generating a composite image.
  • FIG. 12 is another flowchart of the imaging method.
  • FIG. 13 is still another individual flowchart of the imaging method.
  • FIG. 14 is still another individual flowchart of the imaging method.
  • FIG. 15 is a diagram illustrating an example of the effect of multiple exposure and long exposure.
  • FIG. 1 is a diagram illustrating a configuration of a camera system 10 (an imaging apparatus, a camera system) according to the first embodiment.
  • the camera system 10 includes an interchangeable lens 100 (imaging lens, imaging unit) and an imaging apparatus main body 200 (imaging apparatus), and forms a subject image (optical image) on the imaging element 210 by a photography lens including a zoom lens 110 described later.
  • the interchangeable lens 100 and the imaging apparatus main body 200 can be attached and detached via a mount (not shown).
  • the interchangeable lens 100 includes a zoom lens 110 (zoom lens, photographic lens), a focus lens 120 (photographic lens), a diaphragm 130, a lens driving unit 140, and a variable apex angle prism 150.
  • the lens driving unit 140 performs zoom (optical zoom) adjustment and focus adjustment by driving the zoom lens 110 and the focus lens 120 forward and backward according to a command from the image processing device 240 (exposure control unit 240A in FIG. 2).
  • the zoom adjustment and the focus adjustment may be performed according to a zoom operation and a focus operation performed by the user (a zoom ring not shown, a rotation of the focus ring, etc.) in addition to being performed according to a command from the image processing apparatus 240. Good.
  • the lens driving unit 140 controls the diaphragm 130 in accordance with a command from the image processing device 240 to adjust the exposure.
  • information such as the positions of the zoom lens 110 and the focus lens 120 and the degree of opening of the diaphragm 130 are input to the image processing device 240.
  • the interchangeable lens 100 has an optical axis L1.
  • the imaging apparatus main body 200 includes an imaging element 210 (imaging unit), an AFE 220 (AFE: Analog Front End, imaging unit), an A / D converter 230 (A / D: Analog to Digital, imaging unit), and an image processing device 240.
  • the image sensor 210 includes a light receiving surface on which a large number of light receiving elements are arranged in a matrix. Then, the subject light that has passed through the variable apex angle prism 150, the zoom lens 110, the focus lens 120, and the stop 130 is imaged on the light receiving surface of the image sensor 210 and is converted into an electric signal by each light receiver.
  • An R (red), G (green), or B (blue) color filter is provided on the light receiving surface of the image sensor 210, and a color image of the subject can be acquired based on the signals of each color.
  • various photoelectric conversion devices such as a complementary metal-oxide semiconductor (CMOS) and a charge-coupled device (CCD) can be used.
  • CMOS complementary metal-oxide semiconductor
  • CCD charge-coupled device
  • the AFE 220 performs noise removal and amplification of the analog image signal output from the image sensor 210, and the A / D converter 230 converts the captured analog image signal into a digital image signal having a gradation width.
  • FIG. 2 is a diagram illustrating a functional configuration of the image processing apparatus 240 (image processing apparatus).
  • the image processing apparatus 240 includes an exposure control unit 240A (exposure control unit), a determination unit 240B (determination unit), a mode setting unit 240C (mode setting unit), a detection unit 240D (detection unit), and an image generation unit 240E (image generation unit). ), An information presentation unit 240F (information presentation unit), an imaging direction changing unit 240G (imaging direction changing unit), and an informing unit 240H (informing unit), and based on the digital image signal input from the A / D converter 230. Thus, processing such as generation of a composite image is performed. Details of the processing by the image processing apparatus 240 will be described later.
  • the function of the image processing apparatus 240 can be realized by using various processors.
  • the various processors include, for example, a CPU (Central Processing Unit) that is a general-purpose processor that executes various types of functions by executing software (programs).
  • the above-mentioned various processors include programmable logic devices that are processors whose circuit configuration can be changed after manufacture, such as GPU (Graphics Processing Unit) and FPGA (Field Programmable Gate Array) that are specialized for image processing. (Programmable Logic Device: PLD) is also included.
  • the above-mentioned various processors include dedicated electric circuits that are processors having a circuit configuration designed exclusively for executing specific processing such as ASIC (Application Specific Specific Integrated Circuit).
  • ASIC Application Specific Specific Integrated Circuit
  • each unit may be realized by a single processor, or may be realized by a plurality of processors of the same or different types (for example, a plurality of FPGAs, a combination of CPU and FPGA, or a combination of CPU and GPU).
  • a plurality of functions may be realized by one processor.
  • configuring a plurality of functions with one processor first, as represented by a computer such as an image processing apparatus main body and a server, one processor is configured with a combination of one or more CPUs and software. There is a form in which this processor is realized as a plurality of functions.
  • SoC system-on-chip
  • IC integrated circuit
  • a processor (computer) readable code of the software to be executed is stored in a non-temporary recording medium such as a ROM (Read Only Memory).
  • a non-temporary recording medium such as a ROM (Read Only Memory).
  • the software stored in the non-temporary recording medium includes a program for executing image input, zoom processing, composition processing, and the like.
  • the code may be recorded on a non-temporary recording medium such as various magneto-optical recording devices and semiconductor memories instead of the ROM.
  • RAM Random Access Memory
  • EEPROM Electrically Erasable Memory and Programmable Read Only Memory
  • the image processing apparatus 240 includes a ROM 242 (ROM: Read Only Memory, non-temporary recording medium) in addition to the above-described units.
  • the ROM 242 stores computer-readable codes of programs (including a program for executing the imaging method according to the present invention) necessary for image input, zoom processing, composition processing, and the like.
  • the operation unit 250 includes a release button (not shown), operation buttons (for example, a cross button, a Quick button, an OK button, etc.), a dial, a switch, and the like. Various operations such as setting contents and / or degree can be performed.
  • the monitor 270 may be configured as a touch panel and used as the operation unit 250.
  • the storage unit 260 includes various magneto-optical recording media, non-temporary recording media such as a semiconductor memory, and a control circuit thereof, and stores captured images, synthesized images, and the like.
  • the recording medium a type that can be attached to and detached from the imaging apparatus main body 200 can be used.
  • the monitor 270 (display device) is composed of, for example, a liquid crystal display panel, and can display a live view image, a captured image, a composite image, shooting assistance information, and the like.
  • the monitor 270 can be disposed on the back side, the top side, or the like of the imaging apparatus main body 200.
  • the finder 280 is also composed of, for example, a liquid crystal display panel, a prism, a lens, and the like, and the user can visually recognize a live view image, a captured image, a composite image, shooting assistance information, and the like via an eyepiece unit (not shown).
  • an “optical view finder (OVF)”, an “electronic view finder (EVF)”, or a combination thereof, a “hybrid view finder (HVF)” is shown. Can be used.
  • variable apex angle prism 150 (photographing direction polarization unit) is configured by enclosing a liquid with a high refractive index between two transparent plate glasses arranged in the front and rear in the optical axis direction. Is attached to the side).
  • the lens driving unit 140 extends and contracts the bellows-like expansion / contraction unit 152 provided on the side surface to change the inclination of the liquid, thereby changing the refraction angle of the subject light (ie, the shooting direction) Can be made.
  • 3A shows a state in which the optical axis L1 has not changed, and FIG.
  • 3B shows a state in which the optical axis has changed to the optical axis L2 in the downward direction due to the expansion / contraction of the expansion / contraction part.
  • the direction of an optical axis can be changed to 2 axis
  • an optical element such as the variable apex angle prism 150 may be provided not at the distal end portion of the interchangeable lens 100 but at the proximal end portion.
  • ⁇ Changing the shooting direction can also be done by driving a mirror instead of a variable apex angle prism.
  • two mirrors 160 are arranged facing each other as shown in FIG. 4A, and the directions of these mirrors 160 are changed as shown in FIG. 4B (command from the imaging direction changing unit 240G).
  • the lens driving unit 140 is driven
  • the direction of the optical axis that is, the photographing direction
  • part (b) of FIG. 4 shows the optical axis L3 changed downward in the figure).
  • the direction of the optical axis can be changed around the two axes by rotating the mirror 160 around the two axes.
  • Photographing by changing the orientation of the image sensor with a device such as a piezoelectric element instead of, or in addition to, providing the interchangeable lens 100 with members and / or mechanisms such as the variable apex angle prism 150 and the mirror 160 described above.
  • the direction may be changed.
  • FIG. 5 is a flowchart illustrating the imaging method according to the first embodiment.
  • the determination unit 240B has a set exposure time (hereinafter referred to as “initial exposure time”) as a first threshold. It is determined whether the value is equal to or greater than the value (step S100: determination step).
  • the exposure control unit 240A may set the initial exposure time according to the user's operation, or may set it without depending on the user's operation.
  • the exposure control unit 240A can set the initial exposure time according to the content of the effect to be obtained (for example, the background and the subject appear to flow) and the degree thereof.
  • the determination unit 240B determines that “whiteout occurs” and proceeds to step S110. In this case, as will be described in detail later, a first mode (a mode for shooting by multiple exposure) is set. On the other hand, when the initial exposure time is less than the first threshold value, the determination unit 240B determines that “no whiteout occurs” and proceeds to step S120. In this case, as will be described in detail later, a second mode (a mode in which shooting is performed by continuous exposure) is set.
  • out-of-white means “a state in which the output of the image sensor 210 is saturated and the luminance difference of the subject cannot be represented at all or almost in the image (originally the portion where the luminance is different is the highest luminance or the highest luminance). In a state where the luminance is close to).
  • the first threshold value can be about several tens of seconds (for example, 60 seconds), but is not limited to this value.
  • the first threshold value is changed according to conditions such as the content and / or degree of the effect to be obtained and the type of subject. Also good. By setting the first threshold value as described above, whiteout can be prevented and an image with good image quality can be obtained.
  • step S100 The determination of whether or not whiteout occurs in step S100 (specifically, determination whether or not the exposure time is equal to or greater than the first threshold value) is made at the start of exposure.
  • the long exposure mode Judgment is made when AE conditions (AE: Automatic Exposure, automatic exposure control) and AF conditions (AF: Automatic Focus, automatic focus control) are determined in response to a shooting preparation instruction by operating a shutter button (not shown) after shifting to
  • AE conditions AE: Automatic Exposure, automatic exposure control
  • AF Automatic Focus, automatic focus control
  • the present invention is not limited to these modes, but may include a case in which a determination is made before actual exposure is started after a shooting instruction is given by operating a shutter button (not shown).
  • the mode is set in consideration of the focal length of the interchangeable lens 100 in addition to determining whether or not whiteout occurs based on the exposure time. Specifically, when it is determined that “whiteout occurs” (YES in step S100), mode setting unit 240C determines whether or not the focal length of interchangeable lens 100 is equal to or less than the second threshold value. If the determination is affirmative (YES in step S110), the process proceeds to step S200 to set the first mode (1). If the determination in step S110 is negative, the process proceeds to step S300 to set the first mode (2).
  • step S110 when the determination in step S110 is affirmative (focal length ⁇ second threshold value), “first mode (1)” is set in which shooting is performed by multiple exposure and the shooting direction is not automatically changed.
  • Step S200 Mode setting step.
  • step S110 if the determination in step S110 is negative (focal length> second threshold value), the amount of movement of the subject by driving the variable apex angle prism 150 is large and effective shooting is possible.
  • the unit 240G drives the variable apex angle prism 150 via the lens driving unit 140, and sets “first mode (2)” in which shooting is performed with multiple exposure while changing the shooting direction (step S300: mode setting). Process).
  • the threshold for the focal length (the above-mentioned “second threshold”) is, for example, 35 mm or more and 50 mm when the size of the image sensor is converted to “35 mm full size” (so-called “35 mm conversion”). However, the value is not limited to this value.
  • the mode setting unit 240C determines whether or not the focal length of the interchangeable lens 100 is equal to or less than the second threshold value. Is determined (step S120). When the determination is affirmed, the process proceeds to step S400, and the mode setting unit 240C sets the second mode (1) for photographing by long-time exposure (continuous exposure). On the other hand, if the determination in step S110 is negative, the process proceeds to step S500, and the mode setting unit 240C causes the shooting direction changing unit 240G to drive the variable apex angle prism 150 via the lens driving unit 140 and change the shooting direction. A second mode (2) is set for performing photographing with long exposure (continuous exposure).
  • Fig. 6 shows a table summarizing the mode settings according to the exposure time and focal length threshold values.
  • step S600 Shooting and image processing are performed according to the mode set in steps S200, S300, S400, and S500 (in FIG. 5, the whole is collectively described as “step S600”). Hereinafter, specific processing in each mode in step S600 will be described.
  • FIG. 7 shows a process when the first mode (1) is set in step S200 of FIG.
  • the notification unit 240H notifies the user that the first mode (1) has been set by displaying on the monitor 270 and / or the viewfinder 280 (step S210: notification). Process). You may alert
  • the exposure time initial exposure time
  • the first threshold value so that whiteout occurs. Therefore, in the first mode (1), by performing photographing (intermittent image acquisition, multiple exposure) with an exposure time shorter than the “initial exposure time” (hereinafter referred to as “multiple exposure time”) a plurality of times, The effect of long exposure is obtained while preventing whiteout. For example, when it is determined that “whiteout occurs” at an initial exposure time of 10 seconds, it is conceivable that imaging with a multiple exposure time of 0.5 seconds is repeated 10 times within the initial exposure time.
  • the exposure control unit 240A (imaging unit) takes into consideration the load on the system in addition to the effect to be obtained (corresponding to the initial exposure time), the number of images acquired (number of frames), the multiple exposure time, and the number of pixels.
  • Is set step S220: exposure time setting step
  • the image sensor 210 image pickup unit
  • step S230 shooting step
  • the detection unit 240D detects the subject and its movement from the captured image (step S240: detection step), and the information presentation unit 240F presents information that prompts the user to change the shooting direction according to the detection result (step S250: information presentation).
  • the information presentation unit 240 ⁇ / b> F can perform information presentation by displaying a symbol indicating a change in the photographing direction (pan and / or tilt direction) on the display area 1000 of the finder 280. .
  • the arrow 1020 facing left is displayed among the eight arrows 1010 and 1020 in total (the arrow that is actually displayed is shown by a solid line, and the other is shown by a dotted line).
  • Such display may be performed on the monitor 270 instead of the finder 280.
  • the length of the arrow may be changed according to the change amount of the shooting direction.
  • sound may be output by a speaker (not shown) (for example, “pan left”).
  • the detection unit 240D can detect the moving direction and the moving amount of the subject by, for example, calculating the motion vector by comparing the positions of the main subjects between images with different shooting timings.
  • the exposure control unit 240A, the detection unit 240D, and the information presentation unit 240F perform the control from step S230 to S250 (control for acquiring a plurality of images intermittently from the image sensor 210; multiple exposure) for the number of shots N (step S260). Until YES, and the process proceeds to step S270.
  • FIG. 9 shows an example of an image (photographed image) obtained by each exposure (photographing) in step S230 described above.
  • captured images 1031, 1032, and 1033 are images that have different shooting ranges due to movement of subjects 901, 902, and 903 (main subjects), and the image generation unit 240 E combines these captured images 1031, 1032, and 1033.
  • An image is generated (step S270: image generation step). An example of generating a composite image in step S270 will be described below.
  • the image generation unit 240E moves, rotates, enlarges, or reduces each image to match the subjects 901, 902, and 903, and synthesizes a plurality of aligned images to generate a temporary composite image 1100.
  • the temporary composite image 1100 is a panoramic composite image.
  • the image generation unit 240E cuts out a plurality of images with different cutout ranges from the temporary composite image 1100, such as the images 1101 to 1105 in FIG. By changing the clipping range, the effect of moving the subject can be given to the composite image.
  • the image generation unit 240E performs image processing such as blurring, brightness and / or saturation change, enlargement, reduction, and deformation on the cut-out image.
  • the brightness of the individual images may be adjusted so that the final synthesized image has the same brightness as that of the long-time exposure (continuous exposure).
  • the content and degree of image processing may be set according to user designation, or may be set without user designation.
  • the image generation unit 240E generates a final composite image (composite image) by combining the images that have undergone image processing.
  • images 1101 to 1105 cut out from the temporary composite image 1100 and these images are made translucent and superimposed to generate a final composite image 1200 (composite image).
  • the image generation unit 240E stores the final composite image 1200 in the storage unit 260 (image storage process) and displays it on the monitor 270 (image display process). You may memorize
  • the image generation unit 240E shifts the subjects 901, 902, and 903 little by little when aligning the captured image.
  • the amount to be shifted can be set according to the content and degree of the effect to be played (flow, movement, etc.).
  • the image generation unit 240E performs image processing such as blurring, brightness and / or saturation change, enlargement, reduction, and deformation on each captured image.
  • the content and degree of image processing may be set according to user designation, or may be set without user designation.
  • the image generation unit 240E generates a final composite image (composite image) by combining the images that have undergone image processing. By these processes, for example, a final composite image 1300 in FIG.
  • a final composite image 1301 can be obtained that emphasizes the effect of moving the subject from the right to the left of the image.
  • the generated final composite image is stored in the storage unit 260 (image storage process) and displayed on the monitor 270 (image display process).
  • the captured image may be stored and / or displayed.
  • FIG. 12 shows a process when the first mode (2) is set in step S300 of FIG.
  • the notification unit 240H notifies the user that the first mode (2) has been set as in the case of the first mode (1) (step S310: Notification process).
  • the exposure control unit 240A imaging unit
  • the exposure control unit 240A considers the load on the system in addition to the effect to be obtained, and the number of images to be acquired (the number of frames), multiple exposure
  • the time and the number of pixels are set according to the initial exposure time (step S320: exposure time setting step), and the image sensor 210 (image pickup unit) is controlled to perform exposure for the i-th frame (step S330: Shooting process).
  • the first mode (2) is common to the first mode (1) in that multiple exposure is performed (a plurality of images are intermittently acquired from the image sensor 210), but the focal length is as described above. This is a mode for a long case (focal length> second threshold value). Therefore, the shooting direction changing unit 240G drives the variable apex angle prism 150 via the lens driving unit 140 in accordance with the movement of the subject detected by the detection unit 240D, and automatically changes the shooting direction (step S350). : Shooting direction changing step) Shooting with multiple exposure (step S330) is performed. The exposure controller 240A repeats the multiple exposure shooting until the number of shots is N and the determination in step S360 is affirmative. In the first mode (2), effective shooting (shooting with a long exposure effect such as a moving amount of a subject) can be performed by driving the variable apex angle prism 150 by such shooting.
  • step S320 exposure time setting step
  • step S340 detection step
  • a process of generating, storing, and displaying a composite image (final image) from the captured image obtained by each exposure in step S330 an image generation process in step S370, an image storage process in step S380, and an image display process.
  • an image generation process in step S370, an image storage process in step S380, and an image display process Can be performed in the same manner as in the first mode (1) described above.
  • FIG. 13 shows a process when the second mode (1) is set in step S400 of FIG.
  • the notification unit 240H notifies the user that the second mode (1) has been set, as in the first mode (step S410: notification step).
  • the exposure control unit 240A imaging unit controls the image sensor 210 (imaging unit) to start exposure (step S420: imaging process), and the detection unit 240D detects the subject and its moving direction at a specified frame rate.
  • Step S430 detection step).
  • the information presentation unit 240F presents information that prompts the user to change the shooting direction according to the detection result, as in the first mode (see step S440: information presentation step, see FIG. 8 and the like).
  • the user can easily change the shooting direction, and can easily take an image having the effect of long exposure.
  • the exposure control unit 240A stores the photographed image in the storage unit 260 (step S460) and displays it on the monitor 270 (step S470).
  • the exposure control unit 240A since the exposure is performed for a long time (images are continuously acquired from the image sensor 210), an effect corresponding to the exposure time (initial exposure time) can be obtained. Note that the same processing (image cutout, processing, and composition) as described above with reference to FIG. 10 may be performed on the captured image to further enhance the effect of long exposure.
  • FIG. 14 shows processing when the second mode (2) is set in step S500 of FIG.
  • the notification unit 240H notifies the user that the second mode (2) has been set as in the second mode (1) (step S510: notification).
  • the exposure control unit 240A imaging unit controls the image sensor 210 (imaging unit) to start exposure (step S520: imaging step).
  • the second mode (2) is common to the second mode (1) in that continuous exposure is performed (images are continuously acquired from the image sensor 210), but the focal length is as described above. This is a mode for a long case (focal length> second threshold value).
  • the imaging direction changing unit 240G drives the variable apex angle prism 150 via the lens driving unit 140 in accordance with the movement of the subject detected by the detection unit 240D (step S530: detection process), and the imaging direction automatically.
  • Step S540 Shooting direction changing step. Shooting with long exposure is repeated until the designated exposure time (initial exposure time) has elapsed and the determination in step S550 is affirmed.
  • the second mode (2) it is possible to perform effective shooting with a large amount of movement of the subject by driving the variable apex angle prism 150 (shooting with a long-time exposure effect).
  • exposure control unit 240A stores the captured image in storage unit 260 (step S560) and displays it on monitor 270 (step S570). Processing similar to that described above with reference to FIG. 10 (image clipping, processing, and composition) may be performed on the captured image to further enhance the effect of long exposure.
  • FIG. 15 is a diagram showing an example of the effects of multiple exposure and long exposure.
  • An image 2000 shows an example of an image by multiple exposure, and the locus of the subject (black portion in the figure) is discontinuously (stepwise) thinned.
  • an image 2001 shows an example of an image by long-time exposure (continuous exposure), and the locus of the subject (gray portion in the figure) is continuously thinned.
  • the discontinuous change in the case of multiple exposure as shown in the image 2000 becomes continuous when the number of multiple exposures (the number of frames) is increased.
  • steps S220 and S320 the effect to be obtained is obtained.
  • the number of multiple exposures, the multiple exposure time, and the number of pixels in consideration of the load on the system. Since the exposure time of the image 2001 is longer than the total exposure time (multiple exposure time) of the image 2000, the image 2001 is brighter. However, the brightness of each image or the synthesized image is obtained by image processing. By adjusting, the brightness of the image 2000 can be the same as that of the image 2001.
  • the camera system 10 in the first embodiment can be realized by a digital camera, a smartphone, a tablet terminal, or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)

Abstract

L'objectif de la présente invention est de fournir un dispositif d'imagerie, un système de caméra et un procédé d'imagerie par lesquels des images présentant un long effet d'exposition peuvent être obtenues avec une excellente qualité d'image. S'il a été déterminé que des points décolorés apparaissent dans une image acquise, ledit dispositif d'imagerie règle une unité d'imagerie dans un premier mode pour acquérir par intermittence de multiples images à partir de l'élément d'imagerie, et génère une image composite par acquisition de multiples images par exposition intermittente ; d'autre part, s'il est déterminé qu'il n'apparait pas de points décolorés, le dispositif d'imagerie règle l'unité d'imagerie dans un second mode pour une exposition continue pour acquérir une image. Dans le premier mode, en réglant le temps d'exposition d'acquisition d'image intermittente et le nombre d'expositions, il est possible d'obtenir une image composite présentant l'effet d'un long temps d'exposition tout en empêchant l'apparition de points décolorés. D'autre part, dans le second mode, étant donné qu'une image est acquise avec une exposition continue, il est possible d'obtenir une image qui présente un effet correspondant au temps d'exposition.
PCT/JP2019/012410 2018-03-29 2019-03-25 Dispositif d'imagerie, procédé d'imagerie, et système de caméra WO2019188933A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2020510050A JP7015906B2 (ja) 2018-03-29 2019-03-25 撮像装置、撮像方法、及びカメラシステム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018-065048 2018-03-29
JP2018065048 2018-03-29

Publications (1)

Publication Number Publication Date
WO2019188933A1 true WO2019188933A1 (fr) 2019-10-03

Family

ID=68058332

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/012410 WO2019188933A1 (fr) 2018-03-29 2019-03-25 Dispositif d'imagerie, procédé d'imagerie, et système de caméra

Country Status (2)

Country Link
JP (1) JP7015906B2 (fr)
WO (1) WO2019188933A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112822412A (zh) * 2020-12-28 2021-05-18 维沃移动通信有限公司 曝光方法和电子设备

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0798471A (ja) * 1993-09-28 1995-04-11 Canon Inc カメラ
JP2012160852A (ja) * 2011-01-31 2012-08-23 Olympus Corp 画像合成装置、撮像装置、画像合成方法、および、画像合成プログラム
JP2015136087A (ja) * 2014-01-20 2015-07-27 オリンパス株式会社 撮像装置および撮像方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0798471A (ja) * 1993-09-28 1995-04-11 Canon Inc カメラ
JP2012160852A (ja) * 2011-01-31 2012-08-23 Olympus Corp 画像合成装置、撮像装置、画像合成方法、および、画像合成プログラム
JP2015136087A (ja) * 2014-01-20 2015-07-27 オリンパス株式会社 撮像装置および撮像方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112822412A (zh) * 2020-12-28 2021-05-18 维沃移动通信有限公司 曝光方法和电子设备
CN112822412B (zh) * 2020-12-28 2023-04-07 维沃移动通信有限公司 曝光方法、装置、电子设备和存储介质

Also Published As

Publication number Publication date
JPWO2019188933A1 (ja) 2021-03-18
JP7015906B2 (ja) 2022-02-03

Similar Documents

Publication Publication Date Title
US7852401B2 (en) Photographing apparatus and photographing method for exposure control during continuous photographing mode
JP5851070B2 (ja) レンズ交換式デジタルカメラ
JP5022105B2 (ja) カメラの表示制御装置及び方法
KR101700357B1 (ko) 점프 영상 촬영 장치 및 방법
JP3823921B2 (ja) 撮像装置
JP2012227839A (ja) 撮像装置
US8477208B2 (en) Digital image processing apparatus to simulate auto-focus, method of controlling the apparatus, and recording medium having recorded thereon a program for executing the method
JP4528235B2 (ja) デジタルカメラ
JP6214421B2 (ja) 撮像装置、撮像方法
JP2010268388A (ja) 撮像装置
JP6887245B2 (ja) 撮像装置及びその制御方法、プログラム、記憶媒体
US20140192246A1 (en) Digital photographing apparatus, method of controlling the same, and computer-readable recording medium
JP2017129828A (ja) 撮像装置および撮像方法
JP2015204470A (ja) 撮像装置およびその制御方法ならびにプログラム
JP6758950B2 (ja) 撮像装置、その制御方法とプログラム
JP7015906B2 (ja) 撮像装置、撮像方法、及びカメラシステム
JP2010068046A (ja) 撮像装置
JP2006091915A (ja) 撮像装置
US20190094655A1 (en) Imaging device
JP4278429B2 (ja) 撮像装置
JP5105298B2 (ja) 撮像装置及びそのプログラム
JP2004309653A (ja) 撮像装置
JP2006157342A (ja) カメラモジュール
JP2006050139A (ja) 光学機器
JP4378237B2 (ja) 撮影装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19775839

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2020510050

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19775839

Country of ref document: EP

Kind code of ref document: A1