CN115514899B

CN115514899B - Shooting method and device, computer readable medium and electronic equipment

Info

Publication number: CN115514899B
Application number: CN202210976169.5A
Authority: CN
Inventors: 孙少辉
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2022-08-15
Filing date: 2022-08-15
Publication date: 2024-09-27
Anticipated expiration: 2042-08-15
Also published as: CN115514899A

Abstract

The disclosure provides a shooting method and device, a computer readable medium and electronic equipment, and relates to the technical field of image shooting. The method comprises the following steps: acquiring an ambient light distribution image in a shooting scene, wherein the ambient light distribution image comprises at least two image areas; determining the flicker type of the light source of each image area through the ambient light distribution image; determining a target anti-flicker mode of each image area according to the flicker type of the light source; and performing light source flicker elimination on the image frames acquired in the shooting scene based on the target flicker resistance mode to obtain the image frames after the light source flicker elimination. The method and the device can detect different light sources existing in the current shooting scene, determine the flicker type of the light sources in the current shooting scene, further adopt different flicker-resistant modes for different image areas, effectively inhibit the flicker phenomenon of the light sources and improve the quality of acquired video.

Description

Shooting method and device, computer readable medium and electronic equipment

Technical Field

The disclosure relates to the technical field of image shooting, in particular to a shooting method, a shooting device, a computer readable medium and electronic equipment.

Background

With the continuous improvement of the living standard of people, the quality of the photographed image is getting more attention. Light source Flicker (Flicker) refers to a phenomenon that, due to intermittent Flicker of a certain light source, when the Flicker frequency is faster than any shutter movement timing, a strip shadow or a partial color change may occur on an image captured by a camera.

At present, in a related scheme for eliminating light source flicker, the flicker frequency of a light source in an image is generally detected by a frame difference method, but the method for detecting the flicker frequency of the light source by the frame difference method has low accuracy, and light source flicker elimination cannot be performed on a scene with a plurality of flicker frequencies of the light source, so that the quality of a shot video picture is poor.

Disclosure of Invention

The disclosure provides a photographing method, a photographing device, a computer readable medium and an electronic apparatus, so as to solve the problem that light source flicker elimination cannot be performed on a scene with a plurality of light source flicker frequencies in the related art to at least a certain extent, and improve the accuracy of light source flicker elimination.

According to a first aspect of the present disclosure, there is provided a photographing method including:

acquiring an ambient light distribution image in a shooting scene, wherein the ambient light distribution image comprises at least two image areas;

Determining the light source flicker type of each image area through the ambient light distribution image;

Determining a target anti-flicker mode of each image area according to the flicker type of the light source;

And performing light source flicker elimination on the image frames acquired in the shooting scene based on the target flicker resistance mode to obtain image frames after the light source flicker elimination.

According to a second aspect of the present disclosure, there is provided a photographing apparatus including:

The system comprises an ambient light image acquisition module, a camera module and a camera module, wherein the ambient light image acquisition module is used for acquiring an ambient light distribution image in a shooting scene, and the ambient light distribution image comprises at least two image areas;

The light source flicker type determining module is used for determining the light source flicker type of each image area through the ambient light distribution image;

The anti-flicker mode determining module is used for determining a target anti-flicker mode of each image area according to the flicker type of the light source;

And the light source flicker elimination module is used for carrying out light source flicker elimination on the image frames acquired in the shooting scene based on the target flicker resistance mode to obtain the image frames after the light source flicker elimination.

According to a third aspect of the present disclosure, there is provided a computer readable medium having stored thereon a computer program which, when executed by a processor, implements the method described above.

According to a fourth aspect of the present disclosure, there is provided an electronic apparatus, comprising:

a multi-window anti-flicker sensor;

A processor; and

And a memory for storing one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the methods described above.

According to the shooting method provided by the embodiment of the disclosure, an ambient light distribution image in a shooting scene can be obtained, the ambient light distribution image can comprise at least two image areas, the light source flicker type of each image area is determined through the ambient light distribution image, then the target flicker-resisting mode of each image area can be determined according to the light source flicker type, and finally the image frames acquired in the shooting scene can be subjected to light source flicker elimination based on the target flicker-resisting mode, so that the image frames after the light source flicker elimination are obtained. The environment light distribution image can be divided into at least two image areas, and the target anti-flicker modes corresponding to the image areas are respectively determined according to the determined light source flicker types, so that the corresponding anti-flicker modes can be adopted for the light source flicker phenomena distributed at different positions in the shooting scene, the light source flicker phenomena in the multi-light source shooting scene are effectively restrained, the accuracy of a processing result is improved, and the quality of a camera preview picture or a shooting video picture is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. It will be apparent to those of ordinary skill in the art that the drawings in the following description are merely examples of the disclosure and that other drawings may be derived from them without undue effort. In the drawings:

FIG. 1 illustrates a schematic diagram of an exemplary system architecture to which embodiments of the present disclosure may be applied;

FIG. 2 schematically illustrates a schematic diagram of a phenomenon of flickering of a light source in an exemplary embodiment of the present disclosure;

fig. 3 schematically illustrates a flowchart of a photographing method in an exemplary embodiment of the present disclosure;

FIG. 4 schematically illustrates a schematic diagram of a multi-window anti-flicker sensor capturing an image of an ambient light distribution in an exemplary embodiment of the present disclosure;

FIG. 5 schematically illustrates a flow chart for determining a type of flicker of a light source in an exemplary embodiment of the present disclosure;

FIG. 6 schematically illustrates a schematic diagram of a flicker frequency of a light source resulting from ambient light data conversion in an exemplary embodiment of the present disclosure;

FIG. 7 schematically illustrates a flow diagram for light source flicker cancellation by a first anti-flicker approach in an exemplary embodiment of the present disclosure;

FIG. 8 schematically illustrates a schematic diagram of eliminating a flicker phenomenon of a light source by exposure time adjustment in an exemplary embodiment of the present disclosure;

FIG. 9 schematically illustrates a flow diagram for light source flicker cancellation by a second anti-flicker approach in an exemplary embodiment of the present disclosure;

FIG. 10 schematically illustrates a schematic diagram of a core region and a non-core region in an exemplary embodiment of the present disclosure;

FIG. 11 schematically illustrates a flow chart for eliminating flicker of a light source in an exemplary embodiment of the present disclosure;

fig. 12 schematically illustrates a composition diagram of a photographing device in an exemplary embodiment of the present disclosure;

fig. 13 shows a schematic diagram of an electronic device to which embodiments of the present disclosure may be applied.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor devices and/or microcontroller devices.

Fig. 1 illustrates a schematic diagram of a system architecture of an exemplary application environment in which a photographing method and apparatus of embodiments of the present disclosure may be applied.

As shown in fig. 1, the system architecture 100 may include one or more of the terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others. The terminal devices 101, 102, 103 may be various electronic devices having image acquisition and ambient light acquisition functions, and may include, but are not limited to, desktop computers, portable computers, smart phones, tablet computers, and the like. It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation. For example, the server 105 may be a server cluster formed by a plurality of servers.

The photographing method provided by the embodiments of the present disclosure is generally performed in the terminal apparatuses 101, 102, 103, and accordingly, photographing means are generally provided in the terminal apparatuses 101, 102, 103. However, it is easily understood by those skilled in the art that the photographing method provided in the embodiment of the present disclosure may be performed by the server 105, and accordingly, the photographing device may be disposed in the server 105, which is not particularly limited in the present exemplary embodiment. For example, in an exemplary embodiment, a user may collect an ambient light distribution image in a shooting scene through the terminal devices 101, 102 and 103, then upload the ambient light distribution image to the server 105, and after determining the target anti-flicker modes of different image areas through the shooting methods provided by the embodiments of the present disclosure, the server transmits the target anti-flicker modes to the terminal devices 101, 102 and 103 to perform light source flicker elimination, so that the calculation amount of the terminal devices 101, 102 and 103 is effectively reduced.

Because of intermittent flicker of one or more light sources in a shooting scene, a phenomenon of banded shadows or partial discoloration may occur on an image acquired by a camera, referring to fig. 2, the flicker frequency of the light sources may be abstracted into a periodically distributed vertical bar, and time may be represented by a horizontal axis, so that the existence of the flicker frequency 210 of the light sources in the shooting scene may be abstracted by a graph. When the camera of the terminal equipment shoots an image, because the working principle of the electronic rolling shutter of the camera is that the image sensor is reset row by row to achieve the effect of moving the front curtain, then the rear curtain plays a role of reading row by row, and the whole image sensor needs tens of milliseconds to read row by row, therefore, the time for moving the curtain seam of the electronic shutter is longer, when the Flicker frequency of a light source is faster than the moving time of the shutter, such as the Flicker frequency 210 of the light source, the exposure time of the camera is shorter than one period of the Flicker frequency 210 of the light source, the image frame 220 can be shot, the image frame 220 has obvious banded shadow, and rolling black and white stripes, namely the Flicker of the light source can be generated when the image frame 220 previews or shoots videos.

In the related art, the flicker frequency of a light source in a current scene can be detected by a frame difference method, the continuous front and rear image frames are subjected to gray processing, and then the change amount of the banded shadow in the image frames is judged by the front and rear difference, so that the flicker frequency of the light source in the current scene is determined. However, the detection result obtained by the scheme is low in accuracy, and the frame difference method is poor in anti-interference capability for detecting the flicker frequency of the light source due to the fact that interference factors in shooting scenes are more, so that the robustness of the detection result is poor.

In another related technology, the flicker frequency of the light source in the shooting scene can be detected by a device for directly detecting the flicker frequency of the alternating current light source, but the current scheme is generally designed by matching with a single window detection device, so that the frequency space distribution condition under the flicker frequency of various light sources cannot be effectively analyzed, the main frequency cannot be effectively decided, the flicker phenomenon of the light source in different areas cannot be effectively restrained in a partitioning mode, and the flicker elimination result of the light source is poor.

Based on one or more problems in the related art, the present disclosure first provides a photographing method, and a photographing method of exemplary embodiments of the present disclosure will be specifically described below by taking a terminal device configured with a photographing apparatus as an example to perform the method.

Fig. 3 shows a flowchart of a photographing method in the present exemplary embodiment, which may include the following steps S310 to S340:

In step S310, an ambient light distribution image in the shooting scene is acquired, the ambient light distribution image including at least two image areas.

In an exemplary embodiment, the ambient light distribution image refers to an image formed by ambient light data in a shooting scene acquired by the ambient light acquisition device, and the field angle of view of the ambient light acquisition device may be the same as the field angle of view of the camera image sensor, that is, the size of the ambient light distribution image may be the same as the size of an image acquired by the image sensor, so that it can be ensured that the ambient light data in the ambient light distribution image corresponds to an area in an acquired image frame. Of course, the field angle of the ambient light collecting device may be slightly smaller or slightly larger than the field angle of the camera image sensor, which is not particularly limited in the present exemplary embodiment.

The image area refers to an area divided in the ambient light distribution image, for example, two image areas may be divided in the ambient light distribution image, one being a core area, the core area may be an area within a certain range of the center of the ambient light distribution image, and the other may be a non-core area, that is, an image area other than the core area in the ambient light distribution image; the ambient light distribution image may also be divided into four image areas, an upper left corner area, an upper right corner area, a lower left corner area, and a lower right corner area, respectively; of course, the ambient light distribution image may be divided into a plurality of image areas by other manners, and specifically may be customized according to the actual application situation, which is not limited to this embodiment.

In step S320, a light source flicker type of each of the image areas is determined from the ambient light distribution image.

In an exemplary embodiment, the light source blinking type refers to a type preset to describe a distribution of light source blinking frequency in an image area, for example, the light source blinking type may include a no light source blinking type, a single light source blinking type, and a multiple light source blinking type, wherein the no light source blinking type may indicate that the light source blinking frequency does not exist in the image area; the single light source flicker type can indicate that a single type of light source flicker frequency exists in the image area, and if the image area only contains the light source flicker frequency of 60Hz, the image area can be determined to be the single light source flicker type; the multiple light source flicker type can indicate that multiple types of light source flicker frequencies exist in the image area, and if the image area contains the light source flicker frequencies of 60Hz, 100Hz and 200Hz, the image area can be determined to be the multiple light source flicker type; of course, the light source flicker type may also include other types, and specifically may be set in a self-defined manner, which is not limited to this embodiment.

The distribution of the light source flicker frequency is set to be different light source flicker types, so that the distribution mode of the light source flicker frequency of the image area can be rapidly determined, the complex light source shooting scene can be conveniently analyzed, and the analysis efficiency is effectively improved.

In step S330, a target anti-flicker mode of each image area is determined according to the flicker type of the light source.

In an exemplary embodiment, the target anti-flicker mode refers to a processing mode for eliminating the flicker phenomenon of the light source existing in the image area, for example, if the image area is of a flicker type without a light source, the target anti-flicker mode of the image area is not processed; if the image area is of a single light source flicker type, setting exposure time by taking the flicker frequency of the light source of the unique type as the flicker frequency of the target light source, and enabling the exposure time to be an integral multiple of the flicker frequency period of the light source in a mode of adjusting the exposure time so as to eliminate the flicker of the light source of the unique type; if the image area is of a multi-light source flickering type, the target anti-flickering mode of the image area is to determine the main light source flickering frequency in the multi-type light source flickering frequencies, set the exposure time by taking the main light source flickering frequency as the target light source flickering frequency, perform global light source flickering elimination on the image area in a mode of adjusting the exposure time, and then realize the light source flickering elimination in a multi-frame superposition mode for the local area where the other types of light source flickering frequencies are located. Of course, the target anti-flicker mode may be other anti-flicker modes, specifically related to the determined flicker type of the light source, and may be set in a self-defined manner according to actual needs, which is not limited in particular in this exemplary embodiment.

In step S340, light source flicker elimination is performed on the image frames acquired in the shooting scene based on the target anti-flicker mode, so as to obtain image frames after light source flicker elimination.

In an exemplary embodiment, after determining the light source flicker types of different image areas, a light source flicker elimination strategy, that is, a target flicker resisting mode, acting on the image areas may be determined according to the light source flicker types, and then different light source flicker suppression may be implemented for different image areas according to the target flicker resisting modes of different image areas, so that accuracy of light source flicker elimination results may be effectively improved, and quality of a camera preview image or a video shot by a camera in a shooting scene may be improved.

Different types of light source flicker frequencies in a shooting scene are distinguished by setting different image areas, anti-flicker treatment is respectively carried out, so that light source flicker elimination can be carried out on all the light source flicker frequencies in the shooting scene, and the accuracy of a treatment result is improved; by collecting the ambient light distribution image in the shooting scene, and further determining all the flicker frequencies of the light sources existing in the current scene, compared with the mode of determining the flicker frequencies of the light sources by a frame difference method in the related art, the accuracy and anti-interference performance of a detection result can be effectively improved, the robustness of a flicker elimination result of the light sources is improved, and the quality of a camera preview picture or a video shot by a camera in the shooting scene is ensured.

The following describes step S310 to step S340 in detail.

In an exemplary embodiment, the ambient light distribution image in the photographed scene may be acquired by an ambient light collecting device, where the ambient light collecting device may be an anti-flicker sensor (Flickersensor), or may be another sensor or device capable of collecting ambient light data, and the exemplary embodiment is not limited thereto.

The method comprises the steps that an ambient light distribution image in a shooting scene can be obtained through a preset multi-window anti-flicker sensor; the multi-window anti-flicker sensor may include an anti-flicker sensor array comprising at least two anti-flicker sensors, the anti-flicker sensor array generating ambient light data having at least two detection window regions corresponding thereto.

Fig. 4 schematically illustrates a schematic diagram of a multi-window anti-flicker sensor capturing an ambient light distribution image in an exemplary embodiment of the present disclosure.

Referring to fig. 4, the multi-window anti-flicker sensor may include a plurality of anti-flicker sensors 410, where the anti-flicker sensors 410 are spatially arranged according to a preset number of columns and rows to obtain anti-flicker sensors, and specifically, the plurality of anti-flicker sensors 410 may be arranged according to an array of m×n to form an anti-flicker sensor array 420, for example, M may be preset to 10, and N may be preset to 10, and then the multi-window anti-flicker sensor may include 100 anti-flicker sensors 410, and the 100 anti-flicker sensors 410 are spatially arranged to obtain an anti-flicker sensor array of 10×10. Of course, specific M and N may be set in a customized manner according to the actual application scenario, which is not limited in this exemplary embodiment.

When a camera is started in a shooting scene, a multi-window anti-flicker sensor with an M.times.N anti-flicker sensor array 420 structure in space collects ambient light data in the shooting scene to obtain an ambient light distribution image 440 with M.times.N detection window areas 430, and the ambient light data in each detection window area 430 can reflect the flicker frequency of light sources of various light sources in the shooting scene.

In an exemplary embodiment, determining the light source flicker type of each image area through the ambient light distribution image may be implemented according to the steps in fig. 5, and referring to fig. 5, may specifically include:

step S510, obtaining ambient light data corresponding to each detection window area;

Step S520, performing a fast fourier transform on the ambient light data to obtain a light source flicker frequency corresponding to each detection window region;

Step S530, determining a light source flicker type of the image area according to the light source flicker frequency.

Each image area in the ambient light distribution image may correspond to a plurality of detection window areas, that is, the plurality of detection window areas form one image area, and optionally, one detection window area may also be used as one image area, which is not particularly limited in this exemplary embodiment.

The ambient light data is sampling signal data obtained by collecting a light source signal in a shooting scene by the multi-window anti-flicker sensor, the fast Fourier transform (Fast Fourier Transform) is an efficient and fast computing method for computing the discrete Fourier transform (Discrete Fourier Transform, DFT) by a computer, the discrete Fourier transform is the most basic method for signal analysis, the Fourier transform is the core of the Fourier analysis, and the signal can be transformed from a time domain to a frequency domain through the discrete Fourier transform, so that the frequency spectrum structure and the change rule of the signal are researched.

The ambient light data (sampling signals) corresponding to the detection window areas can be converted into the light source flicker frequencies corresponding to the detection window areas through fast Fourier transformation, and then the light source flicker types of the image areas can be determined through the light source flicker frequencies corresponding to the detection window areas.

Fig. 6 schematically illustrates a schematic diagram of a flicker frequency of a light source by ambient light data conversion in an exemplary embodiment of the present disclosure.

Referring to fig. 6, the ambient light data corresponding to the detection window regions may be subjected to a fast fourier transform, and a data chart 600 corresponding to the light source flicker frequency and the light source flicker frequency intensity corresponding to each detection window region may be output, and the horizontal axis of the data chart 600 may represent the light source flicker frequency and the vertical axis of the data chart 600 may represent the light source flicker frequency intensity. The first light source flicker frequency 610 collected in the detection window area can be determined to be 100Hz according to the data chart 600, the corresponding light source flicker frequency intensity is 10500, the second light source flicker frequency 620 is 210Hz, the corresponding light source flicker frequency intensity is 1800, and the intensities of other collected light source flicker frequencies are smaller, so that the light source flicker frequency can be removed in actual use and used as interference waves. Based on this, the light source flicker frequency and the light source flicker frequency intensity for each detection window region can be obtained.

Alternatively, the light source blinking type in the present embodiment may include a no light source blinking type, a single light source blinking type, and a multiple light source blinking type. If the light source flicker frequency does not exist in the image area, the light source flicker type of the image area can be determined to be a non-light source flicker type; if the image area is determined to have the flicker frequency of the light source of the only one type, the flicker type of the light source of the image area can be determined to be the flicker type of the single light source; if it is determined that there are a plurality of types of light source blinking frequencies in the image area, it may be determined that the light source blinking type of the image area is a multiple light source blinking type.

In an exemplary embodiment, the target anti-flicker approach may include a first anti-flicker approach, and if it is determined that the light source flicker type of the image region is a single light source flicker type, it may be determined that the light source flicker cancellation is performed on the image region in the first anti-flicker approach.

Specifically, the performing light source flicker elimination on the image area by adopting the first flicker resisting mode may be implemented through the steps in fig. 7, and referring to fig. 7, the method may specifically include:

Step S710, determining a first exposure time corresponding to the image area according to the light source flicker frequency corresponding to the image area, where the first exposure time is an integer multiple of the light source flicker frequency period;

In step S720, light source flicker elimination is performed on the image area through the first exposure time.

In determining that there is only one type of light source blinking frequency in the image area, the first exposure time may be determined according to the light source blinking frequency, specifically, the first exposure time may be determined by a period of the light source blinking frequency, that is, the first exposure time may be set to an integer multiple of the period of the light source blinking frequency, for example, the period of the light source blinking frequency may be 1/50 seconds, then the exposure time of the camera may be set to 1/50 seconds, 1/25 seconds, 3/50 seconds, or the like, as long as it is ensured that the first exposure time may be set to an integer multiple of the period of the light source blinking frequency, which is not particularly limited in this exemplary embodiment.

Fig. 8 schematically illustrates a schematic diagram of eliminating a flicker phenomenon of a light source by exposure time adjustment in an exemplary embodiment of the present disclosure.

Referring to fig. 8, the flicker frequency of the light source in the image region may be abstracted into a periodically distributed vertical bar, and time is represented by a horizontal axis, and thus the flicker frequency 800 of the light source existing in the image region may be abstracted by a graph. When the camera of the terminal device shoots an image, the flicker frequency of the light source of the alternating current is generally faster than the shutter movement time, that is, the common exposure time 810 of the camera is less than one period of the flicker frequency 800 of the light source, and then an image frame 820 with the flicker phenomenon of the light source can be shot.

At this time, the exposure time 830 of the camera may be determined again according to the frequency value of the light source flicker frequency 800, where the exposure time 830 may be an integer multiple of the period of the light source flicker frequency, and in the figure, the exposure time 830 is a period of the light source flicker frequency, and the image frame 840 may be obtained by shooting with the adjusted exposure time 830, so that compared with the image frame 820, the light source flicker phenomenon is effectively suppressed, and the image frame with better quality is obtained.

In an exemplary embodiment, the target anti-flicker approach may include a second anti-flicker approach, and if it is determined that the light source flicker type of the image region is a multi-light source flicker type, it may be determined that the light source flicker cancellation is performed on the image region using the second anti-flicker approach.

Specifically, the performing light source flicker elimination on the image area by adopting the second flicker resisting mode may be implemented through the steps in fig. 9, and referring to fig. 9, the method may specifically include:

Step S910, determining a primary light source flicker frequency corresponding to the image area;

Step S920, determining a second exposure time according to the primary light source flicker frequency, and performing global light source flicker elimination on the image area through the second exposure time; and

Step S930, determining that a target detection window area with a light source flicker frequency other than the main light source flicker frequency exists in the image area, and performing multi-frame superposition processing on the image content in the target detection window area to implement local light source flicker elimination of the image area.

The main light source flicker frequency refers to a light source flicker frequency with the highest occupation proportion in the image area, for example, the image area may include 10 detection window areas, wherein the light source flicker frequency is 100Hz and has 5 detection window areas, the light source flicker frequency is 200Hz and has 3 detection window areas, the light source flicker frequency is 60Hz and has 2 detection window areas, and the light source flicker frequency 100Hz occupies the highest occupation proportion in the image area at this time, so that the light source flicker frequency 100Hz can be used as the main light source flicker frequency of the image area.

The second exposure time can be determined according to the primary light source flicker frequency, global light source flicker elimination can be performed on the image area through the second exposure time, then a target detection window area with light source flicker frequency except the primary light source flicker frequency in the image area is determined, multi-frame superposition processing is performed on image content in the target detection window area, and local light source flicker elimination of the image area is achieved.

For shooting scenes with various light sources, distinguishing and restraining the light source flickering phenomena with different spatial distributions, determining the second exposure time through comparing the main light source flickering frequencies with higher occupation proportion, performing global light source flickering elimination on an image area, primarily eliminating the most light source flickering in the image area, and then eliminating the light source flickering phenomena with other light source flickering frequencies through a multi-frame superposition mode, so that the accuracy of a light source flickering elimination result can be effectively improved, and the robustness of the result is improved.

It should be noted that, the "first" and "second" in the "first exposure time" and the "second exposure time" in the embodiments of the present disclosure are only used to distinguish exposure times in different anti-flicker manners, and have no special meaning, and should not cause any special limitation to the present exemplary embodiments.

Specifically, when the light source flicker type of the image area is determined to be the no light source flicker type, the light source flicker elimination may not be performed on the image area.

In an exemplary embodiment, the at least two image areas may include a core area and a non-core area, the core area corresponding to the first detection window area and the non-core area corresponding to the second detection window area; specifically, the light source flicker type of the core area can be determined according to the light source flicker frequency corresponding to the first detection window area; the light source flicker type of the non-core area can be determined according to the light source flicker frequency corresponding to the second detection window area.

Optionally, the core area may be a central area corresponding to the ambient light distribution image, and the non-core area may be an edge area corresponding to the ambient light distribution image, for example, a geometric center of the ambient light distribution image may be taken as a center point, a preset geometric shape (such as a rectangle, a circle, etc.) may be taken as an area boundary, the central area is determined in the ambient light distribution image, and an area except for the central area in the ambient light distribution image is taken as an edge area; of course, the position corresponding to the bright light source in the small area in the ambient light distribution image may be taken as the center point of the center area, so that the center area may include the light source facing the camera in the shooting scene.

Fig. 10 schematically illustrates a schematic diagram of a core region and a non-core region in an exemplary embodiment of the present disclosure.

As shown in fig. 10, it is assumed that a shooting scene 1010 includes a light source 1011, a light source 1012, a light source 1013, a light source 1014, and a light source 1015 having different light source flicker frequencies, wherein since the light source 1011 is a light source facing a camera and thus has a large influence on the imaging process of the image sensor, the light source flicker frequency generated by the light source 1011 needs to be prioritized, and thus an area where the light source 1011 that may face the camera is located may be regarded as a core area 1021 of an ambient light distribution image 1020, and an area where other light sources are located may be regarded as a non-core area 1022.

It will be appreciated that while the core region 1021 shown in fig. 10 is a center region determined by the geometric center position of the ambient light distribution image 1020, the non-core region 1022 is an edge region of the ambient light distribution image 1020, the core region may also be a center region determined by determining the position of the light source 1011 facing the camera in the ambient light distribution image 1020; of course, the core area and the non-core area may also be image areas that are otherwise divided in the luminance distribution image, which is not limited to this example embodiment.

Specifically, a multi-window anti-flicker sensor may be acquired to have a plurality of detection window regions Wij (i=1, 2,., m; j=1, 2..once, n) of the ambient light distribution image, as illustrated in fig. 10, taking m=8, n=6 as an example, the core region 1021 may correspond to a first detection window region, e.g., the first detection window region may include a detection window region W22, detection window regions W23, … …, a detection window region W74, and a detection window region W75, and the non-core region 1022 may include a detection window region W11, detection window regions W12, … …, a detection window region W85, and a detection window region W86.

The ambient light distribution image is divided into the core area and the non-core area, and the light source elimination processing is carried out on the core area or the non-core area respectively, so that the interference of the light source in the non-core area to the light source flicker elimination result in the core area is avoided, the robustness of the light source flicker elimination result is effectively improved, the accuracy of the light source flicker elimination result can be further ensured, and the quality of the image frame after the light source flicker elimination is improved.

Fig. 11 schematically illustrates a flowchart for eliminating the flicker phenomenon of the light source in the exemplary embodiment of the present disclosure.

Referring to fig. 11, in step S1110, a multi-window anti-flicker sensor detects ambient light data; the ambient light distribution image in the shooting scene can be obtained through the multi-window anti-flicker sensor;

Step S1120, counting and analyzing the flicker frequency of the light source; the ambient light data of each detection window area in the ambient light distribution image can be subjected to fast Fourier transform to obtain the flicker frequency of the light source;

step S1130, determining whether the image area has a light source flicker frequency, if so, executing step S1140, if not, determining that the image area is of a non-light source flicker type, and ending the current flow;

Step S1140, determining whether there are multiple types of light source flicker frequencies in the image area, if there are multiple types of light source flicker frequencies, determining that the image area is of multiple light source flicker type and executing step S1160, otherwise determining that the image area is of single light source flicker type and executing step S1150;

Step S1150, employing an anti-flicker mode (global) of exposure time adjustment for a main light source flicker frequency in a scene where one type of light source flicker frequency or a plurality of types of light source flicker frequencies exist;

step S1160, determining whether the current light source flicker frequency is the main light source flicker frequency, if yes, executing step S1150, otherwise executing step S1170;

In step S1170, the anti-flicker mode (local) of multi-frame superposition is performed on the detection window regions corresponding to the other light source flicker frequencies except the main light source flicker frequency, and the current flow is ended.

In summary, in this exemplary embodiment, an ambient light distribution image in a shooting scene may be obtained, where the ambient light distribution image may include at least two image areas, a light source flicker type of each image area is determined by the ambient light distribution image, and then a target anti-flicker mode of each image area may be determined according to the light source flicker type, and finally, light source flicker elimination may be performed on an image frame acquired in the shooting scene based on the target anti-flicker mode, so as to obtain an image frame after light source flicker elimination. The environment light distribution image can be divided into at least two image areas, and the target anti-flicker modes corresponding to the image areas are respectively determined according to the determined light source flicker types, so that the corresponding anti-flicker modes can be adopted for the light source flicker distributed at different positions in the shooting scene, the light source flicker phenomenon in the multi-light source shooting scene is effectively restrained, the accuracy of a processing result is improved, and the quality of a camera preview picture or a shooting video picture is improved.

According to the embodiment of the disclosure, the multi-window anti-flicker sensor device is introduced to obtain the light source flicker frequency data of the regional alternating current light source, so that targeted frequency evaluation and screening can be performed on different image areas, the multi-window anti-flicker sensor device can adapt to complex scenes with various light sources, the light source flicker phenomenon of extremely complex light source scenes can be effectively eliminated or greatly weakened, and a better quality video picture or preview picture is provided.

It is noted that the above-described figures are merely schematic illustrations of processes involved in a method according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily appreciated that the processes shown in the above figures do not indicate or limit the temporal order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, among a plurality of modules.

Further, referring to fig. 12, in the embodiment of the present example, there is further provided a photographing apparatus 1200 including an ambient light image acquisition module 1210, a light source flicker type determination module 1220, an anti-flicker manner determination module 1230, and a light source flicker elimination module 1240. Wherein:

The ambient light image acquisition module 1210 is configured to acquire an ambient light distribution image in a shooting scene, where the ambient light distribution image includes at least two image areas;

The light source flicker type determining module 1220 is configured to determine a light source flicker type of each of the image areas from the ambient light distribution image;

The anti-flicker mode determining module 1230 is configured to determine a target anti-flicker mode of each image region according to the flicker type of the light source;

The light source flicker elimination module 1240 is configured to perform light source flicker elimination on the image frame acquired in the shooting scene based on the target anti-flicker mode, so as to obtain an image frame after light source flicker elimination.

In an exemplary embodiment, the ambient light image acquisition module 1210 may be configured to:

Acquiring an ambient light distribution image in a shooting scene through a preset multi-window anti-flicker sensor;

the multi-window anti-flicker sensor comprises an anti-flicker sensor array formed by at least two anti-flicker sensors, and the anti-flicker sensor array is used for generating environment light data corresponding to at least two detection window areas.

In an exemplary embodiment, the light source flicker type determination module 1220 may be configured to:

acquiring ambient light data corresponding to each detection window area;

Performing fast Fourier transform on the ambient light data to obtain the flicker frequency of the light source corresponding to each detection window area;

and determining the light source flicker type of each image area according to the light source flicker frequency.

If the fact that the light source flicker frequency does not exist in the image area is determined, determining that the light source flicker type of the image area is a non-light source flicker type;

If the fact that one type of light source flicker frequency exists in the image area is determined, determining that the light source flicker type of the image area is a single light source flicker type;

And if the fact that the plurality of types of light source flicker frequencies exist in the image area is determined, determining that the light source flicker type of the image area is a multi-light source flicker type.

If the light source flicker type of the image area is a single light source flicker type, determining to perform light source flicker elimination on the image area by adopting a first flicker resisting mode;

and if the light source flicker type of the image area is a multi-light source flicker type, determining to perform light source flicker elimination on the image area by adopting a second flicker resisting mode.

Determining a first exposure time corresponding to the image area according to the light source flicker frequency corresponding to the image area, wherein the first exposure time is an integer multiple of the light source flicker frequency period;

And performing light source flicker elimination on the image area through the first exposure time.

determining the flicker frequency of a main light source corresponding to the image area;

Determining a second exposure time according to the primary light source flicker frequency, and performing global light source flicker elimination on the image area through the second exposure time; and

And determining a target detection window area with the light source flicker frequency except the main light source flicker frequency in the image area, and performing multi-frame superposition processing on the image content in the target detection window area to realize the local light source flicker elimination of the image area.

In an exemplary embodiment, the at least two image areas may include a core area and a non-core area, the core area may correspond to the first detection window area, and the non-core area may correspond to the second detection window area;

the light source flicker type determination module 1220 may be configured to:

Determining the light source flicker type of the core area according to the light source flicker frequency corresponding to the first detection window area;

and determining the light source flicker type of the non-core area according to the light source flicker frequency corresponding to the second detection window area.

In an exemplary embodiment, the core region may include a center region of the ambient light distribution image, and the non-core region may include an edge region of the ambient light distribution image.

The specific details of each module in the above apparatus are already described in the method section, and the details that are not disclosed can be referred to the embodiment of the method section, so that they will not be described in detail.

Those skilled in the art will appreciate that the various aspects of the present disclosure may be implemented as a system, method, or program product. Accordingly, various aspects of the disclosure may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.

Exemplary embodiments of the present disclosure also provide an electronic device. The electronic devices may be the above-described terminal devices 101, 102, 103 and server 105. In general, the electronic device may include a multi-window anti-flicker sensor for acquiring an ambient light distribution image in a photographing scene, a processor for storing executable instructions of the processor, and a memory configured to perform the photographing method described above via execution of the executable instructions.

The configuration of the electronic device will be exemplarily described below with reference to the mobile terminal 1300 in fig. 13. It will be appreciated by those skilled in the art that the configuration of fig. 13 can also be applied to stationary type devices in addition to components specifically for mobile purposes.

As shown in fig. 13, the mobile terminal 1300 may specifically include: processor 1301, memory 1302, bus 1303, mobile communication module 1304, antenna 1, wireless communication module 1305, antenna 2, display screen 1306, camera module 1307, audio module 1308, power module 1309, and sensor module 1310.

Processor 1301 may include one or more processing units, such as: processor 1301 may include an AP (Application Processor ), modem Processor, GPU (Graphics Processing Unit, graphics Processor), ISP (IMAGE SIGNAL Processor ), controller, encoder, decoder, DSP (DIGITAL SIGNAL Processor ), baseband Processor and/or NPU (Neural-Network Processing Unit, neural network Processor), and the like. The photographing method in the present exemplary embodiment may be performed by an AP, a GPU, or a DSP, and when the method involves a neural network-related process, may be performed by an NPU, for example, the NPU may load the neural network parameters and execute the neural network-related algorithm instructions.

An encoder may encode (i.e., compress) an image or video to reduce the data size for storage or transmission. The decoder may decode (i.e., decompress) the encoded data of the image or video to recover the image or video data. Mobile terminal 1300 may support one or more encoders and decoders, for example: image formats such as JPEG (Joint Photographic Experts Group ), PNG (Portable Network Graphics, portable network graphics), BMP (Bitmap), and Video formats such as MPEG (Moving Picture Experts Group ) 1, MPEG10, h.1063, h.1064, HEVC (HIGH EFFICIENCY Video Coding).

Processor 1301 may form a connection with memory 1302 or other components through bus 1303.

Memory 1302 may be used to store computer-executable program code that includes instructions. Processor 1301 executes various functional applications of mobile terminal 1300 and data processing by executing instructions stored in memory 1302. The memory 1302 may also store application data, such as files that store images, videos, and the like.

The communication functions of the mobile terminal 1300 may be implemented by the mobile communication module 1304, the antenna 1, the wireless communication module 1305, the antenna 2, a modem processor, a baseband processor, and the like. The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. The mobile communication module 1304 may provide a mobile communication solution of 3G, 4G, 5G, etc. applied on the mobile terminal 1300. The wireless communication module 1305 may provide wireless communication solutions such as wireless local area network, bluetooth, near field communication, etc. that are applied to the mobile terminal 1300.

The display 1306 is used to implement display functions such as displaying user interfaces, images, video, and the like. The camera module 1307 is used for implementing a shooting function, such as shooting an image, video, etc. The audio module 1308 is used to implement audio functions, such as playing audio, capturing speech, and so forth. The power module 1309 is used to implement power management functions such as charging a battery, powering a device, monitoring battery status, and the like.

The sensor module 1310 may include one or more sensors for implementing corresponding inductive detection functions. For example, the sensor module 1310 may include a multi-window anti-flicker sensor for acquiring an ambient light distribution image in a photographed scene upon detecting that the mobile terminal 1300 starts a camera.

Exemplary embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon a program product capable of implementing the method described above in the present specification. In some possible implementations, various aspects of the disclosure may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the disclosure as described in the "exemplary methods" section of this specification, when the program product is run on the terminal device.

It should be noted that the computer readable medium shown in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Furthermore, the program code for carrying out operations of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A photographing method, comprising:

acquiring an ambient light distribution image in a shooting scene, wherein the ambient light distribution image comprises at least two image areas, the ambient light distribution image is an image formed by ambient light data in the shooting scene acquired by an ambient light acquisition device, and the field angle of the ambient light acquisition device is the same as that of a camera image sensor;

Determining a light source flickering type of each image region through the ambient light distribution image, wherein the light source flickering type comprises a non-light source flickering type, a single light source flickering type and a multi-light source flickering type;

Performing light source flicker elimination on the image frames acquired in the shooting scene based on the target flicker-resisting mode to obtain image frames after the light source flicker elimination;

The acquiring an ambient light distribution image in a shooting scene includes:

The multi-window anti-flicker sensor comprises an anti-flicker sensor array formed by at least two anti-flicker sensors, and the anti-flicker sensor array is used for generating ambient light data corresponding to at least two detection window areas;

The determining the flicker type of the light source of each image area through the ambient light distribution image comprises the following steps:

acquiring ambient light data corresponding to each detection window area;

and determining the light source flicker type of each image area according to the type quantity of the light source flicker frequency of each image area.

2. The method of claim 1, wherein said determining the light source flicker type for each of said image areas based on the number of types of said light source flicker frequencies for each of said image areas comprises:

3. The method of claim 2, wherein said determining a target anti-flicker pattern for each of said image regions based on said light source flicker type comprises:

4. The method of claim 3, wherein said performing light source flicker cancellation on said image region using a first anti-flicker approach comprises:

Determining a first exposure time corresponding to the image area according to the light source flicker frequency corresponding to the image area, wherein the first exposure time is an integer multiple of the period of the light source flicker frequency;

5. The method of claim 3, wherein said performing light source flicker cancellation on said image region using a second anti-flicker approach comprises:

6. The method according to any one of claims 1 to 5, wherein the at least two image areas comprise a core area and a non-core area, the core area corresponding to a first detection window area and the non-core area corresponding to a second detection window area;

the determining the light source flicker type of each image area according to the type number of the light source flicker frequency of each image area includes:

Determining the light source flicker type of the core area according to the type number of the light source flicker frequency corresponding to the first detection window area;

and determining the light source flicker type of the non-core area according to the type number of the light source flicker frequency corresponding to the second detection window area.

7. The method of claim 6, wherein the core region comprises a center region of the ambient light distribution image and the non-core region comprises an edge region of the ambient light distribution image.

8. A photographing apparatus, comprising:

The system comprises an ambient light image acquisition module, a camera image sensor and a camera image sensor, wherein the ambient light image acquisition module is used for acquiring an ambient light distribution image in a shooting scene, the ambient light distribution image comprises at least two image areas, the ambient light distribution image refers to an image formed by ambient light data in the shooting scene acquired by the ambient light acquisition device, and the field angle of the ambient light acquisition device is the same as that of the camera image sensor;

The light source flicker type determining module is used for determining the light source flicker type of each image area through the ambient light distribution image, wherein the light source flicker type comprises a no-light-source flicker type, a single-light-source flicker type and a multi-light-source flicker type;

The light source flicker elimination module is used for eliminating light source flicker of the image frames acquired in the shooting scene based on the target flicker resistance mode to obtain image frames after the light source flicker elimination;

the ambient light image acquisition module is used for:

The light source flicker type determining module is used for:

acquiring ambient light data corresponding to each detection window area;

9. A computer readable medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method according to any one of claims 1 to 7.

10. An electronic device, comprising:

a multi-window anti-flicker sensor;

A processor; and

A memory for storing executable instructions of the processor;

Wherein the processor is configured to perform the method of any one of claims 1 to 7 via execution of the executable instructions.