CN108156369B - Image processing method and device - Google Patents

Abstract

The application provides an image processing method and device, wherein the method comprises the following steps: controlling a main camera to shoot a first main image of a target scene according to standard exposure parameters, controlling a secondary camera to shoot a first secondary image of the target scene according to the standard exposure parameters, and controlling the main camera to shoot a second main image of the target scene according to overexposure parameters; calculating first depth-of-field information of a target image in a target scene according to a first main image and a first auxiliary image through a first thread, simultaneously acquiring a first background image from the first main image through a second thread, and simultaneously acquiring the first target image from the second main image through a third thread; and synthesizing the first background image and the first target image according to the first depth of field information of the target image to obtain a target scene image. Therefore, the target image and the background image in the target scene are shot by adopting the proper exposure parameters respectively, and the shooting effect and the image processing efficiency are improved.

Description

Image processing method and device

technical field

The present application relates to the technical field of image processing, and in particular, to an image processing method and apparatus.

Background technique

At present, the photographing function of the terminal device is used in the daily production and life of the user, and photographing is still a widespread need, and the user records life through the photographing function of the terminal device.

In the related art, in order to ensure the imaging effect of the entire image, the exposure parameters are determined according to the average brightness of the shooting scene. There is a difference between the current shooting subject and the ambient brightness, and the determined shooting exposure parameters will be affected by the ambient light, which is not suitable for the current situation. The subject shot, or the exposure parameter of the shot will be affected by the subject of the ring shot, resulting in underexposed background areas, resulting in poorer image quality.

Application content

The present application provides an image processing method and device to solve the technical problem in the prior art that the brightness of the photographed subject and the background area correspond to each other, resulting in poor photographing effect.

An embodiment of the present application provides an image processing method, including: controlling a main camera to shoot a first main image of a target scene according to standard exposure parameters, and controlling a sub-camera to shoot a first sub-image of the target scene according to standard exposure parameters; The exposure parameter controls the main camera to shoot the second main image of the target scene; calculates the first depth of field information of the target image in the target scene through the first thread according to the first main image and the first sub-image, and at the same time, The first target image is obtained from the first main image through the second thread, and at the same time, the first background image is obtained from the second main image through the third thread; The first target image and the first background image are synthesized to obtain a target scene image.

Another embodiment of the present application provides an image processing device, including: a shooting module, configured to control a main camera to shoot a first main image of a target scene according to standard exposure parameters, and control a sub-camera to shoot the target according to the standard exposure parameters a first sub-image of the scene, and controlling the main camera to shoot the second main image of the target scene according to the overexposure parameter; an acquisition module, configured to calculate the first main image and the first sub-image through the first thread The first depth of field information of the target image in the target scene, at the same time, the first target image is obtained from the first main image through the second thread, and the first target image is obtained from the second main image through the third thread. A background image; a processing module configured to perform synthesis processing on the first target image and the first background image according to the first depth of field information of the target image to obtain a target scene image.

Yet another embodiment of the present application provides a computer device, including a memory and a processor, wherein computer-readable instructions are stored in the memory, and when the instructions are executed by the processor, the processor executes the above implementation of the present application The image processing method described in the example.

Another embodiment of the present application provides a non-transitory computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the image processing method described in the foregoing embodiments of the present application.

The technical solutions provided by the embodiments of the present application may include the following beneficial effects:

Control the main camera to shoot the first main image of the target scene according to the standard exposure parameters, control the sub-camera to shoot the first sub-image of the target scene, and control the main camera to shoot the second main image of the target scene according to the overexposure parameters, through the first thread Calculate the first depth-of-field information of the target image in the target scene according to the first main image and the first sub-image, at the same time, obtain the first target image from the first main image through the second thread, and at the same time, obtain the first target image from the second main image through the third thread The first background image is obtained from the image, and further, the first target image and the first background image are synthesized according to the first depth information of the target image. In this way, not only the imaging effect of the entire image can be guaranteed, especially when the difference between the brightness of the environment and the brightness of the shooting subject is large, but also the image processing efficiency is improved.

Additional aspects and advantages of the present application will be set forth, in part, in the following description, and in part will be apparent from the following description, or learned by practice of the present application.

Description of drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

1 is a flowchart of an image processing method according to an embodiment of the present application;

FIG. 2 is a schematic diagram of the principle of triangulation ranging according to an embodiment of the present application;

3 is a schematic diagram of acquiring depth information of dual cameras according to an embodiment of the present application;

Fig. 4 is a scene schematic diagram of an image processing method according to an embodiment of the present application;

5 is a schematic diagram of the effect of image processing according to the prior art;

6 is a schematic diagram of the effect of image processing according to an embodiment of the present application;

7 is a flowchart of an image processing method according to another embodiment of the present application;

Fig. 8 is a scene schematic diagram of an image processing method according to another embodiment of the present application;

9 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;

10 is a schematic structural diagram of an image processing apparatus according to another embodiment of the present application;

FIG. 11 is a schematic structural diagram of an image processing apparatus according to still another embodiment of the present application; and

FIG. 12 is a schematic diagram of an image processing circuit according to an embodiment of the present application.

Detailed ways

The following describes in detail the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to be used to explain the present application, but should not be construed as a limitation to the present application.

The image processing method and apparatus according to the embodiments of the present application will be described below with reference to the accompanying drawings.

Wherein, the execution subject of the image processing method and apparatus of the present application may be a terminal device, wherein the terminal device may be a hardware device with dual cameras, such as a mobile phone, a tablet computer, a personal digital assistant, a wearable device, etc. The wearable device may be a smart bracelet, a smart watch, smart glasses, or the like.

Based on the above analysis, it can be seen that the difference between the brightness of the subject and the brightness of the environment will affect each other, resulting in poor shooting effect, especially when the difference between the brightness of the subject and the brightness of the environment is large, the exposure parameters of the subject are easily affected by the environment. Inappropriate exposure due to the influence of brightness, and the shooting parameters of the background area are easily affected by the brightness of the subject, resulting in inappropriate exposure. The corresponding images are overexposed, and the relevant fine details in the audience seats are underexposed, etc.

In order to solve the above-mentioned technical problems, the image processing method of the embodiment of the present application is based on appropriate exposure with the shooting background and the shooting subject, and then synthesizing the shooting subject and the background area with better exposure effect, which not only improves the exposure of the image subject effect, and ensure that the details of the entire image are rich.

FIG. 1 is a flowchart of an image processing method according to an embodiment of the present application. As shown in FIG. 1 , the method includes:

Step 101 , controlling the main camera to shoot the first main image of the target scene according to the standard exposure parameters, and at the same time, controlling the sub-camera to shoot the first sub-image of the target scene according to the standard exposure parameters.

Step 102: Over-control the main camera to shoot the second main image of the target scene according to the over-exposure parameter.

Specifically, the terminal device takes pictures through a dual-camera system, and the dual-camera system calculates depth-of-field information through a main image captured by the main camera and a sub-image captured by the sub-camera, wherein the dual-camera system includes a main camera that acquires a main image of the captured subject, and A sub-camera that assists the main image to obtain depth-of-field information, where the main camera and the sub-camera can be set in the horizontal direction, or can also be set in the vertical direction, etc., in order to more clearly describe how the dual cameras To obtain depth of field information, the following describes the principle of obtaining depth of field information with dual cameras with reference to the attached drawings:

In practical applications, the human eye mainly relies on binocular vision to distinguish depth information, which is the same as the principle of dual cameras to distinguish depth information. It is mainly realized by the principle of triangular ranging as shown in Figure 2. In 2, in the actual space, the imaging object, the positions O _R and O _T of the two cameras, and the focal planes of the two cameras are drawn. The distance between the focal plane and the plane where the two cameras are located is f, and in the focal plane Position two cameras for imaging, thereby obtaining two captured images.

Among them, P and P' are the positions of the same object in different captured images, respectively. Wherein, the distance between point P and the left border of the captured image is X _R , and the distance between point P' and the left border of the captured image is X _T . O _R and O _T are two cameras respectively, the two cameras are in the same plane and the distance is B.

Based on the principle of triangulation ranging, the distance Z between the object in Figure 2 and the plane where the two cameras are located has the following relationship:

其中，d为同一对象在不同拍摄图像中的位置之间的距离差。由于B、f为定值，因此，根据d可以确定出对象的距离Z。Based on this, it can be deduced that

Among them, d is the distance difference between the positions of the same object in different captured images. Since B and f are fixed values, the distance Z of the object can be determined according to d.

Of course, in addition to the triangulation method, other methods can also be used to calculate the depth of field information of the main image. The imaging displacement difference, posture difference, etc. are in a proportional relationship. Therefore, in an embodiment of the present application, the above-mentioned distance Z can be obtained according to this proportional relationship.

For example, as shown in Figure 3, the main image obtained by the main camera and the sub-image obtained by the sub-camera are used to calculate the difference between different points. The displacement difference of the point, but since the displacement difference in triangulation is proportional to Z, the disparity map is often used directly as the depth information map.

Specifically, in the embodiment of the present application, the depth of field information for the same object in the main image and the sub-image is calculated by using the main image captured by the main camera and the sub-image captured by the sub-camera, and the main image is used as the final image of the actual image. In order to avoid calculating the depth of field information between the main image and the sub-image, the calculation of the depth-of-field information is inaccurate due to the large difference between the main image and the sub-image, or the imaging effect of the final imaging is not caused by the unclear main image. Well, control the main camera to shoot the first main image of the target scene according to the standard exposure parameters, and control the sub-camera to shoot the first sub-image of the target scene at the same time, wherein the standard exposure parameters are based on the shooting subject in the target scene. , that is, if the brightness of the subject is strong, the aperture size corresponding to the standard exposure parameter is small, and if the subject brightness is dark, the aperture corresponding to the standard exposure parameter is large, etc. Based on prior knowledge, it can be achieved by focusing on The exposure parameter obtained by the subject in the current shooting scene is used as the first exposure parameter.

Therefore, the above-mentioned first main image and first sub-image of the target scene captured based on the standard exposure parameters include relatively clear images of the subject, and based on the first main image and the first sub-image, it is possible to further determine the image of the subject. depth of field information.

In addition, when the current ambient brightness is dark, the exposure parameters are affected by the brightly lit subject, resulting in insufficient exposure. For example, the current ambient brightness is 50 and the subject's brightness is 100, which may be affected by the subject's brightness. The final determined exposure parameter corresponds to a brightness of 60, which obviously makes the background area in the captured image unclear due to underexposure. In the embodiment of this application, the main camera is controlled to capture the target scene according to the overexposure parameter. the second main image, thereby ensuring that the background area in the target scene to be photographed is fully exposed, wherein the overexposure parameter is relative to the above standard exposure parameter, and the overexposure parameter is different from the background area in the currently photographed target scene. The brightness is related. The lower the brightness of the background area relative to the brightness of the subject, the greater the overexposure parameter is than the standard exposure parameter.

Of course, the imaging effect is poor in a dark light environment. Therefore, in the embodiments of the present application, only a common scene of shooting a subject with high brightness in a dark light environment is used as an example for explanation. In practical applications, there are also A shooting scene is to shoot a subject with low brightness in a strong light environment. In this case, the main camera can be controlled to shoot the first main image of the target scene according to the standard exposure parameters, and the sub-camera can be controlled to shoot the first main image of the target scene at the same time. The first sub image, and the second main image of the target scene that is controlled by the main camera according to the underexposure parameter.

In order to further improve the flexibility of the image processing of the present application, the image processing of the embodiments of the present application can also be implemented only when the difference in brightness between the shooting environment and the shooting subject is large, so that it is difficult to ensure a high imaging effect of the entire image. method.

Specifically, in an embodiment of the present application, to detect the first brightness of the environment where the subject is located and the second brightness of the subject in the target scene, for example, the area where the subject is located in the preview image of the shooting target scene and the second brightness of the subject can be detected. Focus metering is performed separately in the environmental area where the subject is located, and the first brightness and the second brightness are determined according to the focus metering parameter value (aperture size, etc.). When an actor on the stage takes a photo, the corresponding relationship between the shooting scene and the first brightness and the second brightness or the difference between the first brightness and the second brightness can be preset and stored, so as to match the pre-stored scene of the current shooting scene , to obtain the corresponding first brightness and second brightness or the difference between the first brightness and the second brightness, and for example, a deep learning model can be constructed in advance according to a large amount of experimental data, and the input of the deep learning model is the brightness distribution of the photographing scene, For example, the closer to the border of the viewfinder, the brighter, or the closer to the center of the viewfinder, the brighter, etc., the output is the difference between the first brightness and the second brightness or the first brightness and the second brightness, so that the brightness of the current photographing scene is The distribution is input into the deep learning model, and the outputted first brightness and second brightness or the difference between the first brightness and the second brightness are obtained. The first brightness of the shooting environment corresponds to the brightness of the background area of the target image in the shooting target scene, and the second brightness of the target image corresponds to the brightness of the current shooting subject. If the difference between the second brightness and the first brightness is determined greater than or equal to the preset threshold, control the main camera to capture the first main image of the target scene according to the standard exposure parameters, control the sub-camera to capture the first sub-image of the target scene, and control the main camera to capture the second sub-image of the target scene according to the overexposure parameter main image.

The preset threshold may be a reference brightness value calibrated according to a large number of experimental data and used to determine whether the brightness of the environment and the brightness of the shooting subject affect the imaging effect, and the preset threshold may also be related to the imaging hardware of the terminal device. The better the light sensitivity, the higher the preset threshold.

Step 103: Calculate the first depth of field information of the target image in the target scene through the first thread according to the first main image and the first sub-image; at the same time, obtain the first target from the first main image through the second thread; Three threads get the first background image from the second main image.

Step 104 , synthesizing the first target image and the first background image according to the first depth-of-field information of the target image to obtain a target scene image.

Based on the above analysis, when dual cameras acquire depth information, it is necessary to acquire the position of the same object in different captured images. The clearer the image corresponding to acquired depth information, the more accurate the acquired depth information. Therefore, in this example, based on The first main image and the first sub-image under the standard exposure parameters are used to obtain the first depth-of-field information of the target image (the shooting subject). The effect is better and the image is clearer, so the acquired first depth of field information is more accurate.

Among them, as analyzed above, the first main image is exposed based on the brightness of the subject. Therefore, the imaging effect of the target image corresponding to the subject in the first main image is better, and the second main image is based on the current shooting Therefore, the imaging effect of the background area captured in the second main image is better. In order to improve the imaging effect of the entire image, in an embodiment of the present application, based on the first main image extraction For the first target image, the first target image can be determined by techniques such as image recognition, contour recognition, etc., the first background image is obtained based on the second main image, and further, an image generated by synthesizing the first main image and the first background image The details of both the subject and the background area are very rich, and the imaging effect of the whole image is good.

In the actual execution process, in order to make the synthesis effect of the first target image and the first background image more natural, the first target image and the first background image are synthesized according to the first depth of field information of the target image. , the specific implementations of synthesizing the first target image and the first background image according to the first depth-of-field information of the target image are different:

As a possible implementation manner, since the greater the depth of field information, the smaller the object in the image, the smaller the depth of field information, the larger the object in the image, therefore, the first target image can be adaptively adjusted based on the first depth of field information. The size of the relevant area in the middle is combined with the first background image, so as to avoid the distortion caused by the inappropriate size when the two are combined.

As another possible implementation manner, since the larger the depth of field information is, the blurrier the image is, and the smaller the depth of field information is, the clearer the image is. Therefore, the first target image may be filled with pixels based on the first depth of field information, and the first target image may be filled with pixels and matched with the first background. The images are combined to make the combined effect of the two more natural.

Further, in order to further optimize the imaging effect, in an embodiment of the present application, the first background image may also be subjected to blurring processing according to the first depth of field information of the target image.

Specifically, according to different application scenarios, the first background image can be blurred according to the first depth of field information of the target image in a variety of different ways:

As a possible implementation manner: determine the blurring intensity of the first background image according to the first depth of field information of the target image, and further perform blurring processing on the corresponding background area according to the blurring strength of the first background area, so that according to the blurring strength of the first background area Different depth of field information is blurred to different degrees, making the blurred image effect more natural and rich in layers.

It should be noted that, according to different application scenarios, different implementation methods can be used to determine the blurring intensity of the first background area according to the depth of field information of the first target image. As a possible implementation method, when the first target image The more accurate the depth of field information is, the clearer the outline of the first target object is. At this time, even if the blurring process is performed on the first background area, it is less likely to cause false blurring of the first target image. At this time, the background area corresponds to Therefore, the corresponding relationship between the calculation accuracy of the depth information of the first target image and the blurring intensity corresponding to the first background area can be established in advance, and then the blurring corresponding to the background area can be obtained according to the corresponding relationship. strength.

As another possible implementation manner, the farther the position of the first target image in the first background image is, the less relevant the first target image in the first background area is. Therefore, the distance between each pixel in the first background image can be determined according to the distance between the pixels in the first background image. The distance of the first target image sets the blurring intensity, and the farther the distance between the pixels in the first background image and the first target image, the higher the blurring strength.

Furthermore, in the embodiment of the present application, since the calculation of the depth of field information takes a long time, the first thread calculates the first depth of field information of the target image in the target scene according to the first main image and the first sub-image, and simultaneously , obtaining the first target image from the first main image through the second thread, and simultaneously obtaining the first background image from the second main image through the third thread.

Therefore, on the one hand, as shown in FIG. 4 , while calculating the first depth of field information, the first target image and the first background image are obtained, so that after obtaining the first depth of field information, the first depth of field information can be directly , the first target image and the first background image are subjected to corresponding image synthesis processing. Compared with the processing method of first obtaining the depth of field information, and then performing synthesis processing on the multi-frame main images captured, the image processing efficiency has been improved. On the other hand, Obtain the first target image and the first background image according to the images captured under the standard exposure parameters and the overexposure parameters, which correspond to the subject image and the background image with richer image details. Compared with the need to use different exposure parameters to shoot multiple frames In the method of further synthesizing the images, the present application only needs to shoot twice to obtain an image with a good imaging effect for the entire image, which reduces the processing pressure of the terminal device. On the other hand, continue to refer to FIG. The target image and the first background image are split into two parallel threads to run, which further shortens the time for depth calculation and the time difference between the acquisition of the first target image and the first background image, and improves the image processing efficiency.

In order to more clearly illustrate the effect of the image processing in the embodiments of the present application, an example is given below in conjunction with specific application scenarios:

The first scenario:

When taking a picture of an actor on the stage in an auditorium with poor ambient light, the image obtained by using the shooting mode in the prior art is shown in Figure 5. The background area is underexposed, resulting in an unclear background area, while the area where the actor is located is not clear. Excessive exposure, resulting in the area where the actor is not clear (the grayscale of the image in the figure represents the sharpness of the image, the clearer the image, the lower the grayscale value of the corresponding image area).

After using the image processing method of the present application, the main camera is controlled to capture the first main image of the target scene according to the standard exposure parameter, the sub-camera is controlled to capture the first sub-image of the target scene, and the main camera is controlled to capture the target scene according to the overexposure parameter. The second main image is calculated by the first thread according to the first main image and the first sub-image of the first depth of field information in the area where the actor is located in the target scene, and at the same time, the image of the actor is obtained from the first main image through the second thread, At the same time, the background image is obtained from the second main image through the third thread, and the actor image and the first background image are synthesized according to the first depth of field information of the actor image, as shown in FIG. All have been properly exposed, so the whole image is clearer.

To sum up, the image processing method of the embodiment of the present application controls the main camera to capture the first main image of the target scene according to the standard exposure parameters, controls the sub-camera to capture the first sub-image of the target scene at the same time, and controls the main camera to capture the first sub-image of the target scene according to the overexposure parameter. The camera shoots the second main image of the target scene, calculates the first depth of field information of the target image in the target scene according to the first main image and the first sub-image through the first thread, and simultaneously obtains the first depth information from the first main image through the second thread. At the same time, a first background image is obtained from the second main image through a third thread, and further, the first target image and the first background image are synthesized according to the first depth information of the target image. In this way, not only the imaging effect of the entire image can be guaranteed, especially when the difference between the brightness of the environment and the brightness of the shooting subject is large, but also the image processing efficiency is improved.

Based on the descriptions of the above embodiments, when the difference between the brightness of the environment and the brightness of the subject is not large, shooting an image based on a unified exposure parameter may have little effect on the display effect of the entire image, and the exposure parameter can ensure a clear image of the subject. , it can also ensure clear imaging of the background area, so as to optimize the imaging effect of the captured image.

Specifically, in another embodiment of the present application, as shown in FIG. 7 , the above step 101 may further include:

Step 201: Detect the first brightness of the shooting environment and the second brightness of the target image.

Step 202 , if it is detected that the difference between the second brightness and the first brightness is smaller than a preset threshold, the exposure parameter is determined according to the difference.

Specifically, if it is detected that the difference between the second brightness and the first brightness is less than a preset threshold, an exposure parameter is determined according to the difference, and the exposure parameter is suitable for both clear imaging of the background area and clear imaging of the subject.

As a possible implementation, a correspondence relationship between the difference between the second brightness and the first brightness and the exposure parameter can be established in advance according to a large amount of experimental data, and then after obtaining the current difference between the second brightness and the first brightness, query the The corresponding relationship is obtained, and exposure parameters corresponding to the target scene currently shot are obtained.

Step 203 , under the environment of exposure parameters, the main camera is controlled to shoot multiple sets of main images of the target scene, and the sub-camera is controlled to shoot multiple sets of secondary images of the target scene at the same time.

Specifically, in order to improve the imaging effect, the main camera is controlled to shoot multiple sets of main images of the target scene under the environment of exposure parameters, and the sub-camera is controlled to shoot multiple sets of secondary images of the target scene at the same time.

Step 204: Acquire a reference main image from a plurality of groups of main images, and acquire a reference sub-image captured in the same group as the reference main image from the plurality of groups of sub-images.

It can be understood that in the embodiments of the present application, since the main camera and the sub-camera simultaneously capture multiple groups of main images and multiple groups of sub-images, the image information of the main images and sub-images belonging to the same group captured at the same time point is relatively high. close, and the depth of field information is calculated according to the source main image and the sub-image, which can ensure that the acquired depth of field information is relatively accurate.

Specifically, a reference main image is selected from multiple sets of main images, and a reference sub-image captured in the same group as the reference main image is selected from multiple sets of sub-images. It should be emphasized that in the actual shooting process, the main image and the sub-image are The images are captured in multiple groups of images at the same frequency, wherein the main image and the sub-image captured at the same time belong to the same group of images. For example, in chronological order, the main images captured by the main camera include the main Image 12..., the multiple sets of main images captured by the sub-camera include sub-image 21, sub-image 22..., then the main image 11 and the sub-image 21 are the same group of images, and the main image 12 and the sub-image 22 are the same group of images... In order to To further improve the efficiency and accuracy of depth information acquisition, it is also possible to select reference main images with higher definition from multiple groups of main images. Of course, when the number of image frames in the acquired image group is large, in order to improve the selection efficiency, It is also possible to preliminarily select several frames of main images and corresponding several frames of sub-images according to the image definition, and select a reference main image and corresponding several frames of sub-images from the several frames of main images and corresponding several frames of sub-images with higher definition.

Step 205, generating a target main image by synthesizing and denoising the multiple groups of main images through the first thread, and obtaining a second target image from the target main image, and at the same time, calculating the target scene according to the reference main image and the reference sub-image through the second thread The second depth of field information of the target image is obtained in the middle, and at the same time, the second background image is obtained from the reference main image through the third thread.

Step 206: Perform synthesis processing on the second background image and the second target image according to the second depth-of-field information of the target image.

Specifically, as shown in FIG. 8 , in an embodiment of the present application, a target main image is generated by performing synthetic noise reduction on multiple groups of main images through the first thread, and a second target image is obtained from the target main image, and at the same time, The second depth information of the target image in the target scene is calculated according to the reference main image and the reference sub-image by the second thread (the reference main image and the reference sub-image are the second frame images captured by the main camera and the sub-camera), and at the same time, the third The thread obtains the second background image from the reference main image, and then performs synthesis processing on the second background image and the second target image according to the second depth-of-field information of the target image, thereby not only improving the image processing efficiency, but also reducing according to the synthesis. The second target image is obtained from the denoised target main image, which further improves the clarity of the image.

Among them, in order to facilitate a clear understanding of the multi-frame composite noise reduction process, the following describes the multi-frame composite noise reduction of the main image in a scene with poor lighting conditions.

When the ambient light is insufficient, imaging devices such as terminal equipment generally use the method of automatically increasing the sensitivity to shoot. But this way of increasing the sensitivity results in more noise in the image. Multi-frame composite noise reduction is to reduce the noise points in the image and improve the quality of the image captured under high light sensitivity. The principle lies in the prior knowledge of the disordered arrangement of noise points. Specifically, after shooting multiple sets of images in a row, the noise points appearing in the same position may be red noise points, green noise points, white noise points, or even There is no noise, so there is a condition for comparison and screening, which can be based on the value of each pixel corresponding to the same position in the multiple sets of captured images (the value of the pixel includes the number of pixels contained in the pixel, and the more pixels are contained. The higher the value of the pixel points, the clearer the corresponding image), and the pixels that belong to the noise (ie, the noise points) are screened out.

Furthermore, after filtering out the noise, color guessing and pixel replacement processing can also be performed on the noise according to the algorithm of the further method, so as to achieve the effect of removing the noise. After this process, a noise reduction effect with extremely low image quality loss can be achieved.

For example, as a relatively simple multi-frame synthesis noise reduction method, after acquiring multiple sets of photographed images, the value of each pixel corresponding to the same position in the multiple sets of photographed images can be read, and the weighted average of these pixels can be calculated. value to generate the value of the pixel at this position in the composite image. In this way, clear images can be obtained.

Of course, in this embodiment, in order to further improve the image processing effect, the second background image can also be blurred according to the second depth of field information of the target image. The image blurring is performed in a similar manner, which is not repeated here.

To sum up, the image processing method of the embodiment of the present application, when the difference between the ambient brightness and the shooting subject is not large, shooting based on unified exposure parameters, not only reduces the processing pressure of the terminal device, but also ensures the imaging effect of the image .

In order to realize the above-mentioned embodiments, the present application also proposes an image processing apparatus. FIG. 9 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application. As shown in FIG. 9 , the image processing apparatus includes: a photographing module 100, The acquisition module 200 and the processing module 300 are obtained.

The shooting module 100 is configured to control the main camera to shoot the first main image of the target scene according to the standard exposure parameters, and simultaneously control the sub-camera to shoot the first sub-image of the target scene according to the standard exposure parameters.

The shooting module 100 is further configured to control the main camera to shoot the second main image of the target scene according to the overexposure parameter.

The obtaining module 200 is used for calculating the first depth information of the target image in the target scene according to the first main image and the first sub-image through the first thread, and at the same time, obtaining the first target image from the first main image through the second thread, At the same time, the first background image is acquired from the second main image through the third thread.

The processing module 300 is configured to perform synthesis processing on the first target image and the first background image according to the first depth information of the target image to obtain the target scene image.

In an embodiment of the present application, as shown in FIG. 10 , the apparatus further includes a detection module 400 and a determination module 500, wherein,

The detection module 400 is configured to detect the first brightness of the environment where the photographed subject is located and the second brightness of the photographed subject in the target scene.

The determination module 500 is configured to determine that the difference between the second brightness and the first brightness is greater than or equal to a preset threshold.

Further, in an embodiment of the present application, as shown in FIG. 11 , the photographing module 100 includes a blurring unit 110 .

The blurring unit 110 is configured to perform blurring processing on the first background image according to the first depth of field information of the target image, and obtain the target scene image after background blurring.

It should be noted that the foregoing descriptions of the method embodiments are also applicable to the apparatuses of the embodiments of the present application, and the implementation principles thereof are similar, which will not be repeated here.

The division of each module in the above image processing apparatus is only for illustration. In other embodiments, the image apparatus may be divided into different modules as required to complete all or part of the functions of the above image apparatus.

To sum up, the image processing apparatus of the embodiment of the present application controls the main camera to shoot the first main image of the target scene according to the standard exposure parameters, and simultaneously controls the sub-camera to shoot the first sub-image of the target scene according to the standard exposure parameters, and controls the first sub-image of the target scene according to the standard exposure parameters. The exposure parameter controls the main camera to shoot the second main image of the target scene, and calculates the first depth of field information of the target image in the target scene through the first thread according to the first main image and the first sub-image. The first target image is obtained from the image, and at the same time, the first background image is obtained from the second main image through the third thread, and further, the first target image and the first background image are synthesized according to the first depth information of the target image. In this way, not only the imaging effect of the entire image can be guaranteed, especially when the difference between the brightness of the environment and the brightness of the shooting subject is large, but also the image processing efficiency is improved.

In order to realize the above-mentioned embodiments, the present application also proposes a computer device, wherein the computer device is any device including a memory for storing a computer program and a processor for running the computer program, for example, a smart phone, a personal computer, etc., The above computer equipment further includes an image processing circuit, which may be implemented by hardware and/or software components, and may include various processing units that define an ISP (Image Signal Processing, image signal processing) pipeline. FIG. 12 is a schematic diagram of an image processing circuit in one embodiment. As shown in FIG. 12 , for the convenience of description, only various aspects of the image processing technology related to the embodiments of the present application are shown.

As shown in FIG. 12 , the image processing circuit includes an ISP processor 1040 and a control logic 1050 . Image data captured by imaging device 1010 is first processed by ISP processor 1040 , which analyzes the image data to capture image statistics that can be used to determine and/or control one or more parameters of imaging device 1010 . The imaging device 1010 (camera) may include a camera having one or more lenses 1012 and an image sensor 1014, wherein, in order to implement the background blurring processing method of the present application, the imaging device 1010 includes two sets of cameras, wherein, with continued reference to FIG. 12, The imaging device 1010 may simultaneously capture scene images based on the main camera and the sub-camera. Image sensor 1014 may include an array of color filters (eg, Bayer filters), image sensor 1014 may obtain light intensity and wavelength information captured with each imaging pixel of image sensor 1014 and provide a set of raw materials that may be processed by ISP processor 1040. image data, wherein the ISP processor 1040 can calculate depth information and the like based on the raw image data obtained by the image sensor 1014 in the main camera and the raw image data obtained by the image sensor 1014 in the sub-camera provided by the sensor 1020 . The sensor 1020 may provide raw image data to the ISP processor 1040 based on the sensor 1020 interface type. The sensor 1020 interface may utilize a SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above interfaces.

The ISP processor 1040 processes raw image data pixel by pixel in various formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 1040 may perform one or more image processing operations on the raw image data, collecting statistical information about the image data. Among them, the image processing operations can be performed with the same or different bit depth precision.

ISP processor 1040 may also receive pixel data from image memory 1030 . For example, the raw pixel data is sent from the sensor 1020 interface to the image memory 1030, and the raw pixel data in the image memory 1030 is then provided to the ISP processor 1040 for processing. The image memory 1030 may be a part of a memory device, a storage device, or an independent dedicated memory in an electronic device, and may include a DMA (Direct Memory Access, direct memory access) feature.

When receiving raw image data from the sensor 1020 interface or from the image memory 1030, the ISP processor 1040 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 1030 for additional processing before being displayed. The ISP processor 1040 receives processed data from the image memory 1030 and performs image data processing in the original domain and in the RGB and YCbCr color spaces on the processed data. The processed image data may be output to the display 1070 for viewing by a user and/or further processed by a graphics engine or a GPU (Graphics Processing Unit, graphics processor). In addition, the output of the ISP processor 1040 may also be sent to the image memory 1030 , and the display 1070 may read image data from the image memory 1030 . In one embodiment, image memory 1030 may be configured to implement one or more frame buffers. Additionally, the output of the ISP processor 1040 may be sent to the encoder/decoder 1060 for encoding/decoding the image data. The encoded image data can be saved and decompressed prior to display on the display 1070 device. The encoder/decoder 1060 may be implemented by a CPU or GPU or a co-processor.

Statistics determined by the ISP processor 1040 may be sent to the control logic 1050 unit. For example, the statistics may include image sensor 1014 statistics such as auto exposure, auto white balance, auto focus, flicker detection, black level compensation, lens 1012 shading correction, and the like. The control logic 1050 may include a processor and/or microcontroller executing one or more routines (eg, firmware) that may determine control parameters of the imaging device 1010 and control of the imaging device 1010 based on the received statistics parameter. For example, control parameters may include sensor 1020 control parameters (eg, gain, integration time for exposure control), camera flash control parameters, lens 1012 control parameters (eg, focal length for focus or zoom), or a combination of these parameters. ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (eg, during RGB processing), and lens 1012 shading correction parameters.

The following are the steps of implementing the image processing method using the image processing technology in Figure 12:

Controlling the main camera to capture the first main image of the target scene according to the standard exposure parameter, simultaneously controlling the sub-camera to capture the first sub-image of the target scene, and controlling the main camera to capture the second main image of the target scene according to the overexposure parameter;

The first depth information of the target image in the target scene is calculated by the first thread according to the first main image and the first sub image, and at the same time, the first target is obtained from the first main image by the second thread image, and at the same time, obtain the first background image from the second main image through the third thread;

The first target image and the first background image are synthesized according to the first depth information of the target image.

In order to implement the above-mentioned embodiments, the present application further proposes a non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor, the image processing methods of the above-mentioned embodiments can be executed.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present application, "plurality" means at least two, such as two, three, etc., unless expressly and specifically defined otherwise.

Any process or method description in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing custom logical functions or steps of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of this application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented by any one of the following techniques known in the art, or a combination thereof: discrete with logic gates for implementing logic functions on data signals Logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Those skilled in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program can be stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations to the present application. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (8)

1. An image processing method, comprising:

detecting first brightness of an environment where a shooting subject is located in a target scene and second brightness of the shooting subject;

determining that the difference value between the second brightness and the first brightness is greater than or equal to a preset threshold value;

controlling a main camera to shoot a first main image of a target scene according to standard exposure parameters, and simultaneously controlling a secondary camera to shoot a first secondary image of the target scene according to the standard exposure parameters;

controlling a main camera to shoot a second main image of the target scene according to the overexposure parameter;

calculating first depth of field information of a target image in the target scene according to the first main image and the first auxiliary image through a first thread, simultaneously acquiring a first target image from the first main image through a second thread, and simultaneously acquiring a first background image from the second main image through a third thread;

and synthesizing the first target image and the first background image according to the first depth of field information of the target image to obtain a target scene image.

2. The method of claim 1, further comprising:

and blurring the first background image according to the first depth of field information of the target image to obtain the target scene image with a blurred background.

3. The method of claim 1, wherein after the detecting a first brightness of the photographic environment and a second brightness of the target image, further comprising:

if the difference value between the second brightness and the first brightness is smaller than a preset threshold value, determining an exposure parameter according to the difference value;

controlling the main camera to shoot a plurality of groups of main images on the target scene under the exposure parameter environment, and simultaneously controlling the auxiliary camera to shoot a plurality of groups of auxiliary images on the target scene;

acquiring a reference main image from the multiple groups of main images, and acquiring a reference auxiliary image which is shot in the same group with the reference main image from the multiple groups of auxiliary images;

synthesizing and denoising the multiple groups of main images through the first thread to generate a target main image, acquiring a second target image from the target main image, meanwhile, calculating second depth of field information of the target image in the target scene according to the reference main image and the reference auxiliary image through the second thread, and meanwhile, acquiring a second background image from the reference main image through the third thread;

and synthesizing the second background image and the second target image according to the second depth information of the target image.

4. The method of claim 3, further comprising:

and performing blurring processing on the second background image according to the second depth of field information of the target image to obtain the target scene image after background blurring processing.

5. An image processing apparatus characterized by comprising:

the detection module is used for detecting first brightness of the environment where the shooting subject is located and second brightness of the shooting subject in the target scene;

the determining module is used for determining that the difference value between the second brightness and the first brightness is greater than or equal to a preset threshold value;

the shooting module is used for controlling the main camera to shoot a first main image of a target scene according to standard exposure parameters and controlling the auxiliary camera to shoot a first auxiliary image of the target scene according to the standard exposure parameters;

the shooting module is further used for controlling a main camera to shoot a second main image of the target scene according to the overexposure parameter;

the acquisition module is used for calculating first depth of field information of a target image in the target scene according to the first main image and the first auxiliary image through a first thread, acquiring a first target image from the first main image through a second thread, and acquiring a first background image from the second main image through a third thread;

and the processing module is used for synthesizing the first target image and the first background image according to the first depth of field information of the target image to acquire a target scene image.

6. The apparatus of claim 5, wherein the photographing module comprises:

and the blurring unit is used for blurring the first background image according to the first depth-of-field information of the target image to acquire the target scene image with a blurred background.

7. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the image processing method as claimed in any one of claims 1 to 4 when executing the program.

8. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the image processing method according to any one of claims 1 to 4.

Images