WO2023098743A1

WO2023098743A1 - Automatic exposure method, apparatus and device, and storage medium

Info

Publication number: WO2023098743A1
Application number: PCT/CN2022/135546
Authority: WO
Inventors: 王远博
Original assignee: 上海闻泰信息技术有限公司
Priority date: 2021-11-30
Filing date: 2022-11-30
Publication date: 2023-06-08
Also published as: CN114257738A; CN114257738B

Abstract

Disclosed are an automatic exposure method, apparatus and device, and a storage medium. The method comprises: acquiring a test script; acquiring a preview image; inputting the preview image into a pre-trained neural network model, and extracting a main region, the neural network model being configured to perform edge detection and region division on the preview image; when a screen touch operation is detected, determining a touch metering region of the preview image; on the basis of the touch metering region and the main region, determining a main subject; and performing metering on the main subject by means of a dynamic weighting method to obtain brightness information of the preview image so as to perform exposure according to the brightness information. The solution improves the accuracy of determining a main subject, and performs metering on the main subject by means of a dynamic weighting method, thus ensuring proper exposure for the main subject in a scene having a large brightness difference between the subject and a background, preventing photos from being overexposed or underexposed, and enhancing the sharpness of photos.

Description

Automatic exposure method, device, equipment and storage medium

This disclosure claims the priority of the Chinese patent application with the application number 202111447995.2 and the title of the invention "automatic exposure method, device, equipment and storage medium" filed with the China Patent Office on November 30, 2021, the entire contents of which are hereby incorporated by reference In this disclosure.

technical field

The present disclosure relates to an automatic exposure method, device, equipment and storage medium.

Background technique

With the rapid development of science and technology, the camera function has been widely used in various electronic products such as mobile phones and computers. It can be used for video conferencing, telemedicine, real-time monitoring and image processing by taking photos or videos. Among them, in order to prevent insufficient or excessive brightness of the captured image, so that the image has a good exposure effect, it is particularly important to perform light metering and exposure control on the scene before capturing the image.

At present, in the related art, an average photometry method, a center-weighted photometry method, and a spot photometry method can be used to meter a photographed picture. However, for some special scenes, such as scenes where there is a big difference between the brightness of the subject and the background, the above-mentioned average photometry and center-weighted photometry cannot make the subject obtain a suitable exposure effect; and for the above-mentioned spot metering method The selection of metering points has high requirements. For ordinary users, if the appropriate metering point cannot be selected, it will easily lead to overexposure or underexposure of the shooting screen, which will affect the clarity of the photos taken.

Contents of the invention

(1) Technical problems to be solved

In related technologies, in a shooting scene where there is a large difference in light and shade between the subject and the background, the shooting picture may be overexposed or underexposed, which affects the clarity of the taken photo.

(2) Technical solutions

According to various embodiments of the present disclosure, an automatic exposure method, device, device, and storage medium are provided.

A method of automatic exposure, the method comprising:

Get the preview image;

Inputting the preview image into a pre-trained neural network model to extract the subject area, the neural network model is configured to perform edge detection and area division on the preview image;

When a screen touch operation is detected, determine the touch metering area of the preview image;

determining a subject object based on the touch metering area and the subject area;

The dynamic weight method is used to measure the light of the main object, and the brightness information of the preview image is obtained, so as to perform exposure according to the brightness information.

In one embodiment, the preview image is input into a pre-trained neural network model to extract the subject area, including:

Divide and process the preview image according to brightness, and extract candidate regions;

The candidate area is input into a pre-trained neural network model for area division processing, so as to extract the subject area.

In one embodiment, determining the subject object based on the touch metering area and the subject area includes:

Respectively acquiring a first coordinate position corresponding to the touch metering area in the preview image and a second coordinate position corresponding to the subject area in the preview image;

determining the area corresponding to the matching position between the first coordinate position and the second coordinate position;

The area corresponding to the matching position is determined as the subject object.

In one embodiment, the dynamic weight method is used to measure the light of the main object, and the brightness information of the preview image is obtained, including:

Segmenting the main body area where the main body object is located, and establishing a Gaussian distributed weight table centered on the touch metering area;

Determine brightness information of the preview image based on the Gaussian distributed weight table.

In one embodiment, the subject area is segmented, and a Gaussian distributed weight table is established centering on the touch metering area, including:

determining an associated photometric area of the touch photometric area, where the associated photometric area is an area other than the touch photometric area in the subject area;

Taking the touch photometry area as the center, assign photometry weight values to the touch photometry area and the associated photometry area respectively, so as to establish the Gaussian distributed weight table.

In one embodiment, respectively assigning photometry weight values to the touch photometry area and the associated photometry area includes:

Assigning the highest photometric weight value to the touch metering area;

Taking the touch metering area as the center, according to the rule that the distance between the associated photometering area and the touch metering area is from near to far, sequentially assign corresponding metering values to the associated photometering area from high to low Weights.

In one embodiment, exposing according to the brightness information includes:

Based on the brightness information, using a preset exposure control algorithm to calculate exposure time and exposure gain;

When a shutter trigger operation is detected, exposure is performed based on the exposure time and the exposure gain.

An automatic exposure device, the device comprising:

an acquisition module configured to acquire a preview image;

The area extraction module is configured to input the preview image into a pre-trained neural network model to extract the subject area, and the neural network model is configured to perform edge detection and area division on the preview image;

An area determination module configured to determine the touch metering area of the preview image when a screen touch operation is detected;

a subject determination module configured to determine a subject object based on the touch metering area and the subject area;

The light metering module is configured to use a dynamic weighting method to measure light on the subject object to obtain brightness information of the preview image, so as to perform exposure according to the brightness information.

In one embodiment, the region extraction module includes: a first extraction unit and a second extraction unit; the first extraction unit is configured to divide the preview image according to brightness and extract candidate regions; the second The second extraction unit is configured to input the candidate area into a pre-trained neural network model for area division processing, so as to extract the subject area.

In one embodiment, the subject determination module includes: an acquisition unit, a first determination unit, and a second determination unit; the acquisition unit is configured to respectively acquire the first corresponding touch metering area in the preview image A coordinate position and a second coordinate position corresponding to the subject area in the preview image; the first determining unit is configured to determine an area corresponding to a position where the first coordinate position matches the second coordinate position; the first A determining unit configured to determine the area corresponding to the matching position as the subject object.

In one embodiment, the photometry module includes: an establishment unit and a third determination unit; the establishment unit is configured to perform segmentation processing on the subject area where the subject object is located, and establish a Gaussian distribution centered on the touch photometry area. Weight table: the third determination unit is configured to determine brightness information of the preview image based on the Gaussian distributed weight table.

In one embodiment, the establishing unit is specifically configured to determine the associated photometric area of the touch photometric area, and the associated photometric area is an area other than the touch photometric area in the subject area; centered on the touch photometric area , respectively assign photometry weight values to the touch photometry area and the associated photometry area, so as to establish a Gaussian distributed weight table.

In one embodiment, the establishment unit is further configured to assign the highest photometric weight value to the touch metering area; with the touch metering area as the center, according to the distance between the associated photometry area and the touch metering area from near to far According to the rules, the corresponding metering weight values are assigned to the associated metering areas in descending order.

In one embodiment, the light metering module is specifically configured to calculate exposure time and exposure gain by using a preset exposure control algorithm based on brightness information; when a shutter trigger operation is detected, perform exposure based on the exposure time and exposure gain.

A computer device, including a memory and one or more processors, the memory stores computer-readable instructions, and the one or more processors execute the computer-readable instructions to implement the method provided by any embodiment of the present disclosure. Steps of the automatic exposure method.

One or more non-transitory computer-readable storage media having stored thereon computer-readable instructions that, when executed by one or more processors, implement any implementation of the present disclosure The steps of the automatic exposure method provided by the example.

Additional features and advantages of the disclosure will be set forth in the description which follows, and, in part, will be obvious from the description, or may be learned by practice of the disclosure. The objectives and other advantages of the disclosure will be realized and attained by the structure particularly pointed out in the written description, claims hereof as well as the accompanying drawings, the details of one or more embodiments of the disclosure being set forth in the accompanying drawings and the description below.

In order to make the above objects, features and advantages of the present disclosure more comprehensible, optional embodiments are given below and described in detail in conjunction with the accompanying drawings.

Description of drawings

Other characteristics, objects and advantages of the present disclosure will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

FIG. 1 is an application scene diagram of an automatic exposure method provided by one or more embodiments of the present disclosure;

FIG. 2 is a schematic flowchart of an automatic exposure method provided by one or more embodiments of the present disclosure;

FIG. 3 is a schematic structural diagram of a convolutional neural network model provided by one or more embodiments of the present disclosure;

FIG. 4 is a schematic flowchart of a method for determining brightness information of a preview image provided by one or more embodiments of the present disclosure;

FIG. 5 is a schematic diagram of establishing a Gaussian distributed weight table centered on a touch metering area provided by one or more embodiments of the present disclosure;

FIG. 6 is a schematic diagram of a Gaussian distributed weight table provided by one or more embodiments of the present disclosure;

FIG. 7 is a schematic flowchart of an automatic exposure method provided by one or more embodiments of the present disclosure;

FIG. 8 is a schematic structural diagram of an automatic exposure device provided by one or more embodiments of the present disclosure;

Fig. 9 is a schematic structural diagram of a computer device provided by one or more embodiments of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present disclosure, not to limit the present disclosure. It should also be noted that, for ease of description, only parts related to the invention are shown in the drawings.

It should be noted that, in the case of no conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and embodiments. For ease of understanding, some technical terms involved in the embodiments of the present disclosure are explained below:

Automatic exposure: (Automatic Exposure, referred to as AE), refers to the camera automatically adjusts the exposure according to the intensity of light to prevent overexposure or underexposure. The purpose of auto exposure is to achieve an appreciative brightness level or so-called target brightness level in different lighting conditions and scenes, so that the captured video or image is neither too dark nor too bulky, in order to achieve this, the lens aperture, sensor Exposure time, sensor analog gain and sensor/ISP digital gain, the process is called auto exposure.

Convolutional Neural Network: (Convolutional Neural Network, referred to as CNN) is a feed-forward neural network that includes convolutional calculations and has a deep structure. It is one of the representative algorithms for deep learning. It consists of product layers and fully connected layers, as well as associated weights and pooling layers.

Feature extraction: refers to the method and process of using a computer to extract characteristic information in an image. In machine learning, pattern recognition, and image processing, feature extraction starts from an initial set of measurement data and builds derived values (features) that are intended to be informative and non-redundant, thereby facilitating subsequent learning and generalization steps.

It can be understood that in the process of increasing intelligence of terminal equipment, the camera function has been more and more commonly used in people's daily life. Among them, high-quality images need to be based on accurate exposure, and accurate exposure is inseparable from the premise of accurate metering, which can provide a basis for exposure control. Metering refers to the measurement of the brightness of the light reflected from the subject, also known as reflective metering, which is used to evaluate the light conditions of the camera.

At present, in the related art, an average photometry method, a center-weighted photometry method, and a spot photometry method can be used to meter a photographed picture. Among them, the average light metering method refers to dividing the picture into multiple areas, each area is independently metered, and then calculates the average light metering value of the entire picture in turn; the center weighted light metering method means that the light metering mode focuses on the center of the picture. The central area is metered, and then the entire scene is averaged. However, for some special scenes, such as scenes where there is a large difference between the brightness and darkness of the subject and the background, the above-mentioned average photometry method and center-weighted photometry method cannot obtain a suitable exposure effect for the subject. The spot metering method refers to the metering of a point, which usually refers to the center of the entire screen, but the above-mentioned spot metering method has high requirements for the selection of the metering point. For ordinary users, if the appropriate metering point cannot be selected Spots of light will cause overexposure or underexposure of the shooting screen, which will affect the clarity of the photos.

Based on the above defects, the present disclosure provides an automatic exposure method, device, equipment and storage medium. Compared with the prior art, this method can accurately extract the subject area through the neural network model, and determine the The main object improves the accuracy of determining the main object, and by using the dynamic weight method to measure the main object, it ensures the appropriate exposure of the main object in the scene with a large difference between the main object and the background light and dark, thereby avoiding The situation of overexposure or underexposure of the shooting picture is generated, and the clarity of photo shooting is improved.

The automatic exposure method provided by the embodiment of the present disclosure may be applied in the application environment as shown in FIG. 1 . FIG. 1 is an application scene diagram of an automatic exposure method in an embodiment. The application environment includes a terminal device 100. The terminal device 100 may be a terminal device with an image acquisition function. The terminal devices include but are not limited to various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices.

The above-mentioned terminal device 100 is configured to obtain a preview image, input the preview image into a pre-trained neural network model, and extract the subject area; when a screen touch operation is detected, determine the touch metering area of the preview image; and the main area to determine the main object; the dynamic weight method is used to measure the main object to obtain the brightness information of the preview image, so as to perform exposure according to the brightness information.

Among them, the above-mentioned way of obtaining the preview image by the terminal device may include, but not limited to, photosensitive acquisition through a charge-coupled device (Charge-coupled Device, referred to as CCD) photosensitive element or a CMOS (English full name is Complementary Metal-Oxide Semiconductor) photosensitive element Photosensitive acquisition.

For ease of understanding and description, the automatic exposure method, device, equipment and storage medium provided by the embodiments of the present disclosure are described in detail below with reference to FIG. 2 to FIG. 9 .

FIG. 2 is a schematic flowchart of an automatic exposure method provided by an embodiment of the present disclosure. The method is applied to a terminal device. As shown in FIG. 2 , the method includes:

S101. Acquire a preview image.

It should be noted that the above preview image refers to the image of the object to be photographed displayed in the image preview area of the terminal device before exposure, for example, when photographing a person or landscape, after the terminal device runs the camera function, it will An image of the task is formed in a certain area of the shooting interface for the user's reference. At this time, the area where the image is displayed in the shooting interface is the image preview area, and the displayed person or landscape image is the preview image.

Optionally, the above-mentioned object to be photographed may be a person, a landscape, an animal, an object, etc., and the object may be a house or a car, for example.

In this step, the terminal device can receive the trigger instruction input by the user, and start the corresponding shooting application program on the terminal device according to the trigger instruction. An image preview area is formed, and a preview image of the object to be photographed is formed in the image preview area.

Exemplarily, for example, when a user uses a smart phone to take pictures of a certain scenery, the smart phone first receives an order to open the "Camera" application, and then the smart phone automatically opens the "Camera" application, and then the smart phone opens the "Camera" application on the screen. An image preview area is formed, and the camera is called to shoot the object to be photographed, so as to form a preview image of the object to be photographed in the image preview area, so that the terminal device obtains the preview image.

S102. Input the preview image into a pre-trained neural network model to extract a subject area, and the neural network model is configured to perform edge detection and area division on the preview image.

In this step, after the preview image is obtained, the preview image can be divided and processed according to the brightness to extract candidate areas, and the candidate areas can be input into the pre-trained neural network model for area division processing to extract the main body area. Optionally, the aforementioned neural network model may be a convolutional neural network model.

Wherein, when the preview image is divided and processed according to brightness, a preset convolutional neural network model can be used to perform brightness division processing to extract candidate regions. The model parameters are obtained after continuous training. It is also possible to obtain the brightness value of each pixel in the preview image, and perform brightness division processing on the preview image according to the brightness value to extract candidate regions.

It can be understood that, as shown in FIG. 3 , the above convolutional neural network model may at least include a convolutional layer, a pooling layer and a fully connected layer. Among them, the convolution layer is to extract the local features in the picture through the filtering of the convolution kernel; the role of the pooling layer is to downsample, reduce dimensionality, remove redundant information, compress features, simplify network complexity, and reduce calculations. amount, small memory consumption, etc., processing through the pooling layer can effectively reduce the size of the parameter matrix, thereby reducing the number of parameters in the last fully connected layer, which can speed up the calculation and prevent overfitting; the fully connected layer mainly It is used for classification to output corresponding results. There can be multiple or one fully connected layers, and multiple or one convolutional layers.

In this embodiment, after the candidate area is obtained, the candidate area is input into the pre-trained neural network model for area division processing, and the candidate area can be sequentially processed through the convolutional layer, the pooling layer, and the fully connected layer. The processing is divided so that the subject area is determined.

Specifically, the candidate area can be preprocessed first to obtain the preprocessed candidate area, and then input to the convolutional layer for feature extraction processing to obtain the output result of the convolutional layer, and the output result of the convolutional layer is processed After nonlinear mapping processing, input the pooling layer for down-sampling to obtain the output result of the pooling layer, and input the output result of the pooling layer into the fully connected layer for processing, thereby extracting the main body area. Wherein, the body area may include body outline and body size.

Optionally, in the process of processing by the pooling layer, average pooling or maximum pooling may be used for processing. Among them, the average pooling (average pooling) refers to calculating the average value of the image area as the pooled value of the area. Max pooling refers to selecting the maximum value of the image area as the pooled value of the area.

The above convolutional neural network can be trained through the following steps: first initialize the weight of the network to build the initial convolutional neural network model, obtain the historical image and the divided main body area, and then input the historical image into the convolutional layer , pooling layer and fully connected layer to obtain the output value of the main area, and calculate the error between the output value of the main area and the target value of the divided main area to obtain the loss function, and minimize the pair according to the loss function The all-straight parameters in the network model are optimized and updated to obtain a trained convolutional neural network.

In this embodiment, the convolutional neural network is used to extract the main body area, which can effectively reduce the dimensionality of large data volume pictures into small data volumes, and effectively preserve image features, and can avoid image points in the background area from being distorted in the main area during photometry. The image points in the subject area are unnecessarily affected, thereby greatly improving the accuracy of metering the image points in the subject area.

S103. When a screen touch operation is detected, determine a touch metering area of the preview image.

Specifically, after obtaining the preview image, the user can click on the preview image on the screen of the terminal device, so that the terminal device detects a touch operation on the screen. Since the touch screen of the terminal device includes a series of sensors, it can detect the capacitance change caused by the finger, When the user's finger touches the screen, it will affect the self-capacitance of each sensor and the mutual capacitance between them. Therefore, the touch metering area can be detected by detecting the change of capacitance, and the area where the capacitance changes can be determined as the area of the preview image. Touch the metering area.

S104. Determine the subject object based on the touch metering area and the subject area.

Specifically, after the touch metering area and the main body area are determined, the subject object to be photographed can be determined according to a preset algorithm, and the corresponding first coordinate position of the touch metering area in the preview image and the main body area in the preview image are obtained respectively. The corresponding second coordinate position in the image, and then determine the area corresponding to the matching position between the first coordinate position and the second coordinate position, and determine the area corresponding to the matching position as the main object.

It should be noted that the subject object in this embodiment refers to the object that the terminal device needs to focus on during exposure. For example, when the AE convergence algorithm is used for exposure, the subject object is the object with the largest brightness weight value in the algorithm. Optionally, the main object in this embodiment can be a specific object, such as a person's face, human body, or an object, or a certain area in the preview image, such as an area of a certain scenery, a person and its surroundings area etc.

For example, when a user uses a touch screen smartphone to take pictures, the user opens the camera application and forms a preview image in the image preview area, and the preview image can be divided into regions through a pre-trained convolutional neural network model , so as to obtain the subject area, and then after the user performs a click operation in the preview image, the smart phone will determine the touch metering area in the preview image according to the position clicked by the user, and a marquee can be formed on the screen interface. The area corresponding to the position where the frame matches the subject area is determined as the subject object.

S105. Using a dynamic weighting method to perform light metering on the subject object to obtain brightness information of the preview image, so as to perform exposure according to the brightness information.

Specifically, after the main object is determined, light metering may be performed on the main object according to the light reflected by the main object, so as to obtain brightness information of the preview image.

As an optional implementation manner, on the basis of the above embodiments, FIG. 4 is a schematic flowchart of a method for determining brightness information of a preview image provided by an embodiment of the present disclosure. As shown in FIG. 4 , the method includes:

S201. Perform segmentation processing on the subject area where the subject object is located, and establish a Gaussian distributed weight table centered on the touch metering area.

S202. Determine brightness information of the preview image based on the Gaussian distributed weight table.

Specifically, the main body area where the main object is located may be segmented, for example, an average segmentation method may be used to determine the associated photometric area of the touch photometric area, and the associated photometric area is the part of the main body area other than the touch photometric area. For other areas, with the touch photometry area as the center, assign photometry weight values to the touch photometry area and the associated photometry area respectively, so as to establish a Gaussian distributed weight table.

Figure 5 is a schematic diagram of establishing a Gaussian distributed weight table centered on the touch metering area, please refer to Figure 5, the above Figure 5 includes the acquired preview image, the main body area extracted after processing through the convolutional neural network model, The detected touch photometric area and the associated photometric area, the associated photometric area is other areas in the main body area except the touch photometric area, and then the main body area where the main object is located is segmented, with the touch photometric area as The center establishes a Gaussian distributed weight table.

It can be understood that since the brightness information of the touch metering area is different from the brightness information of the associated photometering area, different photometry weight values are assigned to it, that is, the photometry weight value of the touch photometer area is higher than that of other photometry areas. The metering weight value of the associated metering area is high. Wherein, the photometry weight value of the touch photometry area is higher than the photometry weight value of the associated photometry area.

In this embodiment, FIG. 6 is a schematic diagram of the established Gaussian distributed weight table provided by the embodiment of the present disclosure. Please refer to FIG. As the center, according to the rule that the distance between the associated photometry area and the touch photometry area is from shortest to farthest, the corresponding photometry weight values are assigned to the associated photometry areas in descending order. For example, the photometric weight value of the brightness information assigned to the touch photometric area is 100%, and the photometric weight values of the brightness information of the associated photometric areas from the shortest to the farthest from the touch photometric area are 90% in turn. , 80%, 60%, 40%, 20%.

Further, after the Gaussian distributed weight table is established, the brightness values of the touch metering area and the associated photometering area can be obtained through photometry, and the photometry can be performed by internal photometry or external photometry. Among them, in the external light metering method, the optical path of the light metering element and the lens is independent, and this light metering method is widely used in head-up viewfinder lens shutter cameras; the internal light metering method is to measure light through the lens.

After determining the luminance values of the touch metering area and the associated photometering area, the Gaussian distributed weight table and the luminance values can be weighted and summed to obtain the luminance information of the preview image. The brightness information may be a brightness value.

Exemplarily, for example, the luminance value obtained after touching the photometry area for photometry is X, and there may be five associated photometry areas. The values are Y, Z, H, G, K in turn, and the light metering weight value of the brightness information of the touch light metering area is 100%, and the brightness of the associated light metering area with the distance from the touch light metering area from near to far The metering weight values of the information are 90%, 80%, 60%, 40%, and 20%, then the final brightness information of the entire preview screen is 100%*X+90%*Y+80%*Z+60%* H+40%*G+20%*K.

Further, after the brightness information is determined, exposure can be performed according to the brightness information.

It should be noted that exposure is a physical quantity used to calculate the luminous flux from the scene to the camera. The image sensor can only get high-quality photos if it gets the correct exposure. If it is overexposed, the image will look too bright; if it is underexposed, the image will look too dark. Among them, the size of the luminous flux reaching the sensor is mainly determined by two factors: the length of the exposure time and the size of the aperture.

Specifically, when the terminal device obtains the brightness information of the preview image, it can use the brightness information as the photometry result, and use the preset exposure control algorithm to calculate the exposure time and exposure gain based on the brightness information. When the user's shutter trigger operation is detected , exposure is performed based on exposure time and exposure gain. Optionally, the above exposure control algorithm may be an AE algorithm. In the process of exposure based on exposure time and exposure gain, the brightness of the image can be adjusted by adjusting the camera aperture size or shutter speed while keeping the sensitivity (International Standardization Organization, ISO) constant, so as to perform exposure It is controlled and processed by the ISP image sensor, so that the photos after exposure adjustment are displayed on the terminal device.

It can be understood that the above AE algorithm may include three steps, which are: first, perform brightness statistics on the brightness set by the current sensor parameter; second, analyze and guess the current brightness to determine the appropriate brightness; third, change the exposure setting , to cycle through the first few steps to maintain the brightness of the exposure.

In this embodiment, the subject area is accurately determined by combining the convolutional neural network model, which can not only meet the requirements of special scenes to obtain accurate exposure of the subject object, but also perform corresponding compensation exposure for different subject objects, thereby obtaining Better shooting experience and shooting effect.

Please refer to Figure 7, taking the terminal device as a smart phone as an example, when it is necessary to take a picture of a person or scenery, the terminal device runs the camera function, collects image information through the camera module, and displays it on the screen of the terminal device Form an image preview area, and form a preview image of the object to be photographed in the image preview area. The preview image can be divided and processed according to the brightness, and the candidate area can be extracted, and the candidate area can be input into the pre-trained CNN network model. Segmentation processing is performed to extract a subject area, which may include a subject outline and a subject size. At the same time, when the user taps the screen, the terminal device detects the user's screen touch operation, determines the touch metering area of the preview image, and then uses the background algorithm to guess the main object to be photographed based on the touch metering area, which can be extracted through the CNN network model The second coordinate position corresponding to the subject area and the first coordinate position corresponding to the light metering area are touched to determine an area corresponding to the matching position, and determine the area corresponding to the matching position as the main object. Then use the dynamic weighting method to measure the light of the main object to obtain the brightness information of the preview image. Specifically, it can segment the main area where the main object is located, determine the associated light metering area of the touch light metering area, and then use the touch light metering The region is the center, and the photometry weight values are respectively assigned to the touch metering region and the associated metering region to establish a Gaussian distributed weight table. After the brightness information is determined, the preset exposure control algorithm AE algorithm is used to adjust the exposure control, and the ISP image sensor is used for processing, so that the terminal device displays the photos after exposure adjustment.

The automatic exposure method provided by the embodiments of the present disclosure obtains a preview image and inputs the preview image into a pre-trained neural network model to extract the subject area. The neural network model is configured to perform edge detection and area division on the preview image. When a screen touch operation is detected, determine the touch metering area of the preview image, determine the main object based on the touch metering area and the main area, use the dynamic weighting method to measure the main object, and obtain the brightness information of the preview image, and use it according to the brightness Information is exposed. This method can accurately extract the subject area through the neural network model, and determine the subject object in combination with the touch metering area, which improves the accuracy of determining the subject object. In order to properly expose the main object in a scene with a large difference between the light and shade of the main body and the background, thereby avoiding the situation of over-exposure or under-exposure of the shooting picture, and improving the clarity of photo shooting.

It should be understood that although the various steps in the flow charts of FIGS. 2-5 are shown sequentially as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in Figures 2-5 may include a plurality of sub-steps or stages, these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, these sub-steps or stages The order of execution is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

On the other hand, FIG. 8 is a schematic structural diagram of an automatic exposure device provided by an embodiment of the present disclosure. The device may be a device in a terminal device. As shown in FIG. 8, the device 600 includes:

An acquisition module 610 configured to acquire a preview image;

The area extraction module 620 is configured to input the preview image into a pre-trained neural network model to extract the subject area, and the neural network model is configured to perform edge detection and area division on the preview image;

The area determination module 630 is configured to determine the touch metering area of the preview image when a screen touch operation is detected;

The subject determination module 640 is configured to determine the subject object based on the touch metering area and the subject area;

The light metering module 650 is configured to use a dynamic weighting method to measure the light of the main object to obtain brightness information of the preview image, so as to perform exposure according to the brightness information.

Optionally, the above region extraction module 620 includes:

The first extraction unit 621 is configured to divide and process the preview image according to brightness, and extract candidate regions;

The second extraction unit 622 is configured to input the candidate area into a pre-trained neural network model for area division processing, so as to extract the main body area.

Optionally, the subject determination module 640 includes:

The acquiring unit 641 is configured to respectively acquire the first coordinate position corresponding to the touch metering area in the preview image and the second coordinate position corresponding to the main body area in the preview image;

The first determining unit 642 is configured to determine an area corresponding to a position where the first coordinate position matches the second coordinate position;

The second determining unit 643 is configured to determine the area corresponding to the matching position as the subject object.

Optionally, the light metering module 650 includes:

The establishment unit 651 is configured to perform segmentation processing on the subject area where the subject object is located, and establish a Gaussian distributed weight table centered on the touch metering area;

The third determining unit 652 is configured to determine brightness information of the preview image based on the Gaussian distributed weight table.

Optionally, the above-mentioned establishing unit 651 is specifically configured as:

Determine the associated photometric area of the touch photometric area, and the associated photometric area is other areas in the main body area except the touch photometric area;

Taking the touch photometry area as the center, assign photometry weight values to the touch photometry area and the associated photometry area respectively, so as to establish a Gaussian distributed weight table.

Optionally, the above-mentioned establishment unit 651 is further configured to:

Assign the highest metering weight value to the touch metering area;

Taking the touch metering area as the center, according to the rule that the distance between the associated metering area and the touch metering area is from shortest to farthest, assign corresponding metering weight values to the associated photometering areas from high to low.

Optionally, the above photometry module 650 is specifically configured as:

Based on the brightness information, the preset exposure control algorithm is used to calculate the exposure time and exposure gain;

When a shutter trigger operation is detected, exposure is performed based on the exposure time and exposure gain.

The automatic exposure device provided by the embodiment of the present disclosure obtains the preview image through the acquisition module, and inputs the preview image into the pre-trained neural network model through the area extraction module to extract the subject area, and then when the screen touch operation is detected, through the area The determination module determines the touch metering area of the preview image, and determines the subject object through the subject determination module based on the touch metering area and the subject area, and then uses the dynamic weight method to measure the subject object through the photometry module to obtain the brightness of the preview image information to make exposure based on brightness information. This method can accurately extract the subject area through the neural network model, and determine the subject object in combination with the touch metering area, which improves the accuracy of determining the subject object. In order to properly expose the main object in a scene with a large difference between the light and shade of the main body and the background, thereby avoiding the situation of over-exposure or under-exposure of the shooting picture, and improving the clarity of photo shooting.

For specific limitations on the automatic exposure device, please refer to the above-mentioned limitations on the terminal aging test method, which will not be repeated here. Each module in the above-mentioned terminal aging test device can be fully or partially realized by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of one or more processors in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that one or more processors can call and execute the above The operation corresponding to the module.

In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure may be as shown in FIG. 9 . The computer device includes one or more processors, memory, communication interface, display screen, and input device connected by a system bus. Among other things, the one or more processors of the computer device are configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer readable instructions. The internal memory provides an environment for the execution of the operating system and computer readable instructions in the non-volatile storage medium. The communication interface of the computer device is configured to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, an operator network, near field communication (NFC) or other technologies. When the computer-readable instructions are executed by one or more processors, a terminal aging test method is realized. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball or a touch pad provided on the casing of the computer device , and can also be an external keyboard, touchpad, or mouse.

Those skilled in the art can understand that the structure shown in FIG. 9 is only a block diagram of a partial structure related to the disclosed solution, and does not constitute a limitation on the computer equipment to which the disclosed solution is applied. The specific computer equipment can be More or fewer components than shown in the figures may be included, or some components may be combined, or have a different arrangement of components.

In one embodiment, the automatic exposure device provided by the present disclosure can be implemented in the form of computer readable instructions, and the computer readable instructions can be run on the computer device as shown in FIG. 9 . Various program modules constituting the terminal aging test device can be stored in the memory of the computer equipment, for example, the acquisition module, region extraction module, region determination module, subject determination module and light metering module shown in FIG. 9 . The computer-readable instructions constituted by each program module cause one or more processors to execute the steps in the automatic exposure method of each embodiment of the present disclosure described in this specification.

For example, the computer equipment shown in FIG. 8 may execute the step of: acquiring a preview image through the acquiring module in the automatic exposure device as shown in FIG. 6 . The computer equipment can perform the step of: inputting the preview image into a pre-trained neural network model to extract the main body area through the area extraction module. The computer device may use the area determination module to perform the step of: determining the touch metering area of the preview image when a screen touch operation is detected. The computer device may perform the step of determining a subject object based on the touch metering area and the subject area through the subject determination module. The computer device may perform the step of: using a dynamic weighting method to measure the subject object through the light metering module to obtain brightness information of the preview image, so as to perform exposure according to the brightness information.

In one embodiment, a computer device is provided, including a memory and one or more processors, the memory stores computer-readable instructions, and the one or more processors execute the computer-readable instructions to implement the following steps:

Acquiring a preview image; inputting the preview image into a pre-trained neural network model, extracting the main body area, and the neural network model is configured to perform edge detection and area division on the preview image; when a screen touch operation is detected, determining the touch metering area of the preview image; determining a subject object based on the touch metering area and the subject area; performing photometry on the subject object using a dynamic weighting method to obtain brightness information of the preview image, to perform exposure according to the brightness information.

In one embodiment, there is provided one or more non-transitory computer-readable storage media having computer-readable instructions stored thereon, the computer-readable instructions being executed by one or more processors When performing the following steps:

In the above-mentioned computer-readable storage medium, the computer-readable instructions obtain the preview image and input the preview image into a pre-trained neural network model to extract the main body area, and the neural network model is configured to perform edge detection and region detection on the preview image. Divide, when a screen touch operation is detected, determine the touch metering area of the preview image, determine the main object based on the touch metering area and the main body area, use the dynamic weight method to measure the main object, and obtain the brightness information of the preview image, to perform exposure based on brightness information. This method can accurately extract the subject area through the neural network model, and determine the subject object in combination with the touch metering area, which improves the accuracy of determining the subject object. In order to properly expose the main object in a scene with a large difference between the light and shade of the main body and the background, thereby avoiding the situation of over-exposure or under-exposure of the shooting picture, and improving the clarity of photo shooting.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile computer In the readable storage medium, the computer-readable instructions may include the processes of the embodiments of the above-mentioned methods when executed. Wherein, any reference to storage, database or other media used in the various embodiments provided by the present disclosure may include at least one of non-volatile and volatile storage. Non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as Static Random Access Memory (SRAM) and Dynamic Random Access Memory (DRAM), among others.

The technical features of the above embodiments can be combined arbitrarily. To make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered to be within the range described in this specification.

The above-mentioned embodiments only express several implementation modes of the present disclosure, and the descriptions thereof are relatively specific and detailed, but should not be construed as limiting the scope of the patent for the invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present disclosure, and these all belong to the protection scope of the present disclosure. Therefore, the scope of protection of the disclosed patent should be based on the appended claims.

Industrial Applicability

The automatic exposure method provided by the present disclosure can accurately extract the subject area through the neural network model, and determine the subject object in combination with the touch metering area, which improves the accuracy of subject object determination, and uses the dynamic weight method to determine the subject object. Metering statistics can ensure proper exposure of the subject in scenes with large differences between the subject and the background, so as to avoid overexposure or underexposure of the shooting picture and improve the clarity of photo shooting. It has a strong industrial practicality.

Claims

An automatic exposure method, characterized in that the method comprises:

Get the preview image;

Inputting the preview image into a pre-trained neural network model to extract the subject area, the neural network model is configured to perform edge detection and area division on the preview image;

When a screen touch operation is detected, determine the touch metering area of the preview image;

determining a subject object based on the touch metering area and the subject area;

The dynamic weight method is used to measure the light of the main object, and the brightness information of the preview image is obtained, so as to perform exposure according to the brightness information.
The method according to claim 1, wherein, inputting the preview image into a pre-trained neural network model to extract the main body region comprises:

performing division processing on the preview image according to brightness, and extracting candidate regions;

The candidate area is input into a pre-trained neural network model for area division processing, so as to extract the subject area.
The method according to claim 2, wherein determining a subject object based on the touch metering area and the subject area comprises:

Respectively acquiring a first coordinate position corresponding to the touch metering area in the preview image and a second coordinate position corresponding to the subject area in the preview image;

determining the area corresponding to the matching position between the first coordinate position and the second coordinate position;

The area corresponding to the matching position is determined as the subject object.
The method according to claim 3, wherein the photometry of the subject object is performed by using a dynamic weight method to obtain the brightness information of the preview image, comprising:

Segmenting the main body area where the main body object is located, and establishing a Gaussian distributed weight table centered on the touch metering area;

Determine brightness information of the preview image based on the Gaussian distributed weight table.
The method according to claim 4, wherein performing segmentation processing on the subject area, and establishing a Gaussian distributed weight table centered on the touch metering area, comprising:

determining an associated photometric area of the touch photometric area, where the associated photometric area is an area other than the touch photometric area in the subject area;

Taking the touch photometry area as the center, assign photometry weight values to the touch photometry area and the associated photometry area respectively, so as to establish the Gaussian distributed weight table.
The method according to claim 5, wherein assigning a photometry weight value to the touch photometry area and the associated photometry area respectively comprises:

Assigning the highest photometric weight value to the touch metering area;

Taking the touch metering area as the center, according to the rule that the distance between the associated photometering area and the touch metering area is from near to far, sequentially assign corresponding metering values to the associated photometering area from high to low Weights.
The method according to claim 6, wherein exposing according to the brightness information comprises:

Based on the brightness information, using a preset exposure control algorithm to calculate exposure time and exposure gain;

When a shutter trigger operation is detected, exposure is performed based on the exposure time and the exposure gain.
The method according to claim 2, wherein said dividing the preview image according to brightness and extracting candidate regions comprises:

Obtain the brightness value of each pixel in the preview image;

performing brightness division processing on the preview image according to the brightness value of each pixel, and extracting the candidate region.
The method according to claim 2, wherein said inputting said candidate regions into a pre-trained neural network model for region division processing to extract subject regions comprises:

Inputting the candidate region into the neural network model for preprocessing to obtain a preprocessed candidate region;

The preprocessed candidate regions are sequentially processed by convolutional layer, pooling layer and fully connected layer for region division to determine the subject region.
An automatic exposure device, characterized in that the device comprises:

an acquisition module configured to acquire a preview image;

The area extraction module is configured to input the preview image into a pre-trained neural network model to extract the subject area, and the neural network model is configured to perform edge detection and area division on the preview image;

An area determination module configured to determine the touch metering area of the preview image when a screen touch operation is detected;

a subject determination module configured to determine a subject object based on the touch metering area and the subject area;

The light metering module is configured to use a dynamic weighting method to measure light on the subject object to obtain brightness information of the preview image, so as to perform exposure according to the brightness information.
The device according to claim 10, wherein the region extraction module comprises: a first extraction unit and a second extraction unit;

The first extraction unit is configured to divide and process the preview image according to brightness, and extract candidate regions;

The second extraction unit is configured to input the candidate area into a pre-trained neural network model for area division processing, so as to extract the subject area.
The device according to claim 11, wherein the subject determination module comprises: an acquisition unit, a first determination unit and a second determination unit;

The acquiring unit is configured to respectively acquire a first coordinate position corresponding to the touch metering area in the preview image and a second coordinate position corresponding to the main body area in the preview image;

The first determining unit is configured to determine an area corresponding to a position where the first coordinate position matches the second coordinate position;

The second determining unit is configured to determine the area corresponding to the matching position as the subject object.
The device according to claim 12, wherein the photometry module comprises: an establishing unit and a third determining unit;

The establishment unit is configured to segment the subject area where the subject object is located, and establish a Gaussian distributed weight table centered on the touch metering area;

The third determination unit is configured to determine brightness information of the preview image based on the Gaussian distributed weight table.
A computer device comprising a memory, one or more processors, and computer readable instructions stored on the memory and executable on the one or more processors, wherein the one or more processors execute the The automatic exposure method according to any one of claims 1-9 is implemented when the computer-readable instructions are used.
One or more non-transitory computer-readable storage media having stored thereon computer-readable instructions configured to implement the claims when executed by one or more processors The automatic exposure method described in any one of 1-9.