WO2022057670A1

WO2022057670A1 - Real-time focusing method, apparatus and system, and computer-readable storage medium

Info

Publication number: WO2022057670A1
Application number: PCT/CN2021/116773
Authority: WO
Inventors: 弓殷强; 邓岳慈; 余新; 李屹
Original assignee: 深圳光峰科技股份有限公司
Priority date: 2020-09-17
Filing date: 2021-09-06
Publication date: 2022-03-24
Also published as: CN114286064A

Abstract

Disclosed are a real-time focusing method, apparatus and system, and a computer-readable storage medium. The method comprises: acquiring video source data and a captured image; processing the video source data and the captured image, so as to obtain feature information of the video source data and feature information of the captured image; and on the basis of the feature information of the video source data and the feature information of the captured image, generating a control instruction, and sending the control instruction to an electric motor, such that the electric motor drives a projection lens to move, thereby realizing focusing. In this way, automatic real-time focusing can be realized by means of the present application.

Description

A real-time focusing method, device, system and computer-readable storage medium

technical field

The present application relates to the field of projection technology, and in particular, to a real-time focusing method, device, system, and computer-readable storage medium.

Background technique

Autofocus is a very important function of the projector, so that the projector can be adapted to more usage scenarios. The specific needs can be refined into the following two aspects: reset the focus surface after the first installation or move the position; In the open state, temperature changes will occur due to heat generation, and various components of the lens and optical path will cause thermal defocusing phenomenon under temperature changes; Real-time-like blurring cannot be corrected by current autofocus technology. The traditional focusing scheme needs to suspend the display and focus on a specific reference image, which cannot achieve non-sensing focusing; in addition, the traditional focusing scheme can also adjust the angle of the focusing motor by setting the motion sensor, but due to the error and noise of the motion sensor. Larger, the movement of the projector and the relative relationship between the projector and the screen are more complicated, and it is impossible to get better results, and it is impossible to adjust the blur caused by thermal defocus.

SUMMARY OF THE INVENTION

The present application provides a real-time focusing method, device, system and computer-readable storage medium, which can realize automatic real-time focusing.

In order to solve the above-mentioned technical problems, the technical solution adopted in the present application is to provide a real-time focusing method, the method includes: acquiring video source data and captured images; The feature information of the captured image; based on the feature information of the video source data and the feature information of the captured image, a control command is generated, and the control command is sent to the motor, so that the motor drives the projection lens to move and realize focusing.

In order to solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a real-time focusing device, the real-time focusing device includes a memory and a processor that are connected to each other, wherein the memory is used to store a computer program, and the computer program is used by the processor. When executed, it is used to implement the above-mentioned real-time focusing method.

In order to solve the above technical problem, another technical solution adopted in the present application is to provide a projection system, the projection system includes: a real-time focusing device, a projection device and a camera device, the real-time focusing device is used for receiving video source data; The focusing device is connected to receive the video source data sent by the real-time focusing device and perform projection display, wherein the projection device includes a projection lens and a motor that are connected to each other; The image is shot to obtain a shot image corresponding to the video source data; wherein, the real-time focusing device is also used to process the video source data and the shot image to obtain feature information of the video source data and feature information of the shot image; based on the video source data The feature information and the feature information of the captured image are generated, and the control command is sent to the motor, so that the motor drives the projection lens to move and realize focusing.

In order to solve the above-mentioned technical problem, another technical solution adopted in this application is to provide a computer-readable storage medium, the computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, it is used to realize the above-mentioned Live focus method.

Through the above scheme, the beneficial effects of the present application are: the received video source data is stored, and the projection device is controlled to display the video source data to obtain a captured image corresponding to the video source data; then the video source data and the captured image are processed. processing to obtain the corresponding feature information; using the feature information of the video source data and the feature information of the captured image, the adjustment direction of the projection lens can be determined; then according to the adjustment direction, the corresponding control instructions are generated to control the motor to drive the projection lens to move, which can be Focusing is performed at the same time as playing the screen, without manual focusing, real-time focusing and non-sensing focusing can be achieved, and thermal defocus can be corrected.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort. in:

1 is a schematic flowchart of an embodiment of a real-time focusing method provided by the present application;

2 is a schematic flowchart of another embodiment of a real-time focusing method provided by the present application;

FIG. 3 is a schematic flowchart of step 131 in the embodiment shown in FIG. 2;

4 is a schematic diagram of a captured image in the embodiment shown in FIG. 2;

5 is a schematic structural diagram of an embodiment of a real-time focusing device provided by the present application;

6 is a schematic structural diagram of an embodiment of a projection system provided by the present application;

Fig. 7 is the connection schematic diagram of the video source and the real-time focusing device in the embodiment shown in Fig. 6;

FIG. 8 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided by the present application.

detailed description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

The application provides a real-time focusing method, the method includes:

Obtain video source data and capture images;

The video source data and the captured image are processed to obtain the feature information of the video source data and the feature information of the captured image;

Based on the feature information of the video source data and the feature information of the captured image, a control command is generated, and the control command is sent to the motor, so that the motor drives the projection lens to move and realize focusing.

Described below in conjunction with specific embodiments:

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of an embodiment of a real-time focusing method provided by the present application. The method includes:

Step 11: Acquire video source data and capture images.

The video source data can be the pixel value of the image, different video source data can be received in different time periods, the video source data can be acquired online in real time, the video source data sent by other devices can be received, or the video source can be read from a storage device data; in order to facilitate the subsequent comparison with the captured images, the video source data can be stored in the real-time focusing device. It can be understood that the amount of video source data stored in the real-time focusing device is relatively small, and can be cleaned regularly, and the video source signals within a preset time before and after the current video source data can be obtained in real time, for example, the video source within 5 seconds can be obtained. data, which is cached.

Further, the video source data sent by the video source can be received, and then the video source data can be sent to the projection device, and the projection device can be controlled to display the video source data to generate a projection display image; after the projection device starts to display, the synchronous shooting can be sent. The signal is sent to the camera device to control the camera device to shoot the projected display image on the projection screen, so as to obtain the captured image corresponding to the video source data, and the captured image is at least partially the same as the video source data; in addition, the real-time focusing device Take out the video source data of the same frame that is buffered, so as to compare the video source data with the captured image.

In other embodiments, all the received video source data may also be buffered, and then a set of video source data is retrieved from the stored video source data for projection display.

It can be understood that the acquisition of video source data may be performed every frame, or may be performed every several frames or several seconds.

Step 12: Process the video source data and the captured image to obtain feature information of the video source data and feature information of the captured image.

Whether it is necessary to focus can be judged according to the degree of blurring of the captured image, but if it is only determined by the degree of blurring of the captured image, it is difficult to determine whether the display is blurred due to blurring caused by defocusing. Therefore, the captured image and video source data can be compared. Contrast; specifically, only a part of the captured image can be compared with a part of the video source data, that is, a local comparison method is used, or the entire region of the captured image can be compared with all the data of the video source data, that is, a global comparison is used. Comparison method; if the video source data is clear and the captured image is clear, no focus adjustment is required; if the video source data is clear but the captured image is blurred, refocusing is required; further, after the captured image is acquired, image processing methods can be used to adjust the focus. The video source data and the captured image are analyzed and processed, and the feature information of the video source data and the feature information of the captured image are extracted respectively.

It can be understood that blurring and clarity are relative terms. If the difference between the sharpness of the video source data and the captured image is within a range, it can be determined that the two are equally clear or blurred; otherwise, the captured image can be determined to be blurred.

In a specific embodiment, the feature information includes feature points, and a specific image processing algorithm or a deep learning algorithm can be used to process the video source data and the captured image to obtain a plurality of feature points and captured images in the video source data respectively. multiple feature points in the image; for example, feature points in the image can be extracted using a feature extraction algorithm.

Step 13: Based on the feature information of the video source data and the feature information of the captured image, a control command is generated, and the control command is sent to the motor, so that the motor drives the projection lens to move and realize focusing.

The projection device includes a projection lens and a motor that are connected to each other. After the feature information is generated, a control command can be generated by using the feature information, and then the control command is sent to the motor, so that the motor drives the projection lens to move, so as to adjust the focal length and focus. position to achieve focus.

In a specific embodiment, the adjustment of the position of the projection lens is taken as an example for description, which specifically includes the following steps:

Step 131 : Determine the corresponding focus area based on the feature information of the video source data and the feature information of the captured image.

After the feature points of the video source data and the feature points of the captured image are extracted, these feature points can be used to determine the focus area, and the method shown in Figure 3 is used for processing, which specifically includes the following steps:

Step 1311 : Match multiple feature points in the video source data with multiple feature points in the captured image to determine a corresponding matching area.

The comparison is to compare the captured image with the source video data. However, due to the influence of the position or viewing angle of the camera, when the camera is used to shoot the picture displayed by the projection device, the captured picture (ie, the captured image) may be different from The actual projected images (ie, projected display images) are not exactly the same, and the captured images cannot represent the actual projected images. Therefore, in order to facilitate comparison, the video source data can be matched with the captured images first; further, the video sources can be compared. The data and the part of the captured image, find the video source data corresponding to at least part of the captured image, and establish the corresponding relationship between the video source data and the captured image; for example, as shown in Figure 4, the captured image is recorded as A, and the captured image The local area in A is denoted as B, and the pixel values in the local area B are matched with the video source data, and the pixel value in the video source data with the smallest difference between the pixel values in the local area B is found. The area composed of these pixel values is The area that matches the local area B is the matching area.

Further, before the matching, the distortion of the captured image can be removed, and then the corners or other feature points of the image can be found, and the positions of these corners or feature points in the video source data and the captured image correspond to relationship to find the matching area between the video source data and the captured image.

In a specific embodiment, in order to prevent the ambient light from affecting the image processing process, the ambient light subtraction process may be performed after the captured image is de-distorted; in the case of weak ambient light or uniform ambient light, the ambient light reduction process may not be performed. Ambient light subtraction processing; or the difference between two different frames can be used to achieve ambient light subtraction. If the difference between two frames is less than a certain threshold, ambient light subtraction processing is not required, for example, the display is still Ambient light subtraction processing is not performed on the screen.

Step 1312: Process the pixels of each matching area respectively to obtain a plurality of focus feature points.

After matching, in order to further decide which part of the image to use for comparison, one or more specific areas can be selected from the matching area, but if there is no sharp image in the found area, you need to use some special The algorithm has higher requirements on the algorithm and longer processing time. Therefore, it is possible to find the sharpest area in the matching area, that is, the high spatial frequency area, and the high spatial frequency area includes multiple focus feature points.

Further, a gradient operator, a Laplacian operator or other edge extraction operators can be used to process the matching area in the video source data and the matching area in the captured image, for example, to extract pixels that change drastically in the matching area. , to obtain a plurality of corresponding high-frequency pixel points; specifically, the position where the pixel value changes violently corresponds to the high-frequency signal area (ie, the high spatial frequency area) in the image, such as the edge; the position where the pixel value changes little is for the low-frequency signal area. , such as a large color block area; because the high-frequency signal area is generally the edge or contour of the image, which can better represent the contour information of the image, so the high-frequency signal area is selected to determine the focus situation.

Then, the pixel changes of each matching area can be counted, and filtered by the set threshold, and the area rich in sharply changed pixels is regarded as a high spatial frequency area, that is, it is determined whether the pixel value of each high frequency pixel is greater than the preset pixel. value; if the pixel value of the high-frequency pixel point is greater than the preset pixel value, the high-frequency pixel point is used as the focus feature point; if the pixel value of the high-frequency pixel point is less than or equal to the preset pixel value, the high-frequency pixel point is used. Pixels are discarded.

Step 1313: Connect the adjacent focus feature points among the multiple focus feature points respectively to obtain the corresponding focus area.

After acquiring the focus feature points, the adjacent focus feature points among the multiple focus features corresponding to the video source data can be connected. Specifically, the distance between each focus feature point and other focus feature points can be calculated, and the distance The two shortest focus feature points are used as adjacent focus feature points, and then the adjacent focus feature points are connected with straight lines to obtain a closed area, that is, the focus area of the video source data; according to the same method, corresponding to the captured image The focus area of the captured image can be obtained by processing the multiple focus feature points, so as to find the approximate focus area from the two images.

Understandably, a plurality of disjoint high spatial frequency regions can be generated simultaneously in order to obtain a more accurate focusing result.

Step 132: Acquire the sharpness of the focus area in the video source data, denoted as the first sharpness, obtain the sharpness of the focus area in the captured image, and record it as the second sharpness; based on the first sharpness and the second sharpness, determine The adjustment direction of the projection lens.

After acquiring the focus area, the image sharpness evaluation function can be used to calculate the first sharpness and the second sharpness, and then calculate the difference between the first sharpness and the second sharpness, and the difference of sharpness can be used to measure the focus situation and the defocus distance, the larger the difference between the two, the larger the defocus distance; specifically, the parameters corresponding to each high spatial frequency region that can indicate high frequency information can be calculated at the same time, and the parameter is used as the clarity, for example , the high frequency part of the spatial spectrum, the sum of the squares of the gradient values, or the sum of the absolute values of the Laplacian operator; in addition, in order to ensure comparison between different images, the difference between the two can also be calculated for area or total brightness and other parameters for normalization; if there are multiple high spatial frequency regions, each can be normalized according to the area and then averaged to obtain the final result.

In a specific embodiment, in order to improve the stability of the system and obtain a stable defocus distance, the final defocus distance can be filtered, for example, an average value filter can be used or the average value of the maximum value and the minimum value can be removed. filtering, etc.

Then it can be determined whether the difference between the first sharpness and the second sharpness is greater than the preset defocus threshold; if the difference between the first sharpness and the second sharpness is greater than the preset defocus threshold, feedback adjustment is performed, and the control Move the projection lens to a preset distance in any direction, and continue to judge the defocusing situation, that is, return to step 11. If the defocusing distance increases after the last adjustment, it means that the defocusing direction is opposite to the last adjustment direction; If the defocus distance decreases after the last adjustment, it means that the defocus direction is the same as the last adjustment direction; after judging the defocus direction, a single fine-tuning or multiple feedback fine-tuning can be performed until the first The difference between the sharpness and the second sharpness is less than or equal to the preset defocusing threshold, and the preset distance is smaller than the preset defocusing threshold, that is, the threshold of the defocusing degree is greater than the defocusing caused by moving the preset distance to prevent adjustment. After the preset distance, the optimal focusing position is exceeded; if the difference between the first sharpness and the second sharpness is less than or equal to the preset defocusing threshold, it is determined that the focusing is successful, and no adjustment is required.

In this embodiment, the matching area between the video source data and the captured image can be calculated according to the obtained feature points, and the corresponding focus feature points can be obtained by processing the pixels in the matching area; based on the distribution of the two groups of focus feature points, by connecting The approximate focus area can be obtained from adjacent focus feature points; according to the obtained focus area, the image sharpness evaluation function can be used to calculate the sharpness of the focus area respectively. If the focus is inaccurate, it will start feedback adjustment, and it can analyze how to complete the focus according to the extracted focus area, and realize automatic focus.

Please refer to FIG. 5. FIG. 5 is a schematic structural diagram of an embodiment of a real-time focusing device provided by the present application. The real-time focusing device 50 includes a memory 51 and a processor 52 that are connected to each other. The memory 51 is used for storing computer programs. When the controller 52 is executed, it is used to realize the above-mentioned real-time focusing method.

This embodiment provides a reliable real-time focusing device 50, which can solve the problem that when the projection device is out of focus, the user does not need to manually determine and control the focus, but can automatically determine the out-of-focus situation. When displaying any video image Capable of real-time autofocus, so that the user experience is not affected, and defocus caused by thermal defocus and vibration can be corrected in real time.

Please refer to FIG. 6. FIG. 6 is a schematic structural diagram of an embodiment of a projection system provided by the present application. The projection system includes: a real-time focusing device 61, a projection device 62, and a camera device 63. The projection device 62 includes a projection lens 621 and a projection screen 622. The imaging device 63 may be a camera.

Before the real-time focusing process starts, the internal and external parameters of the projection lens 621 and the camera device 63 can be calibrated, and the calibration parameters can be saved in the real-time focusing device 61. The calibration parameters include: the distortion parameters of the projection device 62 and the camera device 63 Distortion parameters , the relative position and relative direction of the projection device 62 and the camera device 63 and other information.

After the calibration is completed, the corresponding relationship between the projection pixels and the camera pixels can be roughly confirmed. However, since the distance of the projection screen 622 is not determined, the corresponding relationship between the projection area and the photographing area cannot be determined even after the internal and external parameters are calibrated. The coordinates of each corner point on the camera negative will be described below on how to determine the corresponding relationship.

For fixed installations or fixed usage scenarios, you can first subtract the black and white images to get the projection area after installation, and use traditional corner detection or line detection algorithms (such as Hough transform) to add The upper clustering algorithm (such as K-means clustering algorithm) obtains the coordinates of the four corners; after obtaining the coordinates of the four corners inside and outside the projection lens 621 and the camera, you can use the four-point transformation to obtain the projection after de-distortion The corresponding relationship between the pixel coordinates and the camera pixel coordinates; it is also possible to use two opposite black and white grid images of a specific horizontal and vertical number to subtract, and then use corner detection, line detection or clustering algorithms to obtain a more accurate alignment relationship.

For more complex usage scenarios, for example, when the relative position of the projection lens 621 and the projection screen 622 may change at any time, in order to achieve non-sensing focusing, two adjacent frames of video source data can be down-sampled and made difference, and statistics for each The difference value of each area, and the area with a large difference between the two frames is used as the characteristic area. At the same time, the actual display image corresponding to the two frames of video source data is made difference, and the characteristic area in the video source data is detected in the actual display image. position (for example, using the cross-correlation function of the sliding window), the correspondence between the projected pixels and the camera pixels can be confirmed.

After the correspondence between the projection pixels and the camera pixels is determined, the distance between the projection lens 621 and the projection screen 622 is confirmed, and the parameters of the focus position can be roughly determined according to the design parameters of the projection lens 621; however, due to thermal defocus and other reasons , the precise focusing parameters cannot be determined, so the real-time focusing device 61 provided in this embodiment can be used for adjustment to realize focusing.

The real-time focusing device 61 is used to obtain video source data; the projection device 62 is connected to the real-time focusing device 61, which is used to receive the video source data sent by the real-time focusing device 61, perform projection display, and form a projection display image; specifically, the projection lens 621 The focusing position of the camera is controlled by the real-time focusing device 61 . After receiving the video source data sent by the real-time focusing device 61 , the projection lens 621 can project it out in real time to form a captured image on the projection screen 622 .

The camera device 63 is connected to the real-time focusing device 61, which is used for shooting the projection display image displayed by the projection device 62 to obtain the captured image corresponding to the video source data; After the synchronous shooting signal is received, the projection display image generated by the projection device 62 is captured to obtain the captured image, and the captured image is sent back to the real-time focusing device 61 .

The real-time focusing device 61 is also used to process the video source data and the captured image to obtain the feature information of the video source data and the feature information of the captured image; based on the feature information of the video source data and the feature information of the captured image, a control instruction is generated to The control command is sent to the motor (not shown in the figure), so that the motor drives the projection lens 621 to move to achieve focusing; further, the position of the projection lens 621 or the parameters of the projection lens 621 can be adjusted, and this embodiment only adjusts the projection lens The position of 621 is taken as an example for description.

In a specific embodiment, as shown in FIG. 6 , the projection system further includes a video source 64, and the video source 64 is used to send the video source data to the synchronization module 6111; the real-time focusing device 61 includes a processor 611 and a memory 612 that are connected to each other. , the processor 611 is used to receive the video source data; the memory 612 is used to receive the video source data sent by the processor 611 and store the video source data, and the processor 611 can also obtain the video source data from the memory 612 and receive the camera device 63 sent captured images.

The synchronization module 6111 is connected to the memory 612 and the video source 64 , and is used for receiving the video source data and the captured image sent by the camera 63 , and storing the video source data in the memory 612 .

The feature extraction module 6112 is connected to the synchronization module 6111, and is used for processing the received video source data and the captured image to obtain multiple feature points in the video source data and multiple feature points in the captured image.

The focus decision module 6113 is connected with the feature extraction module 6112, which is used to determine the adjustment direction of the projection lens 621 based on the multiple feature points of the video source data and the multiple feature points of the captured image, and generate a control instruction corresponding to the adjustment direction, and Control commands are sent to the motor.

Further, while controlling the projection device 62 to display the video source data, the synchronization module 6111 can cache the image projected by the projection device 62, and after receiving the picture captured by the camera device 63, the corresponding video source data is stored from the memory. Take out in 612, and send the captured image and video source data to the feature extraction module 6112. The feature extraction module 6112 can use a specific image processing algorithm or a deep learning algorithm to convert the input image into feature information related to the focus intensity, and the The feature information is sent to the focus decision module 6113, and the focus decision module 6113 can analyze the focus situation and control the projection device 62 to perform focus adjustment.

The focus decision module 6113 can analyze how to complete the focus according to the extracted focus area; specifically, first calculate the focus area of the video source data and the captured image according to the obtained feature points, because the two sets of feature points have been matched in the feature extraction module 6112. , so the adjacent feature points can be connected according to the distribution of the two sets of feature points, and the approximate focus area can be obtained from the two images; then the image sharpness evaluation function can be used to calculate the sharpness of the focus area separately, according to the difference in sharpness Determine whether the current focus is accurate, if not, start feedback adjustment, that is, control the projection lens 621 to move a preset distance in any direction, and determine the defocusing situation after the movement, if the defocusing distance increases after the last adjustment is completed, then Indicates that the defocusing direction is opposite to the last adjustment direction, otherwise the defocusing direction is the same as the last adjustment direction; after judging the defocusing direction, you can continue to adjust until the defocusing distance is within the acceptable range.

This embodiment proposes a projection system that can focus in real time. The projection system includes a synchronization module 6111 , a feature extraction module 6112 and a focus decision module 6113 . The synchronization module 6111 is responsible for buffering the video source data sent by the video source 64 , and controls the camera 63 to Capture the image at the correct time, generate the captured image, and pass the video source data and its corresponding captured image to the feature extraction module 6112; the feature extraction module 6112 extracts and matches the feature information that does not depend on ambient light from the two synchronized images; The focus decision module 6113 calculates the focus area according to the input feature information, and calculates the sharpness of the focus area in the video source data and the sharpness (sharpness) of the focus area in the captured image, thereby judging the adjustment direction of the projection lens 621, and the output control The command is sent to the motor, and the motor can drive the projection lens 621 to move under the control of the control command to realize real-time non-sensing focusing, which can solve the thermal defocusing and other defocusing situations of the projection device 62 during the playback process, and provide users with better viewing experience.

Please refer to FIG. 8. FIG. 8 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided by the present application. The computer-readable storage medium 80 is used to store a computer program 81, and when the computer program 81 is executed by a processor, it is used to realize The real-time focusing method in the above embodiment.

The computer-readable storage medium 80 can be a server, a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, etc. medium of program code.

In the several embodiments provided in this application, it should be understood that the disclosed method and device may be implemented in other manners. For example, the device implementations described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other divisions, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.

Units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this implementation manner.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The above are only the embodiments of the present application, and are not intended to limit the scope of the patent of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present application, or directly or indirectly applied in other related technical fields, All are similarly included in the scope of patent protection of the present application.

Claims

A real-time focusing method, comprising:

Obtain video source data and capture images;

Process the video source data and the captured image to obtain feature information of the video source data and feature information of the captured image;

Based on the feature information of the video source data and the feature information of the captured image, a control command is generated, and the control command is sent to a motor, so that the motor drives the projection lens to move and realize focusing.
The real-time focusing method according to claim 1, wherein the control command is generated based on the feature information of the video source data and the feature information of the captured image, and the control command is sent to the motor, so that the The step of the motor driving the projection lens to move includes:

Determine the corresponding focus area based on the feature information of the video source data and the feature information of the captured image;

Obtain the sharpness of the focus area in the video source data, and record it as the first sharpness;

Obtain the sharpness of the focus area in the captured image, and record it as the second sharpness;

Based on the first definition and the second definition, the adjustment direction of the projection lens is determined.
The real-time focusing method according to claim 2, wherein the method further comprises:

Calculate the first sharpness and the second sharpness by using an image sharpness evaluation function;

calculating the difference between the first definition and the second definition;

judging whether the difference between the first sharpness and the second sharpness is greater than a preset defocus threshold;

If yes, control the projection lens to move a preset distance in any direction, and return to the step of acquiring the video source data and the captured image, until the difference between the first definition and the second definition is less than or equal to the the preset defocus threshold;

If not, it is determined that the focus is successful;

Wherein, the preset distance is smaller than the preset defocus threshold.
The real-time focusing method according to claim 2, wherein the feature information includes a plurality of feature points, and the corresponding focus area is determined based on the feature information of the video source data and the feature information of the captured image steps, including:

Matching a plurality of feature points in the video source data with a plurality of feature points in the captured image to determine a corresponding matching area;

respectively processing the pixels of each of the matching areas to obtain a plurality of focus feature points;

The adjacent focus feature points among the plurality of focus feature points are respectively connected to obtain the corresponding focus area.
The real-time focusing method according to claim 4, wherein the step of separately processing the pixels of each of the matching regions to obtain a plurality of focusing feature points comprises:

Using a gradient operator or a Laplacian operator to process the matching area in the video source data and the matching area in the captured image, respectively, to obtain a plurality of high-frequency pixel points;

Determine whether the pixel value of each of the high-frequency pixel points is greater than the preset pixel value;

If so, use the high-frequency pixel point as the focus feature point.
The real-time focusing method according to claim 1, wherein the step of acquiring the video source data and the captured image comprises:

sending the video source data to a projection device for projection display;

A synchronous shooting signal is sent to the camera device to control the camera device to shoot the projected display image on the projection screen to obtain the shot image corresponding to the video source data.
A real-time focusing device, characterized in that it comprises a memory and a processor that are connected to each other, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, it is used to implement claims 1- The real-time focusing method described in any one of 6.
A projection system, characterized in that, comprising:

Real-time focusing device for receiving video source data;

a projection device, connected to the real-time focusing device, for receiving the video source data sent by the real-time focusing device, and performing projection display, wherein the projection device includes a projection lens and a motor connected to each other;

an imaging device, connected with the real-time focusing device, for shooting the projection display image displayed by the projection device to obtain the captured image corresponding to the video source data;

Wherein, the real-time focusing device is further configured to process the video source data and the captured image to obtain feature information of the video source data and feature information of the captured image; based on the features of the video source data The information and the feature information of the captured image are used to generate a control command, and the control command is sent to the motor, so that the motor drives the projection lens to move and realize focusing.
The projection system according to claim 8, wherein the real-time focusing device comprises:

a processor for receiving the video source data;

a memory, connected to the processor, for receiving the video source data sent by the processor, and storing the video source data;

Wherein, the processor is further configured to acquire the video source data from the memory, and receive the captured image sent by the camera device.
The projection system according to claim 9, wherein the feature information comprises a plurality of feature points, the projection device further comprises a projection screen, and the processor comprises:

a synchronization module, connected with the memory, for receiving the video source data and the captured image, and storing the video source data in the memory;

A feature extraction module, connected with the synchronization module, is used to process the received video source data and the captured image, and obtain multiple feature points in the video source data and multiple feature points in the captured image. feature points;

A focus decision-making module, connected with the feature extraction module, is used to determine the adjustment direction of the projection lens based on the multiple feature points of the video source data and the multiple feature points of the captured image, and generate and adjust the adjustment direction of the projection lens. The control command corresponding to the direction is sent to the motor.
The projection system according to claim 10, wherein,

The projection system further includes a video source, the video source is connected to the synchronization module, and is used for sending projection data to the synchronization module, wherein the projection data includes at least one frame of video source data.
A computer-readable storage medium for storing a computer program, characterized in that, when the computer program is executed by a processor, the computer program is used to implement the real-time focusing method according to any one of claims 1-6.