CN115601446A - Method and device for optimizing linearity of iToF camera and electronic equipment - Google Patents
Method and device for optimizing linearity of iToF camera and electronic equipment Download PDFInfo
- Publication number
- CN115601446A CN115601446A CN202211291852.1A CN202211291852A CN115601446A CN 115601446 A CN115601446 A CN 115601446A CN 202211291852 A CN202211291852 A CN 202211291852A CN 115601446 A CN115601446 A CN 115601446A
- Authority
- CN
- China
- Prior art keywords
- duty ratio
- optimal
- value
- phase
- virtual real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000005259 measurement Methods 0.000 claims abstract description 58
- 238000005457 optimization Methods 0.000 claims abstract description 38
- 238000010586 diagram Methods 0.000 description 11
- 238000004364 calculation method Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 238000012935 Averaging Methods 0.000 description 4
- 238000013459 approach Methods 0.000 description 4
- 238000012937 correction Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000009826 distribution Methods 0.000 description 3
- 230000000737 periodic effect Effects 0.000 description 3
- 230000010363 phase shift Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20216—Image averaging
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Measurement Of Optical Distance (AREA)
Abstract
The application relates to a linearity optimization method and device for an iToF camera. The method comprises the following steps: adjusting, by the image sensor, a duty cycle of the emitted light waveform; adjusting a delay value at a preset step length under the current duty ratio, and acquiring a virtual real phase and a target measurement phase corresponding to a corresponding virtual real distance after adjusting the delay value each time; after the delay value of the preset period is adjusted, acquiring linear parameters of the current duty ratio according to all the acquired virtual real phases and the corresponding target measurement phases; when the current duty ratio is smaller than a preset threshold value, acquiring the duty ratio corresponding to the optimal value in all the linear parameters as the optimal duty ratio; and taking the optimal duty ratio as a setting parameter of the image sensor to finish the nonlinear calibration of the iToF camera and realize the linearity optimization of the iToF camera. The method and the device can reduce the nonlinearity of the system, save the calibration cost and improve the linearity of the image sensor measurement.
Description
Technical Field
The application relates to the technical field of ToF ranging, in particular to a linearity optimization method and device for an iToF camera and electronic equipment.
Background
Binocular ranging, structured light and Time-of-Flight (ToF for short) are three major 3D imaging technologies at present, wherein ToF has been gradually applied to the fields of gesture recognition, 3D modeling, unmanned driving, machine vision and the like due to the advantages of simple principle, simple and stable structure, long measurement distance and the like. The working principle of the ToF technology is as follows: the method comprises the steps that continuously modulated emission light is emitted by an external light source (VCSEL or LED or the like), the emission light irradiates the surface of an object to be measured and then is reflected back, the reflection light is captured by an image sensor (sensor) of an iToF camera, and the depth/distance of the object from the camera is obtained by calculating the time difference or phase difference between the emission light and the reflection light. Among them, a method of calculating a distance by a time difference is called a pulse method (Pulsed ToF), and a method of calculating a distance by a phase difference is called a Continuous Wave method (Continuous-Wave ToF).
Indirect Time-of-Flight (iToF) refers to the indirect measurement of the Time of Flight of light by measuring the phase shift. As shown in fig. 1, the iToF camera controls a light emitting module 12 to actively emit a modulatable light signal through a modulation module (modulation) 11; the emitted light is emitted to the surface of a target object 19 to be measured, and a reflected light signal formed after the reflected light is reflected by the target object 19 is sampled by a photosensitive pixel array unit 13 of the image sensor; further, the distance to the target is calculated from the Phase shift (Phase shift) of the emitted light and the reflected light. The light emitting module 12, such as VCSEL, infrared emitter (IR emitter) or LED, is usually driven by the sensor to generate a modulatable square wave, but since the light waveform gradually approaches the sine wave with the increase of the modulation frequency, the higher harmonics in the square wave will bring periodic errors to the measurement, as shown in fig. 2. A wobble error (ringing error) exists in the measurement process due to the presence of aliased harmonics in the correlation waveform, as shown in fig. 3.
A nonlinear error lookup table is directly established through calibration, so that the swing error can be well corrected in principle; however, since multiple distance measurements are required and each measurement needs to be averaged multiple times to remove random noise well, the calibration cost is increased. Nonlinear errors caused by multiple harmonics can be compensated by means of multiple measurements (more than four phases); however, since the iToF camera is usually a global exposure, if a depth map is obtained through multiple measurements, it is equivalent to stretch the depth map from time, which may cause smearing for some moving objects; in addition, due to the increase of measurement data required by one depth map, the load of the system is increased, and the dynamic power consumption of the system is increased.
Therefore, how to reduce the calibration cost of the iToF camera and improve the linearity of measurement is a technical problem to be solved urgently at present.
Disclosure of Invention
The application aims to provide a linearity optimization method and device for an iToF camera and electronic equipment, which are used for solving the problems that the existing iToF camera is high in calibration cost and high in dynamic power consumption of a system, so that the calibration cost of the iToF camera is saved, a proper duty ratio is quickly selected, and the measurement linearity is improved.
In order to achieve the above object, the present application provides a linearity optimization method for an iToF camera, including the following steps: adjusting, by an image sensor of the iToF camera, a duty cycle of an emitted light waveform; adjusting a delay value at a preset step length under the current duty ratio, and acquiring a virtual real phase and a target measurement phase corresponding to a corresponding virtual real distance after adjusting the delay value each time; after the delay value of the preset period is adjusted, acquiring linear parameters of the current duty ratio according to all the acquired virtual real phases and the corresponding target measurement phases; when the current duty ratio is smaller than a preset threshold value, acquiring the duty ratio corresponding to the optimal value in all the linear parameters as the optimal duty ratio; and taking the optimal duty ratio as a setting parameter of the image sensor to finish the nonlinear calibration of the iToF camera and realize the linearity optimization of the iToF camera.
In some embodiments, the method further comprises: selecting a plurality of image sensors and acquiring the optimal duty ratio of each image sensor; obtaining a target duty ratio according to all the optimal duty ratios, wherein the target duty ratio is a median or an average value of all the optimal duty ratios; and backfilling the target duty cycle into settings of all the image sensors.
To achieve the above object, the present application further provides a linearity optimizing apparatus of an iToF camera, including: the adjusting module is used for adjusting the duty ratio of the emitted light waveform through the image sensor; the first acquisition module is used for adjusting a delay value in a preset step length under the current duty ratio and acquiring a virtual real phase and a target measurement phase corresponding to a corresponding virtual real distance after the delay value is adjusted each time; the second acquisition module is used for acquiring linear parameters of the current duty ratio according to all the acquired virtual real phases and the corresponding target measurement phases after the delay value of the preset period is adjusted; the third obtaining module is used for obtaining the duty ratio corresponding to the optimal value in all the linear parameters as the optimal duty ratio when the current duty ratio is smaller than the preset threshold value; and the optimization module is used for finishing the nonlinear calibration of the iToF camera and realizing the linearity optimization of the iToF camera by taking the optimal duty ratio as a setting parameter of the image sensor.
In some embodiments, the second obtaining module is further configured to obtain an absolute value of a difference between the virtual real phase and the target measured phase obtained after the delay value is adjusted each time, and obtain a sum of all the absolute values of the difference in the preset period, where the sum is used as a linear parameter of the current duty cycle; the third obtaining module is further configured to obtain a minimum value of the sum values corresponding to all duty ratios as the optimal value.
To achieve the above object, the present application also provides an electronic device, including a memory, a processor, and a computer-executable program stored on the memory and executable on the processor, where the processor executes the computer-executable program to implement the steps of the linearity optimization method of the iToF camera according to the present application.
According to the method and the device for optimizing the linearity of the iToF camera, the duty ratio of the optical waveform is adjusted through the image sensor, aliasing harmonic components can be inhibited to improve the linearity of measurement of the image sensor, the nonlinearity of the system is reduced, the calibration time required for wiggling correction in phase calculation is saved, and the calibration cost is saved. And then adding time delay through a time delay unit to simulate real distance movement, and searching for an optimal duty ratio through constraint adjustment so as to reduce the nonlinearity to the maximum extent and better improve the linearity measured by the image sensor. The method comprises the steps of randomly selecting a plurality of modules from the same batch of image sensor modules, simultaneously carrying out duty ratio optimization search, determining an optimal solution according to the median or average value of the optimal duty ratios of all the image sensor modules, and backfilling the optimal solution into the setting of all the image sensor modules of the batch, so that the nonlinear calibration of all the image sensor modules of the batch can be completed, the calibration accuracy can be improved, and the calibration time of the image sensor modules of the same batch can be saved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that are required to be used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
FIG. 1 is a schematic illustration of the principle of iToF imaging;
FIG. 2 is a graph of a periodic error due to higher harmonics;
FIG. 3 shows wobble error during measurement;
fig. 4 is a schematic diagram illustrating a method for optimizing linearity of an iToF camera according to an embodiment of the present disclosure;
FIG. 5 is a phase distribution diagram under different duty cycles according to an embodiment of the present disclosure;
FIG. 6 is a diagram illustrating the deviation between modulation and demodulation;
FIG. 7 is a schematic diagram of a virtual calibration plate simulated by a delay circuit;
fig. 8 is a flowchart of a linearity optimization method for an iToF camera according to an embodiment of the present disclosure;
fig. 9 is a block diagram of a linearity optimization apparatus of an iToF camera according to an embodiment of the present disclosure.
Detailed Description
The technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
An embodiment of the application provides a linearity optimization method for an iToF camera.
Fig. 4-7 are also shown, in which fig. 4 is a schematic diagram of a method for optimizing linearity of an iToF camera according to an embodiment of the present disclosure, fig. 5 is a schematic diagram of a phase distribution diagram under different duty ratios according to an embodiment of the present disclosure, fig. 6 is a schematic diagram of a deviation between modulation and demodulation, and fig. 7 is a schematic diagram of a virtual calibration board simulated by a delay circuit.
As shown in fig. 4, the method for optimizing linearity of an iToF camera in this embodiment includes the following steps: s1, adjusting the duty ratio of an emitted light waveform through an image sensor of an iToF camera; s2, adjusting a delay value at a preset step length under the current duty ratio, and acquiring a virtual real phase and a target measurement phase corresponding to a corresponding virtual real distance after adjusting the delay value each time; s3, after the delay value of the preset period is adjusted, acquiring linear parameters of the current duty ratio according to all the acquired virtual real phases and the corresponding target measurement phases; s4, when the current duty ratio is smaller than a preset threshold value, acquiring the duty ratio corresponding to the optimal value in all the linear parameters as the optimal duty ratio; and S5, taking the optimal duty ratio as a setting parameter of the image sensor, completing the nonlinear calibration of the iToF camera, and realizing the linearity optimization of the iToF camera.
With respect to step S1, the duty cycle of the emitted light waveform is adjusted by the image sensor of the iToF camera. Specifically, the light emitting module is usually driven by generating a square wave from an image sensor of the iToF camera, and higher harmonics in the square wave are also main factors causing measurement nonlinearity, which shows periodic variation (as shown in fig. 2). The light waveform also gradually approaches to the sine wave along with the increase of the modulation frequency, and the light waveform influences the linearity of the calculated phase. Therefore, the duty cycle (duty cycle) of the optical waveform is adjusted by the image sensor, aliasing harmonic components can be inhibited to improve the linearity of the image sensor measurement, the nonlinearity of the system is reduced, the calibration time required for wiggling correction in phase calculation is saved, and the calibration cost is saved. And then adding time delay through a time delay unit to simulate real distance movement, and searching for an optimal duty ratio through constraint adjustment so as to reduce the nonlinearity to the maximum extent and improve the linearity of the image sensor measurement better.
The modulation waveform approaches to a sine wave, and the following calculation formula can be adopted for calculating the phase according to the sine wave:
phase＝arctan(I/Q)；
wherein, I = (Q) 3 -Q 4 )、Q＝(Q 1 -Q 2 )；Q 1 For measuring phase with a phase delay of 0 DEG, Q 2 For measuring phase, Q, at a phase delay of 90 DEG 3 For measuring phase, Q, with a phase delay of 180 DEG 4 The measured phase is the phase with a phase delay of 270. I and Q are orthogonal vectors, ideally distributed on a circle. The optical waveform affects the linearity of the calculated phase, so the nonlinearity of the system can be reduced by adjusting the duty ratio of the optical waveform through the image sensor. By adjusting the duty cycle, the I and Q profiles approach a circle more as the duty cycle is reduced, as shown in fig. 5.
The following calculation formula can be adopted according to the phase acquisition depth:
d＝c*(phase/(2*f*phase_max))+d_max*n；
where c is the speed of light, phase is the measured phase, phase _ max is a phase period (e.g., 2 π), f is the modulation frequency of the emitted light, d _ max = (c/2) × (1/f), n is the predetermined number of frames and n ∈ [0,1 … … ].
And S2, adjusting the delay value at a preset step length under the current duty ratio, and acquiring a virtual real phase and a target measurement phase corresponding to the corresponding virtual real distance after adjusting the delay value every time. Specifically, the preset step size can be any one of pi/4, pi/8 and pi/16. In order to improve the calibration precision, the preset step length can also be a smaller value; the adjustment step length of the delay value can be set according to the comprehensive consideration of the calibration precision and the calibration time.
Ideally, modulation and demodulation have a certain skew (skew) due to circuit and manufacturing process, etc., as shown in fig. 6. These fixed deviations can be fixed by adding a delay unit in the modulation or demodulation path, setting certain delay parameters. Then, the real distance is simulated by setting the delay value, i.e., a virtual real distance is generated, as shown in fig. 7. And further, a virtual real phase and a target measurement phase corresponding to the corresponding virtual real distance can be obtained.
In some embodiments, the step of obtaining the virtual real phase and the target measurement phase corresponding to the corresponding virtual real distance after adjusting the delay value in step S2 further includes: 1) Adding a delay unit in a modulation or demodulation path, and setting a delay value to simulate a real distance; 2) Forming a current virtual real distance after adjusting the delay value every time, wherein the current virtual real distance corresponds to a virtual real phase; and 3) acquiring a preset frame depth image, and performing time domain average on the measurement phases of the pixel center points of all the depth images to acquire a corresponding average measurement phase to serve as a target measurement phase corresponding to the current virtual real distance.
The measurement phase of each pixel point in the depth image acquired by the image sensor contains certain random noise, and the transfer function of each pixel point is consistent, so that the requirement of time domain average convergence to a fixed value after preset times is met. Therefore, random noise is removed by performing time domain averaging on the measured phases of the pixel center points of all the depth images. The number of frames of the depth images to be captured can be set according to the requirement of calibration precision to carry out time domain averaging.
For example, m frames of depth images are captured by the image sensor, and after time-domain averaging is performed on the measured phases of the pixel center points of the m frames of depth images, random noise converges to a fixed value (e.g., 0). Specifically, the obtained average measurement phase satisfies the following formula:
phase i ＝phase+ε m ；
wherein, phase i The ith time phase measurement of the pixel center point is the theoretical measurement value of the pixel point after random noise is removed, epsilon m Is a fixed value to which the random noise converges after m time domain averages. Preferably, based on the normal distribution form of the noise, ε m ＝0。
And S3, after the delay value of the preset period is adjusted, acquiring linear parameters of the current duty ratio according to all the acquired virtual real phases and the corresponding target measurement phases. Specifically, the preset period is 2 pi (i.e., one complete period). Under the current duty ratio, a virtual real phase of a complete period and a corresponding target measurement phase are obtained by adjusting a delay value, and then the linear parameters of the current duty ratio can be obtained.
In some embodiments, the step of obtaining the linear parameter of the current duty ratio according to all the obtained virtual real phases and the corresponding target measured phases in step S3 further includes: 1) Obtaining the absolute value of the difference between the virtual real phase and the target measurement phase obtained after the delay value is adjusted each time; and 2) solving the sum of all the absolute values of the difference values in the preset period, and taking the sum as a linear parameter of the current duty ratio.
And after the delay value adjustment of a preset period is completed under the current duty ratio, obtaining a group of measurement phases measure _ avg [ i ] and virtual real phases real [ i ] under the virtual real distance. The linear parameter duty _ cycle of the current duty ratio can be obtained through the following formula:
duty_cycle＝sum(abs(measure_avg[i]-real[i]))。
and S4, when the current duty ratio is smaller than the preset threshold value, acquiring the duty ratio corresponding to the optimal value in all the linear parameters as the optimal duty ratio. Specifically, the duty cycle threshold is preset by the system, that is, the minimum duty cycle that can be set by the system. By acquiring all linear parameters before the adjustment to the preset threshold and constraining the optimal solution, the optimal duty ratio of the system can be acquired.
In some embodiments, the constraint for obtaining the optimal duty cycle is a sum of absolute values of differences between the pseudo-real phase and the target measured phase at each duty cycle. Specifically, the optimal solution of the duty ratio may be obtained by the following formula:
duty-cycle optimization of ＝arg_min(sum(abs(measure_avg[i]-real[i])))。
That is, when the absolute value of the deviation between the phase of the target map quantity and the virtual real phase is minimum under a certain duty ratio, the duty ratio is the optimal duty ratio.
And S5, taking the optimal duty ratio as a setting parameter of the image sensor, completing the nonlinear calibration of the iToF camera, and realizing the linearity optimization of the iToF camera. Specifically, after the optimal duty ratio is obtained, the duty ratio parameter of the image sensor is set to the optimal duty ratio, so that the nonlinearity can be reduced to the maximum extent, and the linearity of the measurement of the image sensor can be improved. In the embodiment, the duty ratio of the emission light waveform is adjusted to replace the traditional wiggling calibration, so that the calibration time required for wiggling correction in phase calculation is saved.
In some embodiments, the method further comprises: 1) Selecting a plurality of image sensors and acquiring the optimal duty ratio of each image sensor; 2) Obtaining a target duty ratio according to all the optimal duty ratios, wherein the target duty ratio is a median or an average value of all the optimal duty ratios; and 3) backfilling the target duty cycle into the settings of all of the image sensors. Specifically, the influence of random error factors existing in a single image sensor module is considered, so that a plurality of image sensor modules are selected to simultaneously carry out duty ratio optimization search, an optimal solution is determined according to the median or average value of the optimal duty ratios of all the image sensor modules, nonlinear calibration is completed, and the calibration accuracy is improved.
In the above embodiment, the step of selecting a plurality of image sensors further includes: randomly selecting a plurality of image sensors from the image sensors of the same batch; the step of backfilling the target duty cycle into the settings of all of the image sensors further comprises: backfilling the target duty cycle into settings of all image sensors of the batch. Specifically, several modules are randomly selected from the image sensor modules in the same batch to carry out duty ratio optimization search at the same time, an optimal solution is determined according to the median or average value of the optimal duty ratios of all the image sensor modules, and the optimal solution is backfilled into the settings of all the image sensor modules in the batch, so that the nonlinear calibration of all the image sensor modules in the batch can be completed, and the calibration time of the image sensor modules in the same batch is saved.
The flow of the linearity optimization method of the iToF camera of the present application is further explained with reference to fig. 8. The specific process of this embodiment is as follows: 1) Selecting n image sensor modules from the same batch of modules; 2) Adjusting the duty cycle; 3) Adjusting a delay value by taking pi/4 as a step length; 4) Capturing m frames of depth images, performing time domain averaging on the measurement phases of the pixel center points of all the depth images to obtain corresponding average measurement phases, and taking the average measurement phases as target measurement phases corresponding to the current virtual real distance, namely obtaining the measurement phases corresponding to the center points under the current delay value; 5) Acquiring measurement phases corresponding to the central points under different delay values, namely acquiring an array measure _ avg [ i ]; 6) Obtaining a virtual real phase according to the virtual real distance corresponding to the delay value, namely obtaining real [ i ]; 7) Judging whether the current delay value is less than or equal to a preset period (for example, 2 pi), namely judging delay < =2 pi; if delay < =2 pi, returning to continue to increase the delay value and follow-up operation, and if delay >2 pi, completing the adjustment of the delay value under the current duty ratio; 8) With Sum (abs (measure _ avg [ i ]) -real [ i ])) as a constraint condition, the minimum duty ratio under all duty ratios is the minimum duty ratio.
According to the content, the duty ratio of the optical waveform is adjusted through the image sensor, the nonlinearity of the system can be reduced, aliasing harmonic components can be inhibited to improve the linearity of the image sensor measurement, the nonlinearity of the system is reduced, the calibration time required for wiggling correction in phase calculation is saved, and the calibration cost is saved. And then adding time delay through a time delay unit to simulate real distance movement, and searching for an optimal duty ratio through constraint adjustment so as to reduce the nonlinearity to the maximum extent and better improve the linearity measured by the image sensor. The method comprises the steps of randomly selecting a plurality of modules from the same batch of image sensor modules, simultaneously carrying out duty ratio optimization search, determining an optimal solution according to the median or average value of the optimal duty ratios of all the image sensor modules, and backfilling the optimal solution into the setting of all the image sensor modules of the batch, so that the nonlinear calibration of all the image sensor modules of the batch can be completed, the calibration accuracy can be improved, and the calibration time of the image sensor modules of the same batch can be saved.
Based on the same inventive concept, the application also provides a linearity optimization device of the iToF camera. The linearity optimization device of the iToF camera provided can adopt the linearity optimization method of the iToF camera shown in fig. 4 to complete the linearity optimization of the iToF camera.
Please refer to fig. 9, which is a block diagram illustrating a linearity optimization apparatus of an iToF camera according to an embodiment of the present application. As shown in fig. 9, the linearity optimizing apparatus of the iToF camera includes: an adjustment module 101, a first acquisition module 102, a second acquisition module 103, a third acquisition module 104, and an optimization module 105.
Specifically, the adjusting module 101 is used for adjusting a duty cycle (duty cycle) of the emitted light waveform through the image sensor. The first obtaining module 102 is configured to adjust a delay value at a preset step length in a current duty cycle, and obtain a virtual real phase and a target measurement phase corresponding to a corresponding virtual real distance after adjusting the delay value each time. The second obtaining module 103 is configured to obtain a linear parameter of the current duty ratio according to all obtained virtual real phases and corresponding target measurement phases after the delay value adjustment of the preset period is completed. The third obtaining module 104 is configured to obtain, as the optimal duty ratio, a duty ratio corresponding to an optimal value in all linear parameters when the current duty ratio is smaller than a preset threshold. The optimization module 105 is configured to use the optimal duty ratio as a setting parameter of the image sensor, complete nonlinear calibration of the iToF camera, and implement linearity optimization of the iToF camera.
In some embodiments, the second obtaining module 103 is further configured to obtain an absolute value of a difference between the virtual real phase obtained after each adjustment of the delay value and the target measured phase, and obtain a sum of all the absolute values of the difference in the preset period, where the sum is used as a linear parameter of the current duty cycle. Correspondingly, the third obtaining module 104 is further configured to obtain a minimum value of the sums corresponding to all duty ratios as the optimal value.
Based on the same inventive concept, the application also provides an electronic device, which comprises a memory, a processor and a computer executable program stored on the memory and capable of running on the processor; the processor, when executing the computer executable program, implements the steps of the method for linearity optimization of an iToF camera as shown in fig. 4.
It is within the scope of the present inventive concept that embodiments may be described and illustrated in terms of modules that perform one or more of the described functions. These modules (which may also be referred to herein as cells, etc.) may be physically implemented by analog and/or digital circuitry, such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuitry, etc., and may optionally be driven by firmware and/or software. The circuitry may be implemented in one or more semiconductor chips, for example. The circuitry making up the modules may be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware to perform some of the functions of the module and a processor to perform other functions of the module. Each module of an embodiment may be physically separated into two or more interacting and discrete modules without departing from the scope of the present inventive concept. Likewise, the modules of the embodiments may be physically combined into more complex modules without departing from the scope of the present inventive concept.
Generally, terms may be understood at least in part from their usage in context. For example, the term "one or more" as used herein may be used in a singular sense to describe a feature, structure, or characteristic, or may be used in a plural sense to describe a feature, structure, or combination of features, at least in part, depending on the context. Additionally, the term "based on" may be understood as not necessarily intended to convey an exclusive set of factors, but may instead allow for the presence of other factors not necessarily expressly described, again depending at least in part on the context.
It should be noted that the terms "comprises" and "comprising," and variations thereof, as used herein, are intended to cover non-exclusive inclusions. The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order, unless otherwise clearly indicated by the context, and it is to be understood that the data so used is interchangeable under appropriate circumstances. In addition, the embodiments and features of the embodiments in the present application may be combined with each other without conflict. Moreover, in the foregoing description, descriptions of well-known components and techniques are omitted so as to not unnecessarily obscure the concepts of the present application. In the above embodiments, each embodiment is described with emphasis on differences from other embodiments, and the same/similar parts among the embodiments may be referred to each other.
The foregoing is only a preferred embodiment of the present application and it should be noted that, for a person skilled in the art, several modifications and refinements can be made without departing from the principle of the present application, and these modifications and refinements should also be regarded as the protection scope of the present application.
Claims (10)
1. A linearity optimization method of an iToF camera is characterized by comprising the following steps:
adjusting, by an image sensor of the iToF camera, a duty cycle of an emitted light waveform;
adjusting a delay value at a preset step length under the current duty ratio, and acquiring a virtual real phase and a target measurement phase corresponding to a corresponding virtual real distance after adjusting the delay value each time;
after the delay value of the preset period is adjusted, acquiring linear parameters of the current duty ratio according to all the acquired virtual real phases and the corresponding target measurement phases;
when the current duty ratio is smaller than a preset threshold value, acquiring the duty ratio corresponding to the optimal value in all the linear parameters as the optimal duty ratio; and
and taking the optimal duty ratio as a setting parameter of the image sensor to finish the nonlinear calibration of the iToF camera and realize the linearity optimization of the iToF camera.
2. The iToF camera linearity optimization method of claim 1, wherein the preset step size is any one of pi/4, pi/8, pi/16; the preset period is 2 pi.
3. The method of claim 1, wherein the step of obtaining the pseudo-real phase and the target measured phase corresponding to the pseudo-real distance after adjusting the delay value each time further comprises:
adding a delay unit in a modulation or demodulation path, and simulating a real distance by setting a delay value;
forming a current virtual real distance after adjusting the delay value every time, wherein the current virtual real distance corresponds to a virtual real phase; and
acquiring a preset frame depth image, and performing time domain average on the measurement phases of the pixel center points of all the depth images to acquire a corresponding average measurement phase to serve as a target measurement phase corresponding to the current virtual real distance.
4. The iToF camera linearity optimization method of claim 1, wherein the step of obtaining the linear parameters of the current duty cycle according to all the obtained virtual real phases and the corresponding target measurement phases further comprises:
obtaining the absolute value of the difference between the virtual real phase and the target measurement phase obtained after the delay value is adjusted each time; and
and solving the sum of all the absolute values of the difference values in the preset period, and taking the sum as a linear parameter of the current duty ratio.
5. The method of claim 4, wherein the step of obtaining the duty cycle corresponding to the optimal value of all the linear parameters as the optimal duty cycle further comprises:
and acquiring the minimum value of the sum values corresponding to all duty ratios as the optimal value.
6. The method for linearity optimization of an iToF camera according to claim 1, further comprising:
selecting a plurality of image sensors and acquiring the optimal duty ratio of each image sensor;
obtaining a target duty ratio according to all the optimal duty ratios, wherein the target duty ratio is a median or an average value of all the optimal duty ratios; and
backfilling the target duty cycle into settings of all of the image sensors.
7. The method of optimizing linearity of an iToF camera of claim 6,
the step of selecting a plurality of image sensors further comprises: randomly selecting a plurality of image sensors from the image sensors of the same batch;
the step of backfilling the target duty cycle into the settings of all of the image sensors further comprises: backfilling the target duty cycle into settings of all image sensors of the batch.
8. An apparatus for linearity optimization of an iToF camera, comprising:
the adjusting module is used for adjusting the duty ratio of the emitted light waveform through the image sensor;
the first acquisition module is used for adjusting a delay value at a preset step length under the current duty ratio and acquiring a virtual real phase and a target measurement phase corresponding to a corresponding virtual real distance after the delay value is adjusted each time;
the second acquisition module is used for acquiring linear parameters of the current duty ratio according to all the acquired virtual real phases and the corresponding target measurement phases after the delay value of the preset period is adjusted;
the third acquisition module is used for acquiring the duty ratio corresponding to the optimal value in all the linear parameters as the optimal duty ratio when the current duty ratio is smaller than the preset threshold value; and
and the optimization module is used for finishing the nonlinear calibration of the iToF camera and realizing the linearity optimization of the iToF camera by taking the optimal duty ratio as a setting parameter of the image sensor.
9. The apparatus of claim 8,
the second obtaining module is further configured to obtain an absolute value of a difference between the virtual real phase and the target measured phase obtained after the delay value is adjusted each time, obtain a sum of all the absolute values of the difference in the preset period, and use the sum as a linear parameter of the current duty cycle;
the third obtaining module is further configured to obtain a minimum value of the sum values corresponding to all duty ratios as the optimal value.
10. An electronic device comprising a memory, a processor, and a computer-executable program stored on the memory and executable on the processor, wherein the processor, when executing the computer-executable program, implements the steps of the method for linearity optimization of an iToF camera according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211291852.1A CN115601446A (en) | 2022-10-20 | 2022-10-20 | Method and device for optimizing linearity of iToF camera and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211291852.1A CN115601446A (en) | 2022-10-20 | 2022-10-20 | Method and device for optimizing linearity of iToF camera and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115601446A true CN115601446A (en) | 2023-01-13 |
Family
ID=84848678
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211291852.1A Pending CN115601446A (en) | 2022-10-20 | 2022-10-20 | Method and device for optimizing linearity of iToF camera and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115601446A (en) |
-
2022
- 2022-10-20 CN CN202211291852.1A patent/CN115601446A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10416296B2 (en) | 3DI sensor depth calibration concept using difference frequency approach | |
| CN106911888B (en) | A device | |
| EP3308193B1 (en) | Time-of-flight (tof) system calibration | |
| Hussmann et al. | Real-time motion artifact suppression in tof camera systems | |
| US20180227475A1 (en) | Imaging device, and solid-state imaging element used for same | |
| US20180149750A1 (en) | Distance-measuring imaging device and solid-state imaging device | |
| US9453743B2 (en) | Distance measuring apparatus | |
| US20190178995A1 (en) | Ranging device and method thereof | |
| JPWO2014097539A1 (en) | Three-dimensional measuring apparatus and three-dimensional measuring method | |
| US10684121B2 (en) | Distance-measuring apparatus which uses different calculations depending on whether the object is dynamically changing | |
| EP2717069B1 (en) | Method for determining and/or compensating range offset of a range sensor | |
| US20150310622A1 (en) | Depth Image Generation Utilizing Pseudoframes Each Comprising Multiple Phase Images | |
| US20210263137A1 (en) | Phase noise and methods of correction in multi-frequency mode lidar | |
| JP7094937B2 (en) | Built-in calibration of time-of-flight depth imaging system | |
| WO2017138032A1 (en) | Time-of-flight distance measuring device and method for detecting multipath error | |
| KR20210033545A (en) | Method and system for increasing the range of a time-of-flight system by clear range toggling | |
| CN113227828B (en) | Nonlinear distance error correction method for three-dimensional distance measurement camera using pulse phase shift | |
| CN111896971B (en) | TOF sensing device and distance detection method thereof | |
| CN110673152A (en) | Time-of-flight sensor and distance measuring method thereof | |
| US11624834B2 (en) | Time of flight sensing system and image sensor used therein | |
| CN115601446A (en) | Method and device for optimizing linearity of iToF camera and electronic equipment | |
| JP2020008483A (en) | Imaging device and method for controlling imaging device | |
| US20220187429A1 (en) | Optical ranging device | |
| Hussmann et al. | Investigation of different polar to Cartesian coordinate system conversion methods for ToF-cameras in the close-up range | |
| CN115601443A (en) | Calibration method and device for ToF depth camera and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20231114 Address after: Room 308 and Room 309, No. 268 Xiangke Road, China (Shanghai) Pilot Free Trade Zone, Pudong New Area, Shanghai, 201210 Applicant after: Shanghai Jieming Technology Co.,Ltd. Address before: 351100 705-706, building a, Jiulong community, No. 999, lihuadong Avenue, Xialin street, Chengxiang District, Putian City, Fujian Province Applicant before: Fujian Jiemu Technology Co.,Ltd. |