CN115272137B - Real-time fixed pattern noise removing method, device, medium and system based on FPGA - Google Patents
Real-time fixed pattern noise removing method, device, medium and system based on FPGA Download PDFInfo
- Publication number
- CN115272137B CN115272137B CN202211186990.3A CN202211186990A CN115272137B CN 115272137 B CN115272137 B CN 115272137B CN 202211186990 A CN202211186990 A CN 202211186990A CN 115272137 B CN115272137 B CN 115272137B
- Authority
- CN
- China
- Prior art keywords
- image
- fourier transform
- fixed pattern
- spectrum image
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 238000001228 spectrum Methods 0.000 claims abstract description 63
- 238000004590 computer program Methods 0.000 claims description 11
- 230000003595 spectral effect Effects 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 4
- 239000000203 mixture Substances 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 4
- 230000000694 effects Effects 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 9
- 238000012545 processing Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000009826 distribution Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000005286 illumination Methods 0.000 description 3
- 229910044991 metal oxide Inorganic materials 0.000 description 3
- 150000004706 metal oxides Chemical class 0.000 description 3
- 239000004065 semiconductor Substances 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- ZOXJGFHDIHLPTG-UHFFFAOYSA-N Boron Chemical compound [B] ZOXJGFHDIHLPTG-UHFFFAOYSA-N 0.000 description 1
- 229910052796 boron Inorganic materials 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000000593 degrading effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/10—Image enhancement or restoration using non-spatial domain filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a real-time fixed mode noise removing method, a device, a medium and a system based on an FPGA. The method comprises the following steps: step 1, acquiring a video image F (x, y) from a sensor, and performing two-dimensional discrete Fourier transform on the acquired video image to obtain a frequency spectrum image F (u, v); step 2, searching special frequency points in the frequency spectrum image F (u, v); step 3, carrying out point value replacement on the special frequency points to generate a new Fourier spectrum image F 1 (u, v); step 4, newly generated Fourier spectrum image F 1 (u, v) performing two-dimensional inverse discrete Fourier transform to obtain an image f with fixed pattern noise removed 1 (x, y). The invention eliminates the stripe fixed pattern noise generated by the difference between the columns output by the EB-CMOS image sensor, adopts the FPGA with the Fourier IP core to complete the operation, can complete the operation in real time and has beneficial technical effects.
Description
Technical Field
The invention relates to the technical field of real-time image processing, in particular to a real-time fixed mode noise removing method, device, medium and system based on an FPGA.
Background
An EB-CMOS sensor (Electron boron compensated Metal Oxide Semiconductor) is an image intensifier in which a CMOS imaging device is added at the focal plane position of an Electron lens, so that electrons emitted from a cathode are directly incident on the CMOS device from the back surface to be imaged. The device has excellent micro-optical performance, can perform imaging under the extremely low illumination of 10-6lx, and the EB-CMOS sensor has high gain, low noise and high resolution, can work under the very low illumination state, even can record single photon, and has the quantum efficiency as high as 90%. The structure is shown in fig. 1.
The EB-CMOS sensor introduces a large amount of Fixed Pattern Noise (FPN) due to the operational nature of Column Buffer (Column Buffer) in the CMOS sensor, which is shown in fig. 2 as Fixed vertical stripe (or horizontal stripe). This is because the CMOS image sensor is a column-based CMOS image sensor in which pixels are uniformly exposed to light, but the output of the pixels is not uniform for different columns due to variations in the processing properties of the output system of the columns. Therefore, the column-based CMOS image sensor exhibits vertical stripe fixed pattern noise, thereby degrading the quality of an image. The conventional method for removing fixed pattern noise (such as the prior art CN 101277386A) generally removes vertical streak fixed pattern noise by subtracting a correction signal corresponding to an offset component and a light intensity-related component from an output signal of an image sensor. However, the circuit structure for implementing the above method is complicated. In addition, a DSP or an ARM processor is generally used as a processor at the rear end of the conventional photoelectric sensor, and a new technology for improving the capability of correcting the fixed pattern noise without losing the real-time processing speed is expected because the calculation speed for completing such a large data amount is slow and the processing speed is not real enough.
Disclosure of Invention
To solve the above technical problems, the present invention provides a real-time fixed pattern noise removing method, apparatus, medium, and system, which are particularly useful for real-time removal of streak fixed pattern noise of an EB-CMOS image sensor, and can remove streak fixed pattern noise generated due to a difference between columns output by the EB-CMOS image sensor. In addition, the operation is completed by adopting the FPGA with the Fourier IP core, and the operation can be completed in real time.
According to one aspect of the present invention, an embodiment of the present invention provides a real-time fixed pattern noise removing method based on an FPGA, including the following steps:
a real-time fixed mode noise removing method based on FPGA comprises the following steps:
step 1, acquiring a video image F (x, y) from a sensor, and performing two-dimensional discrete Fourier transform on the acquired video image to obtain a frequency spectrum image F (u, v);
step 2, searching special frequency points in a frequency spectrum image F (u, v) obtained after two-dimensional discrete Fourier transform;
step 3, carrying out point value replacement on the special frequency points in the frequency spectrum image to generate a new Fourier frequency spectrum image F 1 (u,v)；
Step 4, carrying out Fourier spectrum image F newly generated in step 3 1 (u, v) performing two-dimensional inverse discrete Fourier transform to obtain an image f with fixed pattern noise removed 1 (x,y)。
In a possible implementation, in step 1, the formula of the two-dimensional discrete fourier transform is:
wherein, F (u, v) is a frequency spectrum image obtained after two-dimensional discrete Fourier transform, F (x, y) is an acquired video image, M is the width of the video image, and N is the height of the video image.
In a possible implementation manner, in step 2, specifically, searching for a special frequency point in a spectrum image F (u, v) obtained after two-dimensional discrete fourier transform is: searching a special frequency point which is a series of (u, v) values and is marked as { a } in a preset central position mask by adopting a mixed Gaussian model for a frequency spectrum image obtained after two-dimensional discrete Fourier transform 1 ,a 2 ,a 3 ,…}。
In aIn a possible implementation manner, the point value replacement of the special frequency points in the spectrum image in step 3 specifically includes: searching a series of (u, v) values obtained according to the step 2, replacing the values corresponding to the series of (u, v) values with 0 in the frequency spectrum image obtained after the original two-dimensional discrete Fourier transform, and generating a new Fourier frequency spectrum image F 1 (u,v)：
Wherein, F 1 (u, v) is a newly generated Fourier spectrum image, and F (u, v) is a spectrum image obtained by performing two-dimensional discrete Fourier transform.
In a possible embodiment, in step 4, the formula of the two-dimensional inverse discrete fourier transform is:
wherein f is 1 (x, y) is an image from which fixed pattern noise is removed, F 1 (u, v) is the newly generated Fourier spectrum image, M is the width of the video image, and N is the height of the video image.
In a possible implementation manner, the special frequency points in step 2 are a series of (u, v) values with larger amplitude values and smaller time variation, the larger amplitude values indicate that the amplitude of the spectrum image is larger than a set experience threshold value T, and the smaller variation indicates that the spectrum image meets a criterion formula of a mixed gaussian model method; the criterion formula of the Gaussian mixture model method is as follows:
wherein, F t+1 (u, v) denotes the t +1 th frame spectral image F (u, v), μ i,t Representing the i-th Gaussian model expected value, D, updated according to the t-th frame spectrogram image b Taking the empirical value 3, sigma i,t Representing the ith gaussian model variance value updated from the tth frame spectrogram image.
According to another aspect of the present invention, an embodiment of the present invention further provides a real-time fixed pattern noise removing apparatus based on an FPGA, including:
the two-dimensional discrete Fourier transform module is used for acquiring a video image F (x, y) from the sensor and performing two-dimensional discrete Fourier transform to obtain a frequency spectrum image F (u, v);
the special frequency point searching module is used for searching special frequency points in the frequency spectrum image after the two-dimensional Fourier transform to obtain a series of frequency point values;
a point value replacement module for performing point value replacement on the special frequency points in the frequency spectrum image to generate a new Fourier frequency spectrum image F 1 (u,v)；
The two-dimensional inverse discrete Fourier transform module is used for carrying out two-dimensional inverse discrete Fourier transform on the newly generated Fourier spectrum image to obtain an image f without fixed mode noise 1 (x,y)。
According to another aspect of the present invention, an embodiment of the present invention further provides a real-time fixed pattern noise removing apparatus based on an FPGA, including:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to implement the aforementioned real-time fixed pattern noise removal method when executing the instructions.
According to yet another aspect of the present invention, the embodiment of the present invention further provides a non-volatile computer-readable storage medium, on which computer program instructions are stored, which when executed by a processor implement the aforementioned real-time fixed pattern noise removing method.
According to another aspect of the present invention, an embodiment of the present invention further provides a real-time fixed pattern noise removing system based on an FPGA, which includes an EB-CMOS sensor, an FPGA chip, and a DDR memory, where the real-time fixed pattern noise removing method is executed on the FPGA chip, and intermediate data generated in the execution process is stored in the DDR memory.
Compared with the prior art, the invention has the beneficial effects that: the invention provides a method, a device, a medium and a system for removing real-time fixed pattern noise based on an FPGA (field programmable gate array), which are particularly used for an EB-CMOS (electronic beam-complementary metal oxide semiconductor) image sensor, have a simple circuit structure and can be realized on an FPGA chip, meanwhile, according to the characteristic that the algorithm needs Fourier transformation, the operation is completed by adopting the FPGA with a Fourier IP (Internet protocol) core, the operation can be completed in real time by utilizing the strong data processing capacity of the FPGA, the strip-shaped fixed pattern noise generated by the difference between columns output by the EB-CMOS image sensor is eliminated in real time, and the method, the device, the medium and the system have beneficial technical effects.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It should be apparent that the drawings in the following description are merely exemplary, and that other embodiments can be derived from the drawings provided by those of ordinary skill in the art without inventive effort.
The structures, ratios, sizes, and the like shown in the present specification are only used for matching with the contents disclosed in the specification, so that those skilled in the art can understand and read the present invention, and do not limit the conditions for implementing the present invention, so that the present invention has no technical significance, and any structural modifications, changes in the ratio relationship, or adjustments of the sizes, without affecting the functions and purposes of the present invention, shall fall within the scope covered by the technical contents disclosed in the present invention.
FIG. 1 is a diagram of an EB-CMOS sensor structure;
FIG. 2 is an EB-CMOS sensor fixed pattern noise image;
FIG. 3 is a schematic diagram showing the spectrum of the vertical variation after Fourier variation in the horizontal axis direction;
FIG. 4 is a graph of a Fourier spectrum of an EB-CMOS image centered after discrete Fourier transform;
FIG. 5 is a schematic diagram of a mask for searching Fourier spectrogram feature points after Fourier transform;
FIG. 6 is a schematic diagram of a mask generated after searching Fourier spectrogram feature points after Fourier transform;
FIG. 7 is an image of an EB-CMOS sensor after fixed pattern noise is removed;
FIG. 8 is a flow chart of a FPGA-based EB-CMOS real-time fixed pattern noise removal method;
FIG. 9 is a schematic diagram of a real-time fixed pattern noise removing apparatus for an EB-CMOS sensor;
fig. 10 is a schematic structural diagram of another EB-CMOS sensor real-time fixed pattern noise removing apparatus.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the present invention is described in further detail below with reference to the embodiments and the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
In the description of the present invention, it is to be understood that the terms "comprises/comprising," consists of/8230; \8230 ";" or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a product, device, process or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such product, device, process or method if desired. Without further limitation, an element defined by the phrases "comprising/including" \8230; "comprising 8230;" \8230; and \8230; "comprises;" does not exclude the presence of additional like elements in a product, device, process, or method comprising the element.
The invention provides a real-time fixed pattern noise removing method and a system based on an FPGA (field programmable gate array), which are particularly used for an EB-CMOS (Electron Beam-complementary Metal oxide semiconductor) sensor.
EB-CMOS sensors introduce a lot of vertical stripe shaped fixed pattern noise, V, during operation due to column buffer differences in CMOS sensors out (x, y) is the (x, y) point signal output in the image, gain i For the gain of the column, V in (x,y) is the input of the spot light signal, V offset Is the bias value at this point due to non-uniformity of transistor performance in the light sensitive cell.
The non-uniform Gain in the column buffer results in vertical stripe fixed pattern noise, which is Gain due to low input optical energy under the extremely low illumination of EB-CMOS i The gain is more obvious when the gain is large, and the noise does not change along with the scene. The longitudinal periodic variation in the spatial domain due to the two-dimensional discrete fourier transform of the image is reflected on the horizontal axis in the spectrogram as shown in fig. 3.
The fourier spectrum shown in fig. 4 can be obtained by performing two-dimensional fourier transform on the video image of the EB-CMOS sensor shown in fig. 2. The gray scale value along the horizontal axis direction is larger, so that a large number of periodically-changed patterns along the vertical direction in the original image are reflected, and the gray scale value is consistent with the original image.
FIG. 8 shows a flow chart of an FPGA-based EB-CMOS real-time fixed pattern noise removal method. As shown in fig. 8, the EB-CMOS real-time fixed pattern noise removing method based on FPGA specifically includes the following steps:
1. a two-dimensional discrete fourier transform is performed on the video image acquired from the sensor.
Specifically, a two-dimensional Discrete Fourier Transform (DFT) may be performed on the video image of the EB-CMOS sensor on the FPGA; obtaining a frequency spectrum image F (u, v); the formula for the two-dimensional Discrete Fourier Transform (DFT) is shown below:
wherein, F (u, v) is a frequency spectrum image obtained after two-dimensional discrete Fourier transform, F (x, y) is an acquired video image, M is the width of the video image, and N is the height of the video image.
2. And searching special frequency points in the frequency spectrum image after the two-dimensional Fourier transform to obtain a series of frequency point values.
Specifically, referring to fig. 5 and 6, for a spectrum image after two-dimensional fourier transform on the FPGA, a series of (u, v) values with large amplitude and small time variation are searched in a preset center position mask and are marked as { a 1 ,a 2 ,a 3 ,…}；
The larger amplitude value means that the amplitude of the spectrum image is larger than a set empirical threshold value T, and the smaller change means that the spectrum image meets a criterion formula of a GMM method.
Each pixel point in the frequency spectrum image F (u, v) is described by adopting 3 Gaussian distributions by adopting a GMM (Gaussian mixture model) method, and the expectation and the variance of the 3 Gaussian distributions corresponding to each pixel point are obtained by inputting the first K frames of the t-th frame frequency spectrum image in advance to perform pixel-by-pixel statistical modeling. In a new frame of spectral image F t+1 And (u, v) after input, the frequency point changes less when a criterion formula is established. The criterion formula of the GMM method is as follows:
wherein, F t+1 (u, v) denotes the t +1 th frame image F (u, v), μ i,t Representing the i-th Gaussian model expectation value, D, updated from the t-th frame image b Taking the empirical value 3, sigma i,t Representing the ith gaussian model variance value updated from the t frame image.
The updating strategy of the Gaussian model is as follows: when the input pixel value F t+1 (u, v) when the image belongs to a certain Gaussian distribution corresponding to the pixel, updating the expectation and the variance of the Gaussian distribution model according to the previous K frames of the t +1 th frame of the spectrum image.
3. Carrying out point value replacement on special frequency points in the frequency spectrum image to generate a new Fourier frequency spectrum graph F 1 (u,v)。
Specifically, a series of (u, v) values obtained by searching in the step 2 are searched on the FPGA, and the corresponding values in (u, v) are replaced by 0 in the original two-dimensional Fourier transformed spectrum image; the specific alternative is shown in the following formula:
wherein, F 1 (u, v) is a newly generated Fourier spectrum image, and F (u, v) is a spectrum image obtained by performing two-dimensional discrete Fourier transform.
4. Performing two-dimensional inverse discrete Fourier transform on the Fourier spectrogram newly generated in the step 3 to obtain an image f without fixed pattern noise 1 (x,y)。
Specifically, the two-dimensional inverse discrete fourier transform is performed on the fourier spectrogram newly generated in step 3 on the FPGA, so as to obtain an image with fixed pattern noise removed, as shown in fig. 7. The formula for the two-dimensional inverse discrete fourier transform is shown below:
wherein, f 1 (x, y) is an image from which fixed pattern noise is removed, F 1 (u, v) is the newly generated Fourier spectrum image, M is the width of the video image, and N is the height of the video image.
The intermediate data generated in each of the above method steps is stored in the DDR memory.
As is apparent from comparing fig. 2 and 7, after the vertical stripe noise on the original image shown in fig. 2 is processed by the method for removing the fixed pattern noise in real time according to the present invention, the vertical stripe noise obviously disappears in the image obtained after the processing shown in fig. 7.
The second embodiment of the invention also provides an EB-CMOS sensor real-time fixed pattern noise removing apparatus based on FPGA, which comprises a two-dimensional discrete fourier transform module, a special frequency point searching module, a point value replacing module and a dimensional inverse discrete fourier transform module, as shown in fig. 9, wherein,
a two-dimensional discrete Fourier transform module for acquiring video images from the sensor and performing two-dimensional discrete Fourier transform,
the special frequency point searching module is used for searching special frequency points in the frequency spectrum image after the two-dimensional Fourier transform to obtain a series of frequency point values;
a point value replacement module for performing point value replacement on the special frequency points in the frequency spectrum image to generate a new Fourier spectrogram F 1 (u,v)；
The two-dimensional inverse discrete Fourier transform module is used for carrying out two-dimensional inverse discrete Fourier transform on the newly generated Fourier spectrogram, and the image f with the fixed mode noise removed can be obtained 1 (x,y)。
Although the present invention is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and/or features may be integrated in a single physical device and/or software module, or one or more functions and/or features may be implemented in a separate physical device or software module. It will also be understood that a detailed discussion of the actual implementation of each module is not necessary for an understanding of the present invention. Rather, the actual implementation of the various functional modules in the apparatus disclosed herein will be understood within the ordinary skill of an engineer, given the nature, function, and internal relationship of the modules.
The embodiment III of the invention provides an EB-CMOS sensor real-time fixed mode noise removing system based on an FPGA, which comprises an EB-CMOS sensor, an FPGA chip and a DDR memory, wherein the real-time fixed mode noise removing method is executed on the FPGA chip.
An embodiment of the present invention provides an EB-CMOS sensor real-time fixed pattern noise removing apparatus based on an FPGA, as shown in fig. 10, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the real-time fixed pattern noise removal method when executing the instructions.
The processor may be a Central Processing Unit (CPU), other general purpose processor, a digital signal processor (digital signal processor), an Application Specific Integrated Circuit (Application Specific Integrated Circuit), a field programmable gate array (field programmable gate array) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory may be used to store the computer programs and/or modules, and the processor may implement various functions of the model building apparatus in the invention by operating or executing data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playing function, an image playing function, etc.), and the like. Further, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, at least one magnetic disk storage device, a flash memory device, or other volatile solid state storage device.
The contents in the method embodiments are all applicable to the device embodiments, the functions specifically implemented by the device embodiments are the same as those in the method embodiments, and the beneficial effects achieved by the device embodiments are also the same as those achieved by the method embodiments.
An embodiment five of the present invention provides a non-transitory computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the above real-time fixed pattern noise removal method.
Implementation of the method, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow in the method of implementing the embodiments of the present invention may also be stored in a computer readable storage medium through a computer program, and when the computer program is executed by a processor, the computer program may implement the steps of the above-described method embodiments. Wherein the computer program comprises computer program code, an object code form, an executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying said computer program code, a recording medium, a usb-disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, a point carrier signal, a telecommunications signal, a software distribution medium, etc. It should be noted that the computer readable medium may contain content that is appropriately increased or decreased as required by legislation and patent practice in the jurisdiction.
While embodiments of the present invention have been described above, the above description is illustrative, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. Various changes and modifications of the present invention can be made by those skilled in the art without departing from the technical idea of the present invention, and these changes and modifications still fall within the scope of the present invention.
Claims (8)
1. A real-time fixed mode noise removing method based on FPGA is characterized by comprising the following steps:
step 1, acquiring a video image F (x, y) from a sensor, and performing two-dimensional discrete Fourier transform on the acquired video image to obtain a frequency spectrum image F (u, v);
step 2, searching special frequency points in a frequency spectrum image F (u, v) obtained after two-dimensional discrete Fourier transform, specifically comprising the following steps: searching special frequency points, which are a series of (u, v) values and are marked as { a }, of a frequency spectrum image F (u, v) obtained after two-dimensional discrete Fourier transform by adopting a mixed Gaussian model in a preset central position mask 1 ,a 2 ,a 3 8230, wherein the special frequency points are a series of (u, v) values with larger amplitude values and smaller change along with time, the larger amplitude values mean that the amplitude of the frequency spectrum image F (u, v) is larger than a set experience threshold value T, and the smaller change means that the frequency spectrum image F (u, v) meets a criterion formula of a mixed Gaussian model method;
step 3, carrying out point counting on special frequency points in the frequency spectrum image F (u, v)Value replacement to generate a new Fourier spectral image F 1 (u, v), specifically: searching a series of (u, v) values obtained according to the step 2, replacing the values corresponding to the series of (u, v) values with 0 in the frequency spectrum image obtained after the original two-dimensional discrete Fourier transform, and generating a new Fourier frequency spectrum image F 1 (u,v)：
Wherein, F 1 (u, v) is a newly generated Fourier spectrum image, and F (u, v) is a spectrum image obtained after two-dimensional discrete Fourier transform;
step 4, carrying out Fourier spectrum image F newly generated in step 3 1 (u, v) performing two-dimensional inverse discrete Fourier transform to obtain an image f with fixed pattern noise removed 1 (x,y)。
2. The real-time fixed pattern noise removal method of claim 1, wherein:
in step 1, the formula of the two-dimensional discrete fourier transform is:
wherein, F (u, v) is a frequency spectrum image obtained after two-dimensional discrete Fourier transform, F (x, y) is an acquired video image, M is the width of the video image, and N is the height of the video image.
3. The real-time fixed pattern noise removal method of claim 1, wherein:
in step 4, the formula of the two-dimensional inverse discrete fourier transform is as follows:
wherein f is 1 (x, y) is an image from which fixed pattern noise is removed, F 1 (u, v) is the newly generated Fourier spectrum image, M is the width of the video image, and N is the height of the video image.
4. The real-time fixed pattern noise removal method of claim 1, wherein:
the criterion formula of the Gaussian mixture model method in the step 2 is as follows:
wherein, F t+1 (u, v) denotes the t +1 th frame spectral image F (u, v), μ i,t Indicating the i-th Gaussian model expected value, D, updated according to the t-th frame spectral image F (u, v) b Taking the empirical value 3, sigma i,t Represents the ith gaussian model variance value updated from the t-th frame spectral image F (u, v).
5. A real-time fixed pattern noise removing apparatus used for the real-time fixed pattern noise removing method according to any one of claims 1 to 4, characterized by comprising:
the two-dimensional discrete Fourier transform module is used for acquiring a video image F (x, y) from the sensor and performing two-dimensional discrete Fourier transform to obtain a frequency spectrum image F (u, v);
the special frequency point searching module is used for searching special frequency points in the frequency spectrum image after the two-dimensional Fourier transform to obtain a series of frequency point values;
a point value replacement module for performing point value replacement on the special frequency points in the frequency spectrum image to generate a new Fourier frequency spectrum image F 1 (u,v)；
The two-dimensional inverse discrete Fourier transform module is used for carrying out two-dimensional inverse discrete Fourier transform on the newly generated Fourier spectrum image to obtain an image f without fixed mode noise 1 (x,y)。
6. The utility model provides a real-time fixed pattern noise remove device based on FPGA which characterized in that includes:
a processor;
a memory for storing processor-executable instructions;
wherein the processor is configured to implement the real-time fixed pattern noise removal method of any of claims 1-4 when executing the instructions.
7. A non-transitory computer readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the real-time fixed pattern noise removal method of any of claims 1-4.
8. An FPGA-based real-time fixed pattern noise removal system comprising an EB-CMOS sensor, an FPGA chip and a DDR memory, characterized in that the real-time fixed pattern noise removal method according to any one of claims 1 to 4 is performed on the FPGA chip, and intermediate data generated during the execution is stored in the DDR memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211186990.3A CN115272137B (en) | 2022-09-28 | 2022-09-28 | Real-time fixed pattern noise removing method, device, medium and system based on FPGA |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211186990.3A CN115272137B (en) | 2022-09-28 | 2022-09-28 | Real-time fixed pattern noise removing method, device, medium and system based on FPGA |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115272137A CN115272137A (en) | 2022-11-01 |
| CN115272137B true CN115272137B (en) | 2022-12-20 |
Family
ID=83756435
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211186990.3A Active CN115272137B (en) | 2022-09-28 | 2022-09-28 | Real-time fixed pattern noise removing method, device, medium and system based on FPGA |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115272137B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104580937A (en) * | 2015-01-21 | 2015-04-29 | 中国科学院上海技术物理研究所 | Stripe noise removal method for infrared imaging system |
| CN106462957A (en) * | 2016-05-19 | 2017-02-22 | 深圳大学 | Method and system for removing stripe noise in infrared image |
| CN107481205A (en) * | 2017-08-23 | 2017-12-15 | 电子科技大学 | A kind of Terahertz image fringes noise processing method and system |
| CN107843767A (en) * | 2017-10-19 | 2018-03-27 | 广州致远电子有限公司 | Signal processing method and system for spectrum analysis |
| CN109782244A (en) * | 2018-12-29 | 2019-05-21 | 西安交通大学 | Based on single goal triangular linear frequency modulated continuous wave radar signal processing method |
| CN111311510A (en) * | 2020-01-22 | 2020-06-19 | 京东方科技集团股份有限公司 | Method and device for removing stripe noise in image |
| KR102191736B1 (en) * | 2020-07-28 | 2020-12-16 | 주식회사 수퍼톤 | Method and apparatus for speech enhancement with artificial neural network |
| CN114283272A (en) * | 2021-12-15 | 2022-04-05 | 上海电力大学 | Molten pool image acquisition method based on spatial filtering technology |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA3115423A1 (en) * | 2020-05-01 | 2021-11-01 | Systemes De Controle Actif Soft Db Inc. | A system and a method for sound recognition |
-
2022
- 2022-09-28 CN CN202211186990.3A patent/CN115272137B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104580937A (en) * | 2015-01-21 | 2015-04-29 | 中国科学院上海技术物理研究所 | Stripe noise removal method for infrared imaging system |
| CN106462957A (en) * | 2016-05-19 | 2017-02-22 | 深圳大学 | Method and system for removing stripe noise in infrared image |
| CN107481205A (en) * | 2017-08-23 | 2017-12-15 | 电子科技大学 | A kind of Terahertz image fringes noise processing method and system |
| CN107843767A (en) * | 2017-10-19 | 2018-03-27 | 广州致远电子有限公司 | Signal processing method and system for spectrum analysis |
| CN109782244A (en) * | 2018-12-29 | 2019-05-21 | 西安交通大学 | Based on single goal triangular linear frequency modulated continuous wave radar signal processing method |
| CN111311510A (en) * | 2020-01-22 | 2020-06-19 | 京东方科技集团股份有限公司 | Method and device for removing stripe noise in image |
| KR102191736B1 (en) * | 2020-07-28 | 2020-12-16 | 주식회사 수퍼톤 | Method and apparatus for speech enhancement with artificial neural network |
| CN114283272A (en) * | 2021-12-15 | 2022-04-05 | 上海电力大学 | Molten pool image acquisition method based on spatial filtering technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115272137A (en) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022170706A1 (en) | Defect detection method and apparatus for mold monitoring, and device and medium | |
| CN107071234B (en) | Lens shadow correction method and device | |
| CN104580937B (en) | A kind of infrared imaging system fringes noise minimizing technology | |
| Feng et al. | Learnability enhancement for low-light raw denoising: Where paired real data meets noise modeling | |
| CN111542916A (en) | Compensating for metrology errors caused by distortion of a scanning electron microscope beam using a design | |
| CN106296608B (en) | Mapping table-based fisheye image processing method and system | |
| JPH01102841A (en) | Image forming method | |
| JP2021086616A (en) | Method for extracting effective region of fisheye image based on random sampling consistency | |
| US11074674B2 (en) | Imaging noise reduction system and method | |
| CN118333914B (en) | High dynamic range light field data white image correction method, system and medium | |
| CN115272137B (en) | Real-time fixed pattern noise removing method, device, medium and system based on FPGA | |
| CN106651787A (en) | HDR image synthetic method and system based on X rays | |
| CN115393441B (en) | Light spot high-precision extraction and positioning method based on liquid crystal light closed-loop system | |
| KR100835058B1 (en) | Image processing method for extending depth of field | |
| EP3510764B1 (en) | System and method for high dynamic range digital double sampling | |
| JP2013106131A (en) | Image processing device | |
| CN111178371B (en) | Target detection method, device and computer storage medium | |
| CN115330637A (en) | Image sharpening method and device, computing device and storage medium | |
| CN115578340A (en) | Method and system for detecting breakage of glass backboard of OLED display screen | |
| JP2015163915A (en) | Image processor, imaging device, image processing method, program, and storage medium | |
| JP2023058758A (en) | Data generating method, learning method, estimating method, data generating device, and program | |
| CN115880188B (en) | Polarization direction statistical image generation method, device and medium | |
| Fabijanska et al. | Image noise removal-the new approach | |
| CN117676355A (en) | Method and circuit for removing solar black particles of image sensor and image sensor | |
| US20240320805A1 (en) | Image denoising method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |