CN113491497B - Polarized light endoscope device - Google Patents
Polarized light endoscope device Download PDFInfo
- Publication number
- CN113491497B CN113491497B CN202110849690.8A CN202110849690A CN113491497B CN 113491497 B CN113491497 B CN 113491497B CN 202110849690 A CN202110849690 A CN 202110849690A CN 113491497 B CN113491497 B CN 113491497B
- Authority
- CN
- China
- Prior art keywords
- image
- polarized light
- data
- rgb
- image data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00043—Operational features of endoscopes provided with output arrangements
- A61B1/00045—Display arrangement
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00064—Constructional details of the endoscope body
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00131—Accessories for endoscopes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00163—Optical arrangements
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
- A61B1/05—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances characterised by the image sensor, e.g. camera, being in the distal end portion
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Surgery (AREA)
- Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Pathology (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Biophysics (AREA)
- Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Signal Processing (AREA)
- Endoscopes (AREA)
Abstract
A polarized light endoscopic device comprising: the optical prism divides one path of light beam output by the optical lens into two paths of light beams, wherein one path of light beam is output to the RGB image sensor, and the other path of light beam is output to the polarized light image sensor; the RGB image sensor collects RGB optical signals and converts the RGB optical signals into RGB pixel data; the polarized light image sensor collects polarized light signals and converts the polarized light signals into polarized information pixel data; the image processor synchronously performs image processing on the RGB pixel data and performs sharpening processing on the polarization information pixel data, and outputs the obtained RGB image and polarization information image for display, so that when blood, tissue scraps, fog and other conditions occur in the operation environment, the problem that the image output by the endoscope device is not sharp is solved through sharpening processing on the polarization information pixel data.
Description
Technical Field
The invention relates to the technical field of endoscopes, in particular to a polarized light endoscope device.
Background
The endoscope is a medical electronic optical instrument which can be inserted into human body cavity and internal organ cavity to make direct observation, diagnosis and treatment, and it adopts optical lens with very small size to make optical imaging of intracavity object to be observed by means of miniature objective lens imaging system, then the optical imaging is sent to image processing main machine, and finally the observed image after image processing is outputted on the display screen for doctor to observe and diagnose.
At present, the endoscopes mainly comprise ultra-high-definition 4K endoscopes, three-wafer endoscopes, fluorescence endoscopes, narrow-band light endoscopes, 3D endoscopes and the like, and the endoscopes have good functions and performances in respective fields. The 4K endoscope has ultrahigh definition resolution and good detail restoration; the three-wafer endoscope is provided with three image sensors for respectively collecting a monochromatic light signal in RGB, the RGB color value of each pixel can be obtained, the color reduction degree precision is high, and the performance is better than that of a common single-wafer endoscope; the fluorescence endoscope is a new imaging technology by utilizing fluorescence molecule imaging in a special spectrum environment, and the principle is that contrast of a pathological change tissue and a normal tissue is increased by marking tumor cells by a fluorescent agent, so that cancers and tumors can be generated earlier; the narrow-band light endoscope filters out a broadband spectrum in red, blue and green light waves emitted by an endoscope light source by using an optical filter, and only the narrow-band spectrum is left for accurately observing the epithelial form of the mucous membrane of the alimentary tract, such as an epithelial fovea structure, and the form of an epithelial vascular network can be observed; the camera scene that 3D endoscope shows has depth information, and the doctor can be more accurate carry out operation and treatment to the focus in the operation.
Although the functions and performances of the endoscope are good, clear imaging cannot be output in an operation environment with bloody water, turbid water, small tissue scraps and mist, the doctor is interfered by the unclear imaging during the operation, and sometimes the operation can be performed after the interference environment (the bloody water, the turbid water, the tissue scraps and the mist) is cleaned, so that the operation efficiency is low.
Disclosure of Invention
The endoscope device mainly solves the technical problem that the endoscope device can output clearer images, and the function of clear imaging in underwater environments (blood, turbidity, tissue scraps and fog) is realized.
According to a first aspect, an embodiment provides a polarized light endoscope apparatus, comprising a camera and an image processing system, wherein the apparatus further comprises:
the optical prism is arranged in the camera and used for dividing one path of light beam output by the optical lens into two paths of light beams;
the RGB image sensor is arranged in the camera and used for collecting RGB optical signals of one path of light beam output by the optical prism and converting the RGB optical signals into RGB pixel data;
the polarized light image sensor is arranged in the camera and used for collecting a polarized light signal of the other path of light beam output by the optical prism and converting the polarized light signal into polarized information pixel data;
and the image processor is arranged in the image processing system and connected with the camera, and is used for synchronously carrying out image processing on the RGB pixel data and carrying out sharpening processing on the polarization information pixel data, acquiring an RGB image and a polarization information image, and outputting the RGB image and the polarization information image for display.
In an embodiment, the sharpening the polarization information pixel data includes:
performing layering processing on the current frame image data in the polarization information pixel data to obtain base layer image data and detail layer image data corresponding to the current frame image data;
denoising the base layer image data and the detail layer image data respectively;
and reconstructing the denoised image data of the base layer and the image data of the detail layer to obtain a polarization information image corresponding to the image data of the current frame.
In an embodiment, the performing layered processing on the current frame image data in the polarization information pixel data to obtain base layer image data and detail layer image data corresponding to the current frame image data includes:
extracting image data of continuous frames from the polarization information pixel data, wherein the image data of the continuous frames comprises current frame image data;
calculating a Stokes vector by using a preset polarization angle of the image data with continuous frames; the preset bias angle is a polarization angle of the polarized light image sensor;
determining a polarization angle corresponding to the maximum value of the polarized light intensity and a polarization angle corresponding to the minimum value of the polarized light intensity in the image data according to the Stokes vector;
and separating the current frame image data into base layer image data and detail layer image data according to the polarization angle corresponding to the maximum value of the polarized light intensity and the polarization angle corresponding to the minimum value of the polarized light intensity.
In an embodiment, the performing denoising processing on the base layer image data and the detail layer image data respectively includes:
performing wavelet transformation on the detail layer image data to obtain a plurality of frequency domain data, wherein each frequency domain data corresponds to a different wavelet transformation coefficient;
carrying out bilateral filtering processing on the plurality of frequency domain data so as to carry out denoising on the plurality of frequency domain data;
reconstructing the denoised frequency domain data to obtain denoised detail layer image data;
carrying out bilateral filtering on the image data of the basic layer in a space domain and a transform domain so as to carry out denoising on the image data of the basic layer;
and carrying out iterative optimization processing on the base layer image data after bilateral filtering processing to obtain the denoised base layer image data.
In one embodiment, the polarized light image sensor is a polarized light image sensor having 4 polarization angles, which include 0 °, 45 °, 90 °, and 135 °.
In one embodiment, the method further comprises:
the image sending module is arranged in the camera, is respectively connected with the polarized light image sensor and the RGB image sensor, and is used for sending the RGB pixel data and the polarization information pixel data;
and the image receiving module is arranged in the image processing system and used for receiving the RGB pixel data and the polarization information pixel data and forwarding the RGB pixel data and the polarization information pixel data to an image processor.
In one embodiment, the method further comprises:
a first display for displaying the RGB image;
a second display for displaying the polarization information image;
the display interface circuit is arranged in the image processing system and connected with the image processor and used for receiving the RGB image and the polarization information image output by the image processor, respectively encoding and converting the data formats of the RGB image and the polarization information image into a preset data format, outputting the RGB image in the preset data format to the first display and outputting the polarization information image in the preset data format to the second display.
In one embodiment, the method further comprises:
the wireless transmission circuit is arranged in the image processing system and connected with the image processor, and is used for the image processor to perform data interaction with the external mobile terminal and transmit the RGB image and the polarization information image to the external mobile terminal for display and/or storage.
In one embodiment, the method further comprises:
the clock circuit is arranged in the camera, is respectively connected with the RGB image sensor and the polarized light image sensor, and is used for generating a synchronous clock signal and sending the synchronous clock signal to the RGB image sensor and the polarized light image sensor;
the function selection module is arranged in the camera and connected to the image processor, and comprises a key assembly and is used for outputting at least one function selection signal to the image processor under the triggering of the key assembly so as to control the polarized light endoscope device to complete at least one corresponding function.
In one embodiment, the method further comprises:
the exposure synchronizing signal generating circuit is arranged in the image processing system, is respectively connected with the RGB image sensor and the polarized light image sensor, and is used for outputting a synchronizing exposure signal to the RGB image sensor and the polarized light image sensor; the RGB image sensor and the polarized light image sensor are specifically used for respectively and synchronously acquiring the RGB optical signals and the polarized light signals under the triggering of the synchronous exposure signals;
a panel input circuit disposed within the image processing system for user selection and switching of at least one function of the polarized light endoscope apparatus;
the external interface circuit is arranged in the image processing system and is used for providing an interface for external equipment to be accessed into the polarized light endoscope device;
and the power supply circuit is arranged in the image processing system and used for supplying power to each module and circuit of the polarized light endoscope device.
According to the polarized light endoscope device of the embodiment, the optical prism divides one path of light beam output by the optical lens into two paths of light beams, wherein one path of light beam is output to the RGB image sensor, and the other path of light beam is output to the polarized light image sensor; the RGB image sensor collects RGB optical signals output by the optical lens and converts the RGB optical signals into RGB pixel data; the polarized light image sensor collects polarized light signals output by the optical lens and converts the polarized light signals into polarized information pixel data; the image processor synchronously performs image processing on the RGB pixel data and performs sharpening processing on the polarization information pixel data, and outputs the obtained RGB image and polarization information image for display, so that when blood, tissue scraps, fog and other conditions occur in the operation environment, the problem that the image output by the endoscope device is not sharp is solved through sharpening processing on the polarization information pixel data.
Drawings
FIG. 1 is a schematic structural diagram of a polarized light endoscope apparatus according to an embodiment;
FIG. 2 is a schematic diagram showing a detailed structure of the polarized light endoscope apparatus shown in FIG. 1;
FIG. 3 is a flow chart of a data processing method for an endoscopic device according to an embodiment;
fig. 4 is a detailed flowchart of the data processing method for the endoscope apparatus shown in fig. 3.
Detailed Description
The present invention will be described in further detail with reference to the following detailed description and accompanying drawings. Wherein like elements in different embodiments are numbered with like associated elements. In the following description, numerous details are set forth in order to provide a better understanding of the present application. However, those skilled in the art will readily recognize that some of the features may be omitted or replaced with other elements, materials, methods in different instances. In some instances, certain operations related to the present application have not been shown or described in detail in order to avoid obscuring the core of the present application from excessive description, and it is not necessary for those skilled in the art to describe these operations in detail, so that they may be fully understood from the description in the specification and the general knowledge in the art.
Furthermore, the features, operations, or characteristics described in the specification may be combined in any suitable manner to form various embodiments. Also, the various steps or actions in the method descriptions may be transposed or transposed in order, as will be apparent to one of ordinary skill in the art. Thus, the various sequences in the specification and drawings are for the purpose of describing certain embodiments only and are not intended to imply a required sequence unless otherwise indicated where such sequence must be followed.
The numbering of the components as such, e.g., "first", "second", etc., is used herein only to distinguish the objects as described, and does not have any sequential or technical meaning. The term "connected" and "coupled" when used in this application, unless otherwise indicated, includes both direct and indirect connections (couplings).
Referring to fig. 1, fig. 1 is a schematic structural diagram of a polarized light endoscope apparatus according to an embodiment, the endoscope apparatus electrically includes a camera 100 for acquiring pixel data of a target object and an image processing system 200 for processing the pixel data, and the camera 100 and the image processing system 200 perform signal transmission through a transmission cable 300.
Referring to fig. 2, fig. 2 is a schematic structural diagram of the polarized light endoscope apparatus shown in fig. 1. The camera 100 includes an optical lens 101, an optical prism 102, an RGB image sensor 103, a polarized light image sensor 104, an image transmission module 105, and a function selection module 106. The image processing system 200 includes an image receiving module 201, an image processor 202, a display interface circuit 203, a wireless transmission circuit 204, an exposure synchronizing signal generating circuit 205, a panel input circuit 206, a peripheral interface circuit 207, and a power supply circuit 208.
The optical lens 101 is an optical path system composed of a plurality of optical mirrors, and provides an optical path for white light generated by a light source to irradiate to a target object, the white light can form return light after irradiating to the target object, the optical path system simultaneously provides an optical path for the return light, the optical path system can guide the return light to the optical prism 102, the optical prism 102 divides the return light into two paths of white light, one path of white light is transmitted to a light field range where the RGB image sensor 103 collects RGB light signals, and the other path of white light is transmitted to a light field range where the polarized light image sensor 104 collects polarized light signals.
The RGB image sensor 103 is configured to collect RGB optical signals and convert the RGB optical signals into RGB pixel data.
The RGB image sensor 103 in this embodiment can simultaneously collect light signals of three colors of red (R), green (G), and blue (B), and convert the light signals of the three colors into RGB pixel data. The RGB image sensor in the present embodiment is an RGB image sensor having a resolution of 4K, such as a CCD image sensor and a CMOS image sensor. In addition, the RGB pixel data finally output by the RGB image sensor has a resolution of 4K, and the three chip endoscopes are three independent monochrome image sensors, which respectively collect light signals of three colors of red (R), green (G) and blue (B), respectively convert the light signals into R pixel data, G pixel data and B pixel data, and synthesize the R pixel data, G pixel data and B pixel data into RGB pixel data.
The polarized light image sensor 104 is used for collecting polarized light signals and converting the polarized light signals into polarized information pixel data.
In the present embodiment, the polarized light image sensor 104 is capable of collecting a polarized light signal from the above-described return light and converting the polarized light signal into polarization information pixel data, that is, converting an optical signal (polarized light) into an electrical signal (polarization information pixel data).
The image sending module 105 is configured to send the RGB pixel data and the polarization information pixel data to the transmission cable 300, and the transmission cable 300 includes a transmission line for transmitting various signal data. In this embodiment, the input terminal of the image sending module 105 is connected to the output terminals of the RGB image sensor 103 and the polarized light image sensor 104, and the output terminal of the image sending module 105 is connected to the transmission cable 300.
The function selection module 106 is disposed in the camera and connected to the image processor 202, the function selection module 108 includes a key assembly disposed outside the camera housing, the function selection module 108 is configured to output at least one function selection signal to the image processor 202 under the trigger of the key assembly, the image processor 202 controls the endoscope apparatus to complete at least one corresponding function according to the function selection signal, for example, functions such as capturing, freezing, and recording real-time image data after being collected and processed may be performed, and each button on the key assembly triggers the endoscope apparatus to complete the corresponding function when being pressed.
The image transmission module 105 in the present embodiment transmits the RGB pixel data and the polarization information pixel data to the image reception module 201 through the serial transmission data line in the transmission cable 300.
The image receiving module 201 is configured to receive RGB pixel data and polarization information pixel data. In this embodiment, the input end of the image receiving module 201 is connected to the transmission cable 300, and the output end is connected to the input end of the image processor 202.
In an embodiment, the image receiving module 201 is configured to receive the two paths of serial data (RGB pixel data and polarization information pixel data), and convert each path of serial data into a parallel signal, so that the image processor 202 performs image processing
The image processor 202 is configured to perform image processing on the RGB pixel data and perform sharpening processing on the polarization information pixel data synchronously, acquire an RGB image and a polarization information image, and output the RGB image and the polarization information image for display. The output terminal of the image processor 202 in this embodiment is connected to the input terminal of the display interface circuit 203.
The image processor 202 may first process the RGB pixel data and the polarization information pixel data output by the image receiving module 201 through the existing image processing algorithms, such as linear correction, dead pixel removal, white balance, gamma, automatic exposure control, and also adjust the parameters of brightness, saturation, contrast, sharpness, etc. of the output image data. In addition, the present embodiment further adopts a data processing method to perform sharpening processing on the polarization information pixel data, so as to achieve the effect that the endoscope apparatus outputs a sharp polarization information image in an underwater environment (blood, turbid water, tissue debris, fog, etc.), and the RGB image and the polarization information image obtained after being processed by the image processor 202 are sent to the display interface circuit 203 to be displayed on the display.
The display interface circuit 203 receives the RGB image and the polarization information image in synchronization, and code-converts the data formats of the RGB image and the polarization information image into a preset data format, which is associated with a display interface protocol, such as SDI, DVI, DP, or the like.
In an embodiment, the display interface circuit 203 is connected to a first display and a second display by two display cables, the first display is used for displaying RGB images, the second display is used for displaying polarization information images, and the first display for displaying RGB images is used as a main screen and the second display for displaying polarization information images is used as a secondary screen. Since the resolution of the RGB image sensor used in this embodiment is 4K, a 4K RGB image is displayed on the first display.
In another embodiment, a display cable may be further used to connect the display interface circuit 203 to a display, and the display is configured to display the RGB image and the polarization information image in a split-screen display manner in a split-screen manner. Optionally, the first display frame and the second display frame may be in a frame-in-screen manner, that is, an in-screen display, for example, the first display frame is a square screen matched with a display screen, and the second display frame is a circular screen arranged in a preset area in the first display frame.
The wireless transmission circuit 204 is connected with an output end of the image processor 202, and the wireless transmission circuit 204 is used for data interaction between the image processor and an external mobile terminal, and transmitting the RGB image and the polarization information image to the external mobile terminal for display and/or storage. In this way, the image output by the endoscope apparatus can be directly viewed through the mobile terminal, which in this embodiment includes but is not limited to: smart phones, tablet computers, and the like.
The exposure synchronizing signal generating circuit 205 is connected to the RGB image sensor 103 and the polarized light image sensor 104 in the camera 100, and the synchronizing exposure signal generating circuit 205 is configured to output a synchronizing exposure signal to the RGB image sensor and the polarized light image sensor; and the RGB image sensor and the polarized light image sensor synchronously acquire the RGB light signals and the polarized light signals under the triggering of the synchronous exposure signals.
The panel input circuit 206 is used for a user to select and switch at least one function of the endoscope apparatus, for example, for the user to input parameter information of the endoscope to control the endoscope apparatus to implement a function. The input circuit in this embodiment may be a panel key, and may be a display screen with an interactive function, and the like.
The peripheral interface circuit 207 is used for providing an interface for external devices to access the polarized light endoscope apparatus, wherein the external devices may include a mouse keyboard, a USB mobile storage device, an internet port, a serial port device, and the like.
The power supply circuit 208 is used to supply power to each module and circuit of the endoscope apparatus.
In one embodiment, the image processor 202 performs sharpening on the polarization information pixel data, including:
and carrying out layering processing on the current frame image data in the polarization information pixel data to obtain base layer image data and detail layer image data corresponding to the current frame image data. Wherein the base layer image data has a higher contrast and the detail layer image data has a lower contrast.
And respectively carrying out denoising processing on the image data of the base layer and the image data of the detail layer.
And reconstructing the denoised image data of the base layer and the image data of the detail layer to obtain a polarization information image corresponding to the image data of the current frame. Therefore, the finally obtained polarization information image can clearly show a clear scene image under the condition of an underwater environment (blood water, turbid water, tissue debris, fog and the like).
In one embodiment, the image processor 202 performs a hierarchical process on the current frame image data in the polarization information pixel data, including:
extracting image data of continuous frames from the polarization information pixel data, wherein the image data of continuous frames comprises current frame image data, and the current frame image data is a current frame polarization picture.
Calculating a stokes vector (stokes vector) by using a preset polarization angle of image data of continuous frames; the preset bias angle is the polarization angle of the polarized light image sensor. The polarized light image sensor provided by the present embodiment is a polarized light image sensor having 4 polarization angles, which include 0 °, 45 °, 90 °, and 135 °. Therefore, with the 4-polarization-angle image sensor provided in the present embodiment, consecutive 4 frames of image data, that is, consecutive 4 frames of polarization pictures, are extracted from the polarization information pixel data, and the stokes vectors are calculated by the polarization angles 0 °, 45 °, 90 °, and 135 ° of the consecutive 4 frames of polarization pictures. The stokes vector can represent the polarized light intensity of the polarized picture.
And determining the polarization angle corresponding to the maximum value of the polarized light intensity and the polarization angle corresponding to the minimum value of the polarized light intensity in the image data according to the Stokes vector.
And separating the current frame image data into base layer image data and detail layer image data according to the polarization angle corresponding to the maximum value of the polarized light intensity and the polarization angle corresponding to the minimum value of the polarized light intensity. In this embodiment, a joint two-domain filtering method is used to separate the current frame image data into the base layer image data and the detail layer image data.
In an embodiment, the denoising processing is performed on the base layer image data and the detail layer image data, respectively, to obtain the base layer image data and the detail layer image data corresponding to the current frame image data, and the denoising processing includes:
and performing wavelet transformation on the detail layer image data to obtain a plurality of frequency domain data, wherein each frequency domain data corresponds to a different wavelet transformation coefficient.
And carrying out bilateral filtering processing on the plurality of frequency domain data so as to carry out denoising on the plurality of frequency domain data.
And reconstructing the denoised frequency domain data to obtain the denoised detail layer image data.
And carrying out spatial domain and transform domain bilateral filtering on the base layer image data so as to carry out denoising on the base layer image data.
And carrying out iterative optimization processing on the base layer image data after bilateral filtering processing to obtain the denoised base layer image data.
In the embodiment of the invention, the optical lens and the optical corner angle respectively transmit the reflected white light to the RGB image sensor and the polarized light image sensor, the RGB image sensor collects RGB optical signals and outputs RGB pixel data, the polarized light image sensor collects polarized light signals and outputs polarized information pixel data, the image processor synchronously processes the RGB pixel data and the polarized light pixel data, wherein the polarized light pixel data is processed by adopting the data processing method provided by the embodiment of the invention, the polarized light pixel data is separated into detail layer image data and basic layer image data, and the detail layer image data and the basic layer image data are respectively denoised, the noise in the polarized light information pixel data can be removed in different dimensions, and finally the denoised detail layer image data and basic layer image data are reconstructed into a polarized information image, the endoscope device can clearly image in underwater environment (blood, turbidity, tissue scraps and fog).
Referring to fig. 3, fig. 3 is a flowchart of a data processing method for an endoscope apparatus according to an embodiment, the data processing method is applied to an image processor in an image processing system, and the data processing method includes the following steps:
And step 20, synchronously performing image processing on the RGB pixel data and performing sharpening processing on the polarization information pixel data to obtain an RGB image and a polarization information image, and outputting the RGB image and the polarization information image for display.
Referring to fig. 4, fig. 4 is a detailed flowchart of the data processing method for the endoscopic device shown in fig. 3. In one embodiment, the step 20 of sharpening the polarization information data includes the following steps:
and step 21, performing layering processing on the current frame image data in the polarization information pixel data to obtain base layer image data and detail layer image data corresponding to the current frame image data.
And 22, denoising the base layer image data and the detail layer image data respectively.
And step 23, reconstructing the denoised image data of the base layer and the image data of the detail layer to obtain a polarization information image corresponding to the image data of the current frame.
In one embodiment, the step 21 of performing a layering process on the current frame image data in the polarization information pixel data includes the following steps:
step 211, extracting image data of continuous frames from the polarization information pixel data, wherein the image data of continuous frames comprises current frame image data.
Step 212, calculating a Stokes vector by using the preset polarization angle of the image data of the continuous frames; the preset bias angle is a polarization angle of the polarized light image sensor.
And step 213, determining the polarization angle corresponding to the maximum value of the polarized light intensity and the polarization angle corresponding to the minimum value of the polarized light intensity in the image data according to the Stokes vector.
Step 214, according to the polarization angle corresponding to the maximum value of the polarized light intensity and the polarization angle corresponding to the minimum value of the polarized light intensity, the current frame image data is separated into the base layer image data and the detail layer image data.
In an embodiment, the step 22 of performing denoising processing on the base layer image data and the detail layer image data respectively includes the following steps:
In the embodiment, a plurality of frequency domain data are obtained by changing the scale factor and the response factor of the wavelet transform, wherein the plurality of frequency domain data refer to detail layer image data with different frequencies, different bandwidths and different time instants.
And 223, reconstructing the denoised frequency domain data to obtain the denoised detail layer image data.
And 224, carrying out spatial domain and transform domain bilateral filtering on the base layer image data so as to carry out denoising on the base layer image data. In the embodiment, the image data of the base layer is denoised by controlling the size change of the kernel function transform domain and the spatial domain of the bilateral filtering, so that the image data of the base layer with high precision and low noise is obtained.
And 225, performing iterative optimization processing on the base layer image data after bilateral filtering processing to obtain the denoised base layer image data.
Those skilled in the art will appreciate that all or part of the functions of the various methods in the above embodiments may be implemented by hardware, or may be implemented by computer programs. When all or part of the functions of the above embodiments are implemented by a computer program, the program may be stored in a computer-readable storage medium, and the storage medium may include: a read only memory, a random access memory, a magnetic disk, an optical disk, a hard disk, etc., and the program is executed by a computer to realize the above functions. For example, the program may be stored in a memory of the device, and when the program in the memory is executed by the processor, all or part of the functions described above may be implemented. In addition, when all or part of the functions in the above embodiments are implemented by a computer program, the program may be stored in a storage medium such as a server, another computer, a magnetic disk, an optical disk, a flash disk, or a removable hard disk, and may be downloaded or copied to a memory of a local device, or may be version-updated in a system of the local device, and when the program in the memory is executed by a processor, all or part of the functions in the above embodiments may be implemented.
The present invention has been described in terms of specific examples, which are provided to aid understanding of the invention and are not intended to be limiting. Numerous simple deductions, modifications or substitutions may also be made by those skilled in the art in light of the present teachings.
Claims (10)
1. A polarized light endoscope apparatus comprising a camera and an image processing system, characterized by further comprising:
the optical prism is arranged in the camera and used for dividing one path of light beam output by the optical lens into two paths of light beams;
the RGB image sensor is arranged in the camera and used for collecting RGB optical signals of one path of light beam output by the optical prism and converting the RGB optical signals into RGB pixel data;
the polarized light image sensor is arranged in the camera and used for collecting a polarized light signal of the other path of light beam output by the optical prism and converting the polarized light signal into polarized information pixel data;
and the image processor is arranged in the image processing system and connected with the camera, and is used for synchronously carrying out image processing on the RGB pixel data and carrying out sharpening processing on the polarization information pixel data, acquiring an RGB image and a polarization information image, and outputting the RGB image and the polarization information image for display.
2. The polarized light endoscopic device of claim 1 wherein said sharpening said polarization information pixel data comprises:
performing layering processing on the current frame image data in the polarization information pixel data to obtain base layer image data and detail layer image data corresponding to the current frame image data;
denoising the base layer image data and the detail layer image data respectively;
and reconstructing the denoised image data of the base layer and the image data of the detail layer to obtain a polarization information image corresponding to the image data of the current frame.
3. The polarized light endoscope apparatus according to claim 2, wherein said layering current frame image data in said polarization information pixel data to obtain base layer image data and detail layer image data corresponding to the current frame image data comprises:
extracting image data of continuous frames from the polarization information pixel data, wherein the image data of the continuous frames comprises current frame image data;
calculating a Stokes vector by using a preset polarization angle of the image data with continuous frames; the preset bias angle is a polarization angle of the polarized light image sensor;
determining a polarization angle corresponding to the maximum value of the polarized light intensity and a polarization angle corresponding to the minimum value of the polarized light intensity in the image data according to the Stokes vector;
and separating the current frame image data into base layer image data and detail layer image data according to the polarization angle corresponding to the maximum value of the polarized light intensity and the polarization angle corresponding to the minimum value of the polarized light intensity.
4. The polarized light endoscope apparatus of claim 2 wherein said de-noising of said base layer image data and detail layer image data, respectively, comprises:
performing wavelet transformation on the detail layer image data to obtain a plurality of frequency domain data, wherein each frequency domain data corresponds to a different wavelet transformation coefficient;
carrying out bilateral filtering processing on the plurality of frequency domain data so as to carry out denoising on the plurality of frequency domain data;
reconstructing the denoised frequency domain data to obtain denoised detail layer image data;
carrying out bilateral filtering on the image data of the basic layer in a space domain and a transform domain so as to carry out denoising on the image data of the basic layer;
and carrying out iterative optimization processing on the base layer image data after bilateral filtering processing to obtain the denoised base layer image data.
5. The polarized light endoscope apparatus of claim 1 wherein the polarized light image sensor is a polarized light image sensor having 4 polarization angles, the polarization angles including 0 °, 45 °, 90 ° and 135 °.
6. The polarized light endoscope apparatus of claim 1 further comprising:
the image sending module is arranged in the camera, is respectively connected with the polarized light image sensor and the RGB image sensor, and is used for sending the RGB pixel data and the polarization information pixel data;
and the image receiving module is arranged in the image processing system and used for receiving the RGB pixel data and the polarization information pixel data and forwarding the RGB pixel data and the polarization information pixel data to an image processor.
7. The polarized light endoscope apparatus of claim 1 further comprising:
a first display for displaying the RGB image;
a second display for displaying the polarization information image;
the display interface circuit is arranged in the image processing system and connected with the image processor and used for receiving the RGB image and the polarization information image output by the image processor, respectively encoding and converting the data formats of the RGB image and the polarization information image into a preset data format, outputting the RGB image in the preset data format to the first display and outputting the polarization information image in the preset data format to the second display.
8. The polarized light endoscope apparatus of claim 1 further comprising:
the wireless transmission circuit is arranged in the image processing system and connected with the image processor, and is used for the image processor to perform data interaction with the external mobile terminal and transmit the RGB image and the polarization information image to the external mobile terminal for display and/or storage.
9. The polarized light endoscope apparatus of claim 1 further comprising:
the clock circuit is arranged in the camera, is respectively connected with the RGB image sensor and the polarized light image sensor, and is used for generating a synchronous clock signal and sending the synchronous clock signal to the RGB image sensor and the polarized light image sensor;
the function selection module is arranged in the camera and connected to the image processor, and comprises a key assembly and is used for outputting at least one function selection signal to the image processor under the triggering of the key assembly so as to control the polarized light endoscope device to complete at least one corresponding function.
10. The polarized light endoscopic apparatus of claim 1, further comprising:
the exposure synchronizing signal generating circuit is arranged in the image processing system, is respectively connected with the RGB image sensor and the polarized light image sensor, and is used for outputting a synchronizing exposure signal to the RGB image sensor and the polarized light image sensor; the RGB image sensor and the polarized light image sensor are specifically used for respectively and synchronously acquiring the RGB optical signals and the polarized light signals under the triggering of the synchronous exposure signals;
a panel input circuit disposed within the image processing system for user selection and switching of at least one function of the polarized light endoscope apparatus;
the external interface circuit is arranged in the image processing system and is used for providing an interface for external equipment to be accessed into the polarized light endoscope device;
and the power supply circuit is arranged in the image processing system and used for supplying power to each module and circuit of the polarized light endoscope device.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110849690.8A CN113491497B (en) | 2021-07-27 | 2021-07-27 | Polarized light endoscope device |
| PCT/CN2022/094337 WO2023005374A1 (en) | 2021-07-27 | 2022-05-23 | Polarized light endoscope apparatus and data processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110849690.8A CN113491497B (en) | 2021-07-27 | 2021-07-27 | Polarized light endoscope device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113491497A CN113491497A (en) | 2021-10-12 |
| CN113491497B true CN113491497B (en) | 2022-08-12 |
Family
ID=77996825
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110849690.8A Active CN113491497B (en) | 2021-07-27 | 2021-07-27 | Polarized light endoscope device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113491497B (en) |
| WO (1) | WO2023005374A1 (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113491497B (en) * | 2021-07-27 | 2022-08-12 | 重庆西山科技股份有限公司 | Polarized light endoscope device |
| CN114176485B (en) * | 2021-12-16 | 2023-08-04 | 重庆西山科技股份有限公司 | Endoscope device camera polarized light optical system, camera and endoscope device |
| CN114176483B (en) * | 2021-12-16 | 2023-08-29 | 重庆西山科技股份有限公司 | Polarized light endoscope device camera optical system, camera and endoscope device |
| CN114176484A (en) * | 2021-12-16 | 2022-03-15 | 重庆西山科技股份有限公司 | Polarized light endoscope device, camera and camera optical system |
| CN115147394A (en) * | 2022-07-21 | 2022-10-04 | 重庆西山科技股份有限公司 | Model training method, image processing method, endoscope apparatus, and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1510080A2 (en) * | 2002-05-13 | 2005-03-02 | Visionsense Ltd | Optical device |
| JP2005096086A (en) * | 2003-09-22 | 2005-04-14 | Minolta Co Ltd | Image forming apparatus |
| WO2009009414A2 (en) * | 2007-07-06 | 2009-01-15 | Lawrence Livermore National Security, Llc | Simultaneous acquisition of differing image types |
| JP2012024283A (en) * | 2010-07-22 | 2012-02-09 | Fujifilm Corp | Endoscope diagnostic apparatus |
| JP2012045029A (en) * | 2010-08-24 | 2012-03-08 | Fujifilm Corp | Endoscope apparatus and endoscope diagnostic apparatus |
| WO2012174633A1 (en) * | 2011-06-21 | 2012-12-27 | Isee3D Inc. | Method and apparatus for generating three-dimensional image information |
| JP2014108193A (en) * | 2012-11-30 | 2014-06-12 | Panasonic Corp | Image processing device and endoscope |
| US10924689B1 (en) * | 2019-11-22 | 2021-02-16 | Karl Storz Imaging, Inc. | Method and apparatus to improve high dynamic range image capture using image sensor with polarization |
| CN112686865A (en) * | 2020-12-31 | 2021-04-20 | 重庆西山科技股份有限公司 | 3D view auxiliary detection method, system, device and storage medium |
| CN113138467A (en) * | 2021-04-15 | 2021-07-20 | 西北农林科技大学 | LCVR-based compressed sensing hyperspectral polarization imaging system and method |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5100457B2 (en) * | 2008-03-10 | 2012-12-19 | オリンパスメディカルシステムズ株式会社 | Endoscope observation system |
| JP5764747B2 (en) * | 2011-09-02 | 2015-08-19 | パナソニックＩｐマネジメント株式会社 | Polarization imaging device and endoscope |
| EP2676596A4 (en) * | 2011-11-29 | 2015-09-23 | Olympus Medical Systems Corp | Polarization observation device |
| WO2013175686A1 (en) * | 2012-05-22 | 2013-11-28 | パナソニック株式会社 | Image pickup processing device and endoscope |
| KR101617068B1 (en) * | 2012-10-11 | 2016-05-02 | 이문기 | Image processing system using polarization difference camera |
| CN106683056A (en) * | 2016-12-16 | 2017-05-17 | 凯迈（洛阳）测控有限公司 | Airborne photoelectric infrared digital image processing method and apparatus thereof |
| JP2020062243A (en) * | 2018-10-18 | 2020-04-23 | 株式会社ニコン | Endoscopic apparatus and imaging method for the same |
| CN111198445B (en) * | 2018-11-16 | 2021-10-01 | 华为技术有限公司 | Equipment and method for light-splitting polarization imaging |
| CN109754384B (en) * | 2018-12-18 | 2022-11-22 | 电子科技大学 | Infrared polarization image fusion method of uncooled infrared focal plane array |
| EP3816692A1 (en) * | 2019-10-30 | 2021-05-05 | Carl Zeiss Microscopy GmbH | Image conversion module with a microelectromechanical optical system and method for applying the same |
| CN214342215U (en) * | 2020-11-30 | 2021-10-08 | 重庆西山科技股份有限公司 | Endoscope device with detection function |
| CN113491497B (en) * | 2021-07-27 | 2022-08-12 | 重庆西山科技股份有限公司 | Polarized light endoscope device |
-
2021
- 2021-07-27 CN CN202110849690.8A patent/CN113491497B/en active Active
-
2022
- 2022-05-23 WO PCT/CN2022/094337 patent/WO2023005374A1/en active Application Filing
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1510080A2 (en) * | 2002-05-13 | 2005-03-02 | Visionsense Ltd | Optical device |
| JP2005096086A (en) * | 2003-09-22 | 2005-04-14 | Minolta Co Ltd | Image forming apparatus |
| WO2009009414A2 (en) * | 2007-07-06 | 2009-01-15 | Lawrence Livermore National Security, Llc | Simultaneous acquisition of differing image types |
| JP2012024283A (en) * | 2010-07-22 | 2012-02-09 | Fujifilm Corp | Endoscope diagnostic apparatus |
| JP2012045029A (en) * | 2010-08-24 | 2012-03-08 | Fujifilm Corp | Endoscope apparatus and endoscope diagnostic apparatus |
| WO2012174633A1 (en) * | 2011-06-21 | 2012-12-27 | Isee3D Inc. | Method and apparatus for generating three-dimensional image information |
| JP2014108193A (en) * | 2012-11-30 | 2014-06-12 | Panasonic Corp | Image processing device and endoscope |
| US10924689B1 (en) * | 2019-11-22 | 2021-02-16 | Karl Storz Imaging, Inc. | Method and apparatus to improve high dynamic range image capture using image sensor with polarization |
| CN112686865A (en) * | 2020-12-31 | 2021-04-20 | 重庆西山科技股份有限公司 | 3D view auxiliary detection method, system, device and storage medium |
| CN113138467A (en) * | 2021-04-15 | 2021-07-20 | 西北农林科技大学 | LCVR-based compressed sensing hyperspectral polarization imaging system and method |
Non-Patent Citations (1)
| Title |
|---|
| 多尺度水下偏振成像方法;韩平丽；刘飞；张广;《物理学报》;20180531;第67卷(第5期);054202-1-10 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113491497A (en) | 2021-10-12 |
| WO2023005374A1 (en) | 2023-02-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113491497B (en) | Polarized light endoscope device | |
| US10966592B2 (en) | 3D endoscope apparatus and 3D video processing apparatus | |
| US7634305B2 (en) | Method and apparatus for size analysis in an in vivo imaging system | |
| KR101921268B1 (en) | Capsule endoscopy apparatus for rendering 3d image, operation method of said capsule endoscopy, receiver rendering 3d image interworking with said capsule endoscopy, and capsule endoscopy system | |
| US10799087B2 (en) | Endoscope system, operation method for endoscope system, and program for balancing conflicting effects in endoscopic imaging | |
| US10943333B2 (en) | Method and apparatus of sharpening of gastrointestinal images based on depth information | |
| KR20110016896A (en) | System and method for generating a multi-dimensional image | |
| JP7449736B2 (en) | Medical image processing device and medical observation system | |
| US20110135170A1 (en) | System and method for display speed control of capsule images | |
| CN109068035B (en) | Intelligent micro-camera array endoscopic imaging system | |
| CN110738655A (en) | Image report generation method, device, terminal and storage medium | |
| CN214342215U (en) | Endoscope device with detection function | |
| CN114693593A (en) | Image processing method, device and computer device | |
| CN112884666A (en) | Image processing method, image processing device and computer storage medium | |
| US11354783B2 (en) | Method and apparatus of sharpening of gastrointestinal images based on depth information | |
| JPWO2018088215A1 (en) | Endoscope system | |
| US9323978B2 (en) | Image processing device, endoscope apparatus, and image processing method | |
| KR102212964B1 (en) | Deblurring image processing method and apparatus | |
| JPH07360A (en) | Image pickup system | |
| US20220007925A1 (en) | Medical imaging systems and methods | |
| CN215383853U (en) | Polarized light endoscope device | |
| US10624533B2 (en) | Endoscope with images optimized based on depth map derived from structured light images | |
| WO2021124942A1 (en) | Imaging device, information processing method, and program | |
| JP2010092283A (en) | Stereoscopic image creation device and method, and endoscope examination system | |
| KR101878891B1 (en) | Image processing method of capsule endoscope, capsule endoscope apparatus, receiver interworking with capsule endoscope, image processing method of the receiver, and capsule endoscope system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |