CN117197064A - Automatic non-contact eye red degree analysis method - Google Patents
Automatic non-contact eye red degree analysis method Download PDFInfo
- Publication number
- CN117197064A CN117197064A CN202311115420.XA CN202311115420A CN117197064A CN 117197064 A CN117197064 A CN 117197064A CN 202311115420 A CN202311115420 A CN 202311115420A CN 117197064 A CN117197064 A CN 117197064A
- Authority
- CN
- China
- Prior art keywords
- image
- eye
- area
- map
- color
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 201000005111 ocular hyperemia Diseases 0.000 title claims abstract description 44
- 238000004458 analytical method Methods 0.000 title claims abstract description 27
- 238000001514 detection method Methods 0.000 claims abstract description 26
- 238000007781 pre-processing Methods 0.000 claims abstract description 24
- 230000003044 adaptive effect Effects 0.000 claims abstract description 8
- 238000003706 image smoothing Methods 0.000 claims abstract description 8
- 210000003786 sclera Anatomy 0.000 claims description 31
- 238000000034 method Methods 0.000 claims description 21
- 238000004364 calculation method Methods 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 11
- 238000003384 imaging method Methods 0.000 claims description 11
- 230000011218 segmentation Effects 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 8
- 239000008280 blood Substances 0.000 claims description 7
- 210000004369 blood Anatomy 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 5
- 230000007246 mechanism Effects 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 5
- 238000013136 deep learning model Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 238000013135 deep learning Methods 0.000 claims description 3
- 230000000717 retained effect Effects 0.000 claims 1
- 238000003745 diagnosis Methods 0.000 abstract description 5
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000001186 cumulative effect Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 238000005315 distribution function Methods 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 208000003556 Dry Eye Syndromes Diseases 0.000 description 1
- 206010013774 Dry eye Diseases 0.000 description 1
- 206010061218 Inflammation Diseases 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000004204 blood vessel Anatomy 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000004054 inflammatory process Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000005316 response function Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
本发明公开一种无接触眼红程度自动分析方法,包括以下步骤:对准脸部正面拍摄人眼,对拍摄图像进行人眼区域检测及图像预处理,进行缩放和预处理后使用U‑Net++模型进行巩膜区域检测,得到二值图,将二值图与缩放图进行与操作,得到巩膜区域彩色图;进行图像平滑处理与自适应直方图均衡化来增强图像对比度,使用B‑COSFIRE滤波器对图像增强图进行滤波处理,同时将图像增强图转化为LAB颜色模型,获取得到图像掩膜,最后将滤波处理图和图像掩膜进行与操作,获取血丝二值图;分别计算巩膜区域彩色图和血丝二值图中非零像素点个数,再进行比值计算,得到眼红占比。本发明提供的一种无接触眼红程度自动分析方法,自动判定适应性和准确性高,能够实现患者眼红程度的智能辅助诊断。
The invention discloses a non-contact automatic analysis method of eye redness, which includes the following steps: aiming at the front of the face to capture human eyes, performing human eye area detection and image preprocessing on the captured images, scaling and preprocessing, and then using the U‑Net++ model. Perform scleral area detection to obtain a binary image, and operate the binary image and the zoom image to obtain a color image of the scleral area; perform image smoothing and adaptive histogram equalization to enhance image contrast, and use the B‑COSFIRE filter to The image enhancement map is filtered, and at the same time, the image enhancement map is converted into a LAB color model to obtain the image mask. Finally, the filtered map and the image mask are ANDed to obtain the bloodshot binary image; the scleral area color map and The number of non-zero pixels in the bloodshot binary image is then calculated as a ratio to obtain the ratio of jealousy. The invention provides a non-contact automatic analysis method for the degree of eye redness, which has high adaptability and accuracy in automatic determination and can realize intelligent auxiliary diagnosis of the patient's degree of eye redness.
Description
技术领域Technical field
本发明涉及一种无接触眼红程度自动分析方法,属于图像处理及医疗辅助诊断技术领域。The invention relates to a non-contact automatic analysis method of eye redness, and belongs to the technical fields of image processing and medical auxiliary diagnosis.
背景技术Background technique
眼红分析可判断眼表炎症的严重程度,作为干眼检测的重要指标之一,在眼科医疗诊断领域有着重要作用。传统的眼红分析由医生根据经验判定,存在误差较大的缺陷。另外,虽然出现数字图像处理方法,但是目前的数字图像处理方法的自动判定适应性和准确性不高,且无法实现患者眼红程度的智能辅助诊断。Eye redness analysis can determine the severity of ocular surface inflammation. As one of the important indicators for dry eye detection, it plays an important role in the field of ophthalmic medical diagnosis. Traditional eye redness analysis is judged by doctors based on experience, which has the disadvantage of large errors. In addition, although digital image processing methods have emerged, the current digital image processing methods have low adaptability and accuracy in automatic determination, and cannot achieve intelligent auxiliary diagnosis of the patient's degree of redness.
发明内容Contents of the invention
本发明要解决的技术问题是,克服现有技术的缺陷,提供一种无接触眼红程度自动分析方法,自动判定适应性和准确性高,能够实现患者眼红程度的智能辅助诊断。The technical problem to be solved by the present invention is to overcome the shortcomings of the existing technology and provide a non-contact automatic analysis method for the degree of eye redness, which has high adaptability and accuracy in automatic determination and can realize intelligent auxiliary diagnosis of the patient's degree of eye redness.
为解决上述技术问题,本发明采用的技术方案为:In order to solve the above technical problems, the technical solutions adopted by the present invention are:
一种无接触眼红程度自动分析方法,包括以下步骤:A non-contact automated eye redness analysis method, including the following steps:
使用可见光成像设备对准脸部正面拍摄人眼,对拍摄图像进行人眼区域检测及图像预处理,得到预处理图像;Use visible light imaging equipment to shoot the human eyes from the front of the face, perform human eye area detection and image preprocessing on the captured images to obtain preprocessed images;
将预处理图像进行缩放后得到缩放图,对缩放图进行预处理后使用U-Net++模型进行巩膜区域检测,得到待检图像巩膜区域的二值图,将二值图与缩放图进行与操作,得到巩膜区域彩色图;The preprocessed image is scaled to obtain a zoomed image. After preprocessing the zoomed image, the U-Net++ model is used to detect the scleral area, and a binary image of the scleral area of the image to be inspected is obtained. The binary image is combined with the zoomed image. Get a color map of the sclera area;
将巩膜区域彩色图进行图像平滑处理与自适应直方图均衡化来增强图像对比度,得到图像增强图,然后使用B-COSFIRE滤波器对图像增强图进行滤波处理,得到滤波处理图,同时将图像增强图转化为LAB颜色模型,获取得到图像掩膜,最后将滤波处理图和图像掩膜进行与操作,获取血丝二值图;Perform image smoothing and adaptive histogram equalization on the scleral area color map to enhance image contrast and obtain an image enhancement map. Then use the B-COSFIRE filter to filter the image enhancement map to obtain a filtered map. At the same time, the image is enhanced. The image is converted into a LAB color model, and the image mask is obtained. Finally, the filtered image and the image mask are ANDed to obtain the bloodshot binary image;
分别计算巩膜区域彩色图和血丝二值图中非零像素点个数,再对血丝二值图非零像素点个数和巩膜区域彩色图非零像素点个数进行比值计算,得到眼红占比。Calculate the number of non-zero pixels in the color map of the sclera area and the binary blood map respectively, and then calculate the ratio of the number of non-zero pixels in the binary blood map and the color map of the sclera area to obtain the proportion of eye redness. .
所述人眼区域检测包括以下步骤:The human eye area detection includes the following steps:
对拍摄图像进行检测时,利用训练好的分类器在拍摄图像中从左上角开始逐区域搜索,采用相似性准则判断是否为人眼;When detecting the captured image, use the trained classifier to search area by area starting from the upper left corner of the captured image, and use the similarity criterion to determine whether it is a human eye;
若判断结果为人眼,则框选出眼部区域所在的范围,判断眼部区域范围是否满足不低于256×256,若是则进入下一步流程,若否则则提示人眼图像太小,要求重新拍摄;If the judgment result is human eyes, select the range where the eye area is located and determine whether the eye area range is no less than 256×256. If so, proceed to the next step. If not, it will prompt that the human eye image is too small and require a new process. Photography;
若判断结果为非人眼,则提示输入图像错误,要求重新拍摄;If the judgment result is not human eyes, it will prompt that the input image is wrong and require re-shooting;
其中,人眼分类器训练过程为:采用深度学习方法,先从网络上采集大量的人眼和非人眼样本进行预训练,得到深度学习模型参数以构建人眼分类器。Among them, the training process of the human eye classifier is: using the deep learning method, first collect a large number of human eye and non-human eye samples from the Internet for pre-training, and obtain the deep learning model parameters to build the human eye classifier.
拍摄图像进行预处理包括以下步骤:Taking images for preprocessing involves the following steps:
根据眼部区域框选结果,将人眼区域长宽按原比例扩展1.2倍,使最终框选的眼部区域包含眼角完整的眼部信息,以扩展后的长度为基准,平行中心线为基线,5:4的长宽比进行裁剪,使所有眼部图像符合尺寸要求及有效区域最大化。According to the result of the eye area frame selection, the length and width of the human eye area are expanded by 1.2 times according to the original proportion, so that the final eye area selected by the frame contains complete eye information at the corner of the eye. The expanded length is used as the benchmark, and the parallel center line is the baseline. , cropped with an aspect ratio of 5:4 so that all eye images meet the size requirements and maximize the effective area.
缩放图预处理包括以下步骤:利用颜色阈值法,若判断图像整体颜色偏红,则提取图像RGB三通道中的的R通道,继续进行巩膜分割操作,否则使用直方图均衡化的方法进行图像增强后,仍然以RGB三通道彩色图进行巩膜分割操作。The zoom image preprocessing includes the following steps: using the color threshold method, if the overall color of the image is judged to be reddish, extract the R channel of the three RGB channels of the image and continue the scleral segmentation operation; otherwise, use the histogram equalization method for image enhancement Finally, the scleral segmentation operation is still performed using the RGB three-channel color image.
U-Net++模型为改进的U-Net++模型,输入图像是缩放图,输出图像是单通道的二值图,输入输出图像形状都是512×512;在U-Net++的每一层卷积操作与ReLU激活函数操作之间,添加了归一化操作;在上层与下层之间,添加dropout操作,并在之后添加注意力机制,随后进行池化操作。The U-Net++ model is an improved U-Net++ model. The input image is a zoomed image, the output image is a single-channel binary image, and the input and output image shapes are both 512×512; the convolution operation of each layer of U-Net++ is the same as Between the ReLU activation function operations, a normalization operation is added; between the upper layer and the lower layer, a dropout operation is added, and an attention mechanism is added afterwards, followed by a pooling operation.
二值图与缩放图进行与操作具体为:将二值图中像素值为0的所有像素点,对应缩放图中的像素点值置为0,仅保留图像中巩膜区域,得到巩膜区域图。The specific operation of the binary image and the zoom image is as follows: set all the pixels with a pixel value of 0 in the binary image to 0 for the corresponding pixel values in the zoom image, and only retain the sclera area in the image to obtain the sclera area map.
眼红占比具体计算方式如下:The specific calculation method of jealousy proportion is as follows:
其中,Degree表示眼红占比,Bpixels表示血丝二值图中非0像素点个数,Spixels表示巩膜区域图中非0像素点个数。Among them, Degree represents the proportion of jealousy, B pixels represents the number of non-0 pixels in the bloodshot binary image, and S pixels represents the number of non-0 pixels in the sclera area map.
将眼红占比计算结果乘以100后向下取整,使得最终结果位于[0,100]范围内,最后使用分数等级制,得出相应眼红程度级别。Multiply the calculation result of the jealousy proportion by 100 and round down to the nearest whole, so that the final result is within the range of [0,100]. Finally, use the fractional scale to obtain the corresponding jealousy level.
一种无接触眼红程度自动分析装置,包括:A non-contact automatic eye redness analysis device, including:
眼部图像获取与处理模块,用于使用可见光成像设备对准脸部正面拍摄人眼,对拍摄图像进行人眼区域检测及图像预处理,得到预处理图像;The eye image acquisition and processing module is used to use visible light imaging equipment to shoot human eyes from the front of the face, and perform human eye area detection and image preprocessing on the captured images to obtain preprocessed images;
巩膜区域检测模块,用于将预处理图像进行缩放后得到缩放图,对缩放图进行预处理后使用U-Net++模型对缩放图进行巩膜区域检测,得到待检图像巩膜区域的二值图,将二值图与缩放图进行与操作,得到巩膜区域彩色图;The scleral area detection module is used to scale the preprocessed image to obtain a zoomed image. After preprocessing the zoomed image, use the U-Net++ model to detect the scleral area of the zoomed image to obtain a binary image of the scleral area of the image to be inspected. Perform operations on the binary image and the zoom image to obtain a color image of the sclera area;
眼红区域提取模块,用于提取将巩膜区域彩色图进行图像平滑处理与自适应直方图均衡化来增强图像对比度,得到图像增强图,然后使用B-COSFIRE滤波器对图像增强图进行滤波处理,得到滤波处理图,同时将图像增强图转化为LAB颜色模型,获取得到图像掩膜,最后将滤波处理图和图像掩膜进行与操作,获取血丝二值图;The eye-red area extraction module is used to extract the color image of the sclera area, perform image smoothing and adaptive histogram equalization to enhance the image contrast, and obtain the image enhancement map, and then use the B-COSFIRE filter to filter the image enhancement map to obtain Filter the processing map, and at the same time convert the image enhancement map into a LAB color model to obtain the image mask. Finally, perform the AND operation on the filtering processing map and the image mask to obtain the bloodshot binary image;
眼红占比计算模块,用于分别计算巩膜区域彩色图和血丝二值图中非零像素点个数,再对血丝二值图非零像素点个数和巩膜区域彩色图非零像素点个数进行比值计算,得到眼红占比。The eye red proportion calculation module is used to calculate the number of non-zero pixels in the sclera area color image and the bloodshot binary image respectively, and then calculate the number of non-zero pixels in the bloodshot binary image and the sclera area color image. Perform a ratio calculation to get the jealousy ratio.
一种计算机可读存储介质,其上存储有计算机程序,该计算机程序被处理器执行时,实现所述接触眼红程度自动分析方法。A computer-readable storage medium has a computer program stored thereon. When the computer program is executed by a processor, the method for automatic analysis of the degree of contact envy is implemented.
本发明的有益效果:本发明提供一种无接触眼红程度自动分析方法,使用可见光成像设备对准脸部正面拍摄人眼,以使用普通智能手机照相功能或相机拍照实现,无接触的眼部图像获取,无辅助光源,结构简单,无不适;能够避免眼红呈现区域,眼周充血带来的误差,眼红位置及面积精准,提供准确数据,自动判定适应性和准确性高。Beneficial effects of the present invention: The present invention provides a non-contact automatic analysis method of eye redness, using visible light imaging equipment to aim at the front of the face to photograph human eyes, and using the camera function of an ordinary smartphone or camera to achieve non-contact eye images Acquisition, no auxiliary light source, simple structure, no discomfort; it can avoid errors caused by eye redness appearing areas and congestion around the eyes, accurate eye red position and area, provide accurate data, and automatically determine the adaptability and accuracy.
附图说明Description of the drawings
图1为本发明一种无接触眼红程度自动分析方法的工作流程图。Figure 1 is a work flow chart of a non-contact automatic eye redness analysis method according to the present invention.
图2为本发明中眼部图像获取与处理流程图;Figure 2 is a flow chart of eye image acquisition and processing in the present invention;
图3为本发明中巩膜区域检测流程图;Figure 3 is a flow chart of sclera area detection in the present invention;
图4为本发明中眼红区域提取流程图;Figure 4 is a flow chart of eye red area extraction in the present invention;
图5为本发明中眼红占比计算及眼红程度判断流程图。Figure 5 is a flow chart for calculating the ratio of eye redness and judging the degree of eye redness in the present invention.
具体实施方式Detailed ways
下面结合附图对本发明作进一步描述,以下实施例仅用于更加清楚地说明本发明的技术方案,而不能以此来限制本发明的保护范围。如图1所示,本发明公开一种无接触眼红程度自动分析方法,包括以下步骤:The present invention will be further described below in conjunction with the accompanying drawings. The following examples are only used to illustrate the technical solutions of the present invention more clearly, but cannot be used to limit the scope of protection of the present invention. As shown in Figure 1, the present invention discloses a non-contact automatic analysis method for eye redness, which includes the following steps:
步骤一,眼部图像获取与处理。Step 1: Eye image acquisition and processing.
使用一个可见光成像设备对准脸部正面拍摄人眼,成图中须包含完整的单只人眼。因成像设备分辨率的不同,会对巩膜区域检测神经网络训练占用的显存、巩膜区域检测结果及算法执行效率造成影响,需先进行人眼区域检测及图像预处理。使用可见光成像设备对准脸部正面拍摄人眼,可以使用普通智能手机照相功能或相机拍照实现,只需要满足眼部区域成像大小不低于256×256,并不需要额外的设备或眼科的专用仪器。Use a visible light imaging device to shoot the human eye from the front of the face, and the image must contain a complete single human eye. Due to different resolutions of imaging equipment, it will affect the video memory occupied by scleral area detection neural network training, scleral area detection results and algorithm execution efficiency. Human eye area detection and image preprocessing must be performed first. Using visible light imaging equipment to aim at the front of the face to take pictures of human eyes can be achieved by using the camera function of an ordinary smartphone or camera. It only needs to satisfy that the image size of the eye area is not less than 256×256, and does not require additional equipment or special ophthalmology equipment. instrument.
人眼区域检测,通过对采集到的大量人眼和非人眼样本进行预训练,得到模型参数以构建人眼分类器。对拍摄图像进行检测时,利用训练好的分类器在拍摄图像中从左上角开始逐区域搜索,采用相似性准则判断是否为人眼。逐区域搜索,是将输入图像进行划分,划分出若干个小的矩形区域,对每个小区域分别进行人眼的检测判断。相似性准则,是指在逐区域搜索过程中,对每个小区域内检测到的物体使用人眼分类器模型进行相似性特征的计算,计算结果表示检测到的物体与人眼的相似程度。For human eye area detection, by pre-training a large number of collected human eye and non-human eye samples, the model parameters are obtained to build a human eye classifier. When detecting the captured image, use the trained classifier to search region by region starting from the upper left corner of the captured image, and use the similarity criterion to determine whether it is the human eye. Area-by-area search is to divide the input image into several small rectangular areas, and perform human eye detection and judgment on each small area. The similarity criterion refers to using the human eye classifier model to calculate the similarity features of the objects detected in each small area during the area-by-area search process. The calculation results represent the degree of similarity between the detected objects and the human eye.
若判断结果为人眼,还需要框选出眼部区域所在的范围,眼部区域范围不低于256×256,则继续进行下一步图像预处理,否则提示人眼图像太小,要求重新拍摄。若判断结果为非人眼,则提示输入图像错误,要求重新拍摄。图像预处理,是根据眼部区域框选结果,将人眼区域长宽按原比例扩展1.2倍,使最终框选的眼部区域包含眼角等完整的眼部信息。以扩展后的长度为基准,平行中心线为基线、5:4的长宽比进行裁剪,使所有眼部图像符合尺寸要求及有效区域最大化。If the judgment result is human eyes, you also need to frame the range of the eye area. The range of the eye area is not less than 256×256. Then continue to the next step of image preprocessing. Otherwise, it will prompt that the human eye image is too small and require re-shooting. If the judgment result is not human eyes, it will prompt that the input image is wrong and require re-shooting. Image preprocessing is to expand the length and width of the human eye area by 1.2 times according to the original proportion based on the eye area frame selection results, so that the final frame-selected eye area contains complete eye information such as the corners of the eyes. Using the expanded length as the benchmark, the parallel center line as the baseline, and the aspect ratio of 5:4, the images are cropped to ensure that all eye images meet the size requirements and maximize the effective area.
人眼分类器是一个经过大量人眼与非人眼图像样本预训练的深度学习模型,用于区分输入图像中的人眼和非人眼,对输入结果检测为人眼的图像进行眼部区域框选,对符合眼部区域范围要求的框选结果,继续进行后续的图像预处理,巩膜区域检测、眼红区域提取与眼红占比计算分析。对眼部区域框选结果不符合要求的图像,以及输入结果检测为非人眼的图像,算法终止并给出相应的提示信息。其中,预训练,是指在目标任务之前使用大规模数据集进行模型的初始训练,让模型学习到一些通用的特征和表示。眼部区域框选,是根据人眼判定结果,使用标注框将判定为人眼的区域进行框选,并返回框选区域的图像坐标和框选结果的长宽比。框选结果应当包含内外眼角等完整的眼部信息,而尽量减少输入图像中与人眼无关的其他部分信息。眼部区域范围,是要求框选出的眼部区域长宽比不能低于256×256,便于进行后续的巩膜区域检测和眼红区域提取等步骤。若框选的眼部区域长宽比不低于256×256,则输入的眼部图像符合要求,能够继续进行后续的步骤。否则提示人眼图像太小,要求重新拍摄。The human eye classifier is a deep learning model that has been pre-trained with a large number of human eye and non-human eye image samples. It is used to distinguish human eyes and non-human eyes in the input image, and perform eye region framing on images that are detected as human eyes in the input result. Select, and continue to perform subsequent image preprocessing on the frame selection results that meet the eye area range requirements, including scleral area detection, eye red area extraction, and eye red proportion calculation and analysis. For images whose eye area frame selection results do not meet the requirements, and for images whose input results are detected as non-human eyes, the algorithm terminates and gives corresponding prompt information. Among them, pre-training refers to using a large-scale data set to conduct initial training of the model before the target task, so that the model can learn some common features and representations. Eye area frame selection is based on the human eye determination result, using the label box to select the area determined to be the human eye, and returns the image coordinates of the frame selection area and the aspect ratio of the frame selection result. The frame selection result should contain complete eye information such as the inner and outer corners of the eyes, while minimizing other parts of the input image that are irrelevant to the human eye. The scope of the eye area requires that the aspect ratio of the eye area selected by the frame should not be less than 256×256, which facilitates subsequent steps such as scleral area detection and eye red area extraction. If the aspect ratio of the selected eye area is not less than 256×256, then the input eye image meets the requirements and you can continue with the subsequent steps. Otherwise, it will prompt that the human eye image is too small and require re-shooting.
步骤二,巩膜区域检测。Step two, sclera area detection.
将输入图像裁剪后获得的实际眼部图像缩放为512×512大小,进行预处理后使用经过改进的U-Net++模型进行巩膜区域检测。模型检测结果为待检图像巩膜区域的二值图,将二值图与缩放图进行与操作,得到巩膜区域的彩色图像。The actual eye image obtained after cropping the input image is scaled to a size of 512×512, and after preprocessing, the improved U-Net++ model is used for scleral area detection. The model detection result is a binary image of the sclera area of the image to be inspected. The binary image and the zoom image are ANDed to obtain a color image of the sclera area.
图像缩放,是对输入图像进行眼部区域裁剪与预处理后图片长度小于512×512的进行超分辨,长度大于512×512的进行下采样,减少因缩放带来的图像细节丢失。超分辨,是对实际分辨率低于要求分辨率的目标图像,通过插值方法将图像分辨率提高,从而提高图像的细节和质量。下采样,是对实际分辨率高于要求分辨率的目标图像中的一些像素进行平均化处理,从而降低图像的空间分辨率。Image scaling is to perform eye area cropping and preprocessing on the input image. After the image length is less than 512×512, super-resolution is performed. If the length is greater than 512×512, down-sampling is performed to reduce the loss of image details caused by scaling. Super-resolution is to increase the image resolution through interpolation method for target images whose actual resolution is lower than the required resolution, thereby improving the details and quality of the image. Downsampling is to average some pixels in the target image whose actual resolution is higher than the required resolution, thereby reducing the spatial resolution of the image.
图像预处理,是利用颜色阈值法,若判断图像整体颜色偏红,则提取图像RGB三通道中的的R通道,继续进行巩膜分割操作。否则使用直方图均衡化的方法进行图像增强后,仍然以RGB三通道彩色图进行巩膜分割操作。Image preprocessing uses the color threshold method. If the overall color of the image is judged to be reddish, the R channel of the three RGB channels of the image is extracted and the scleral segmentation operation is continued. Otherwise, after using the histogram equalization method for image enhancement, the RGB three-channel color image is still used for the scleral segmentation operation.
颜色阈值法,是通过计算图像中红色通道的平均值,判断是否需要对图像提取红色单通道后再进行巩膜分割。当颜色通道的平均值超过设定的阈值时,认为需要提取图像的红色单通道,否则不需要。The color threshold method calculates the average value of the red channel in the image to determine whether it is necessary to extract a single red channel from the image before performing scleral segmentation. When the average value of the color channels exceeds the set threshold, it is considered that the red single channel of the image needs to be extracted, otherwise it is not needed.
直方图均衡化,通过统计图像中每个像素值的出现频率,得到像素值的直方图,对直方图进行归一化后,计算直方图的累积分布函数(Cumulative Distribution Function,CDF),用于表示每个像素值的累积出现概率,根据CDF计算出每个像素值在新的直方图中的映射值,将原来的像素映射到新的像素值上。使用新的像素值替换原来的像素值,得到均衡化后的图像。累积分布函数CDF使用如下公式表示:Histogram equalization obtains a histogram of pixel values by counting the frequency of occurrence of each pixel value in the image. After normalizing the histogram, calculates the cumulative distribution function (CDF) of the histogram, which is used to Represents the cumulative occurrence probability of each pixel value, calculates the mapping value of each pixel value in the new histogram according to the CDF, and maps the original pixel to the new pixel value. Use new pixel values to replace the original pixel values to obtain the equalized image. The cumulative distribution function CDF is expressed using the following formula:
其中,x是像素值,P(i)是归一化后的图像中像素值为i的频率。Among them, x is the pixel value, and P(i) is the frequency of the pixel value i in the normalized image.
直方图归一化,是将直方图中的频率值进行标准化,使得频率值在0到1之间。此处采用的归一化方法是将每个频率值除以总像素数目,以确保归一化后的频率值之和为1。使用如下公式表示:Histogram normalization is to normalize the frequency values in the histogram so that the frequency values are between 0 and 1. The normalization method used here is to divide each frequency value by the total number of pixels to ensure that the normalized frequency values sum to 1. Expressed using the following formula:
其中,N(i)是归一化后的频率值,H(i)是原始直方图中像素值为i的频率值,Npixels是总像素数目。归一化后的频率值表示了像素值i出现的相对概率。Among them, N(i) is the normalized frequency value, H(i) is the frequency value with pixel value i in the original histogram, and N pixels is the total number of pixels. The normalized frequency value represents the relative probability of occurrence of pixel value i.
RGB三通道分别指图像的红(R)、绿(G)、蓝(B)三个颜色通道,图像每个像素由三个颜色通道的数值组成,分别表示红色、绿色和蓝色的强度或亮度,每个颜色通道的数值通常是介于0到255之间的整数,数值越高,该通道颜色分量越大,反应到该单通道上越亮。The three RGB channels refer to the three color channels of red (R), green (G), and blue (B) of the image. Each pixel of the image is composed of the values of the three color channels, which represent the intensity or intensity of red, green, and blue respectively. Brightness, the value of each color channel is usually an integer between 0 and 255. The higher the value, the greater the color component of the channel, and the brighter it is reflected on the single channel.
U-Net++模型是一个用于图像分割任务的深度学习模型,是对经典U-Net模型的扩展和改进,U-Net++通过引入一种递归的网络结构来增强模型的表达能力,并提高了图像分割的性能,在医学图像分割领域得到了广泛的应用。本发明中经过改进的U-Net++模型,主要是调整了原U-Net++网络中的输入输出部分、每一层的卷积操作,以及上下层之间的部分结构。输入输出部分,输入图像是经过预处理步骤后的图像,输出图像是单通道的二值图,输入输出图像形状都是512×512。在U-Net++的每一层卷积操作与ReLU激活函数操作之间,添加了一个归一化操作。在上层与下层之间,添加了dropout操作,并在之后添加注意力机制,随后进行池化操作。按上述操作进行改进之后,在测试数据上能得到2%~3%的准确率提升。The U-Net++ model is a deep learning model for image segmentation tasks. It is an expansion and improvement of the classic U-Net model. U-Net++ enhances the expressive ability of the model by introducing a recursive network structure and improves image quality. Segmentation performance has been widely used in the field of medical image segmentation. The improved U-Net++ model in the present invention mainly adjusts the input and output parts of the original U-Net++ network, the convolution operation of each layer, and some structures between the upper and lower layers. In the input and output part, the input image is the image after the preprocessing step, the output image is a single-channel binary image, and the input and output image shapes are both 512×512. A normalization operation is added between each layer of convolution operation and ReLU activation function operation of U-Net++. Between the upper layer and the lower layer, a dropout operation is added, and an attention mechanism is added later, followed by a pooling operation. After improving according to the above operations, the accuracy can be improved by 2% to 3% on the test data.
单通道二值图是指仅包含单个通道(灰度通道)的二值图像,每个像素点只有两个可能的取值:黑色和白色。通常使用灰度值0表示黑色像素,使用灰度值255(或1)表示白色像素。A single-channel binary image refers to a binary image that contains only a single channel (grayscale channel). Each pixel has only two possible values: black and white. Typically a grayscale value of 0 is used to represent a black pixel, and a grayscale value of 255 (or 1) is used to represent a white pixel.
ReLU激活函数全称是Rectified Linear Unit,中文名称是线性整流函数,是神经网络中常用的激活函数。通常意义下,其指代数学中的斜坡函数。The full name of ReLU activation function is Rectified Linear Unit, and its Chinese name is linear rectification function, which is a commonly used activation function in neural networks. In a common sense, it refers to the slope function in mathematics.
dropout是深度学习中被广泛应用于解决模型过拟合问题的策略,dropout解决了co-adaption问题,使得训练更宽的网络成为可能。Dropout is a strategy widely used in deep learning to solve the problem of model overfitting. Dropout solves the co-adaption problem and makes it possible to train wider networks.
co-adaption问题是指共适应问题,网络中的一些节点会比另外一些节点有更强的表征能力,随着网络的不断训练,具有更强表征能力的节点被不断的强化,而更弱的节点则不断弱化直到对网络的贡献可以忽略不计。这时候只有网络中的部分节点才会被训练,浪费了网络的宽度和深度,进而导致模型的效果上升受到限制。The co-adaption problem refers to a co-adaptation problem. Some nodes in the network will have stronger representation capabilities than other nodes. As the network continues to train, nodes with stronger representation capabilities are continuously strengthened, while weaker ones The nodes continue to weaken until their contribution to the network is negligible. At this time, only some nodes in the network will be trained, which wastes the width and depth of the network, thus limiting the effectiveness of the model.
注意力机制是人们在机器学习模型中嵌入的一种特殊结构,用来自动学习和计算输入数据对输出数据的贡献大小。通过引入注意力机制,神经网络能够自动地学习并选择性地关注输入中的重要信息,提高模型的性能和泛化能力。The attention mechanism is a special structure that people embed in machine learning models to automatically learn and calculate the contribution of input data to output data. By introducing the attention mechanism, the neural network can automatically learn and selectively focus on important information in the input, improving the performance and generalization ability of the model.
池化操作是模仿人的视觉系统对数据进行降维,主要是在一定的区域内提出该区域的关键信息。The pooling operation imitates the human visual system to reduce the dimensionality of the data, mainly to propose the key information of the area in a certain area.
模型检测是使用已经训练好的改进后的U-Net++模型对输入图像进行检测,输入图像需要先经过第一步骤中的人眼分类器的检测,裁剪出人眼图像后调整图像大小为512×512,作为模型的输入,模型能够准确地检测出输入图像中的巩膜区域,并给出检测图像结果的二值图。Model detection is to use the trained and improved U-Net++ model to detect the input image. The input image needs to be detected by the human eye classifier in the first step. After cropping the human eye image, adjust the image size to 512× 512. As the input of the model, the model can accurately detect the sclera area in the input image and give a binary map of the detection image result.
二值图与缩放图进行与操作,将二值图中像素值为0(黑色部分)的所有像素点,对应缩放图中的像素点值置为0,仅保留图像中巩膜区域,得到巩膜区域图。Perform the AND operation on the binary image and the zoom image, set all pixels with a pixel value of 0 (black part) in the binary image, and set the corresponding pixel values in the zoom image to 0, and only retain the sclera area in the image to obtain the sclera area. picture.
步骤三,眼红区域提取。Step 3: Extract the eye red area.
以上一步获得的巩膜区域彩色图像作为输入图像,对输入图像进行图像平滑处理与自适应直方图均衡化增强图像对比度,对处理之后的图像使用B-COSFIRE滤波器对血丝进行提取,同时对图像增强处理后的彩色图像获取掩膜图,彩色图像的RGB颜色模型转为LAB颜色模型,根据L(亮度)分量设定阈值重新生成合适的掩膜,将低亮度区域(即黑灰色的背景区域)的掩膜设为0,其余部分掩膜设为1,这样可以有效的避免黑色与彩色的交界处被算法认定是血丝。对处理之后的图像使用B-COSFIRE滤波器对血丝进行提取,提取到的结果图像为血丝二值图。The color image of the sclera area obtained in the previous step is used as the input image. Image smoothing and adaptive histogram equalization are performed on the input image to enhance the image contrast. The B-COSFIRE filter is used on the processed image to extract blood streaks and enhance the image at the same time. The mask image is obtained from the processed color image, and the RGB color model of the color image is converted into a LAB color model. A suitable mask is regenerated according to the threshold value set for the L (brightness) component, and the low-brightness area (i.e., the black-gray background area) is regenerated. The mask is set to 0, and the remaining masks are set to 1. This can effectively prevent the intersection between black and color from being recognized as bloodshot by the algorithm. Use the B-COSFIRE filter to extract blood filaments from the processed image, and the extracted result image is a binary image of blood filaments.
图像平滑处理是采用高斯滤波方法,对图像进行高斯核函数的卷积操作来实现平滑效果,去除图像中的高频噪声和平滑图像,并保持图像的整体细节和边缘。Image smoothing uses the Gaussian filtering method to perform the convolution operation of the Gaussian kernel function on the image to achieve a smoothing effect, remove high-frequency noise in the image and smooth the image, and maintain the overall details and edges of the image.
自适应直方图均衡化是将原始图像分割成均匀的小块或根据特定算法进行分割,对每个小块部分进行直方图均衡化,同时根据相邻小块的直方图分布情况,进行亮度修正,最终将所有小块重新组合成一幅增强后的图像。均衡化后的图像能够通过避免相对均匀区域中的噪声过度放大来改善局部对比度。Adaptive histogram equalization is to divide the original image into uniform small blocks or divide it according to a specific algorithm, perform histogram equalization on each small block, and perform brightness correction according to the histogram distribution of adjacent small blocks. , and finally recombine all the small patches into an enhanced image. Equalized images can improve local contrast by avoiding excessive amplification of noise in relatively uniform areas.
RGB颜色模型是一种用于描述彩色的模型,基于红(Red)、绿(Green)、蓝(Blue)三个基本颜色通道的组合,每隔颜色通道的值通常在0到255之间,0代表最小亮度,255代表最大亮度。The RGB color model is a model used to describe colors, based on a combination of three basic color channels: red, green, and blue. The value of each color channel is usually between 0 and 255. 0 represents the minimum brightness and 255 represents the maximum brightness.
LAB颜色模型是一种用于描述颜色的模型,由亮度(L)和颜色对立性(A和B)两个通道组成,A通道表示从绿色到红色的对立性,B通道表示从蓝色到黄色的对立性。The LAB color model is a model used to describe color, consisting of two channels: brightness (L) and color opposition (A and B). The A channel represents the opposition from green to red, and the B channel represents the opposition from blue to The opposite nature of yellow.
B-COSFIRE滤波器是条状选择(Bar-selective)移位滤波器响应组合(Combination Of Shifted FIlterREsponses)的简称。B-COSFIRE滤波器通过计算一组高斯差分滤波器输出的加权几何平均值来实现方向选择性,这些滤波器的支持区域按线性方式对齐。通过对两个旋转不变的B-COSFIRE滤波器的响应进行求和并进行阈值处理,可以实现对眼部血管的分割。B-COSFIRE filter is the abbreviation of Bar-selective shift filter response combination (Combination Of Shifted FIlterREsponses). B-COSFIRE filters achieve direction selectivity by computing a weighted geometric mean of the outputs of a set of Gaussian difference filters whose support regions are aligned in a linear fashion. Segmentation of ocular blood vessels is achieved by summing and thresholding the responses of two rotation-invariant B-COSFIRE filters.
旋转不变是指在处理图像时,算法或方法对于物体在不同旋转角度下具有相同的识别或分析结果。即无论物体如何旋转,算法都能够正确地识别或处理它,而不会受到旋转的影响。Rotation invariance means that when processing images, the algorithm or method has the same recognition or analysis results for objects at different rotation angles. That is, no matter how the object is rotated, the algorithm can correctly identify or process it without being affected by the rotation.
B-COSFIRE滤波器响应函数为:The B-COSFIRE filter response function is:
上式中,σ为决定响应范围的高斯函数标准差。In the above formula, σ is the standard deviation of the Gaussian function that determines the response range.
步骤四,眼红占比计算及眼红程度判断。Step 4: Calculate the proportion of jealousy and judge the degree of jealousy.
针对上述操作步骤,获取到的血丝二值图和巩膜区域图,分别计算血丝二值图和巩膜区域图中非零像素点个数,再进行比值计算,二者的比值即为眼红占比。将眼红占比计算结果乘以100后向下取整,使得最终结果位于[0,100]范围内。最后使用下述分数等级制,得出相应眼红程度级别,如表1。Based on the above operation steps, the obtained bloodshot binary image and scleral area map are calculated respectively. The number of non-zero pixels in the bloodshot binary image and scleral area map is calculated, and then the ratio is calculated. The ratio between the two is the eye red ratio. Multiply the calculation result of the jealousy ratio by 100 and round down, so that the final result is within the range of [0,100]. Finally, the following score rating system is used to obtain the corresponding jealousy level, as shown in Table 1.
其中,Degree表示眼红程度,Bpixels表示血丝二值图中非0像素点个数,Spixels表示巩膜区域图中非0像素点个数。Among them, Degree represents the degree of jealousy, B pixels represents the number of non-0 pixels in the bloodshot binary image, and S pixels represents the number of non-0 pixels in the sclera area map.
表1眼红程度等级Table 1 Degree of jealousy
本发明还公开一种无接触眼红程度自动分析装置,包括:The invention also discloses a non-contact automatic eye redness analysis device, which includes:
眼部图像获取与处理模块,用于使用可见光成像设备对准脸部正面拍摄人眼,对拍摄图像进行人眼区域检测及图像预处理,得到预处理图像;The eye image acquisition and processing module is used to use visible light imaging equipment to shoot human eyes from the front of the face, and perform human eye area detection and image preprocessing on the captured images to obtain preprocessed images;
巩膜区域检测模块,用于将预处理图像进行缩放后得到缩放图,对缩放图进行预处理后使用U-Net++模型对缩放图进行巩膜区域检测,得到待检图像巩膜区域的二值图,将二值图与缩放图进行与操作,得到巩膜区域彩色图;The scleral area detection module is used to scale the preprocessed image to obtain a zoomed image. After preprocessing the zoomed image, use the U-Net++ model to detect the scleral area of the zoomed image to obtain a binary image of the scleral area of the image to be inspected. Perform operations on the binary image and the zoom image to obtain a color image of the sclera area;
眼红区域提取模块,用于提取将巩膜区域彩色图进行图像平滑处理与自适应直方图均衡化来增强图像对比度,得到图像增强图,然后使用B-COSFIRE滤波器对图像增强图进行滤波处理,得到滤波处理图,同时将图像增强图转化为LAB颜色模型,获取得到图像掩膜,最后将滤波处理图和图像掩膜进行与操作,获取血丝二值图;The eye-red area extraction module is used to extract the color image of the sclera area, perform image smoothing and adaptive histogram equalization to enhance the image contrast, and obtain the image enhancement map, and then use the B-COSFIRE filter to filter the image enhancement map to obtain Filter the processing map, and at the same time convert the image enhancement map into a LAB color model to obtain the image mask. Finally, perform the AND operation on the filtering processing map and the image mask to obtain the bloodshot binary image;
眼红占比计算模块,用于分别计算巩膜区域彩色图和血丝二值图中非零像素点个数,再对血丝二值图非零像素点个数和巩膜区域彩色图非零像素点个数进行比值计算,得到眼红占比。The eye red proportion calculation module is used to calculate the number of non-zero pixels in the sclera area color image and the bloodshot binary image respectively, and then calculate the number of non-zero pixels in the bloodshot binary image and the sclera area color image. Perform a ratio calculation to get the jealousy ratio.
一种计算机可读存储介质,其上存储有计算机程序,该计算机程序被处理器执行时,实现本发明接触眼红程度自动分析方法。A computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method for automatic analysis of the degree of contact envy of the present invention is implemented.
以上所述仅是本发明的优选实施方式,应当指出:对于本技术领域的普通技术人员来说,在不脱离本发明原理的前提下,还可以做出若干改进和润饰,这些改进和润饰也应视为本发明的保护范围。The above are only the preferred embodiments of the present invention. It should be pointed out that those of ordinary skill in the art can make several improvements and modifications without departing from the principles of the present invention. These improvements and modifications can also be made. should be regarded as the protection scope of the present invention.
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311115420.XA CN117197064A (en) | 2023-08-30 | 2023-08-30 | Automatic non-contact eye red degree analysis method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311115420.XA CN117197064A (en) | 2023-08-30 | 2023-08-30 | Automatic non-contact eye red degree analysis method |
Publications (1)
Publication Number | Publication Date |
---|---|
CN117197064A true CN117197064A (en) | 2023-12-08 |
Family
ID=89002774
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202311115420.XA Pending CN117197064A (en) | 2023-08-30 | 2023-08-30 | Automatic non-contact eye red degree analysis method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN117197064A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117649347A (en) * | 2024-01-30 | 2024-03-05 | 宁乡爱尔眼科医院有限公司 | Remote eye examination method and system based on ultra-wide-angle fundus imaging |
-
2023
- 2023-08-30 CN CN202311115420.XA patent/CN117197064A/en active Pending
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117649347A (en) * | 2024-01-30 | 2024-03-05 | 宁乡爱尔眼科医院有限公司 | Remote eye examination method and system based on ultra-wide-angle fundus imaging |
CN117649347B (en) * | 2024-01-30 | 2024-04-19 | 宁乡爱尔眼科医院有限公司 | Remote eye examination method and system based on ultra-wide-angle fundus imaging |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN108446617B (en) | A fast face detection method against profile interference | |
CN104050471B (en) | Natural scene character detection method and system | |
WO2019104767A1 (en) | Fabric defect detection method based on deep convolutional neural network and visual saliency | |
WO2018145470A1 (en) | Image detection method and device | |
WO2020007307A1 (en) | Sky filter method for panoramic images and portable terminal | |
CN111915704A (en) | Apple hierarchical identification method based on deep learning | |
CN107220624A (en) | A kind of method for detecting human face based on Adaboost algorithm | |
CN115331245B (en) | Table structure identification method based on image instance segmentation | |
CN103149214B (en) | Method for detecting flaw on surface of fruit | |
Wazalwar et al. | A design flow for robust license plate localization and recognition in complex scenes | |
WO2022088856A1 (en) | Fundus image recognition method and apparatus, and device | |
CN111611907A (en) | An infrared target detection method with image enhancement | |
CN114926407A (en) | Steel surface defect detection system based on deep learning | |
CN115063331B (en) | Ghost-free multi-exposure image fusion method based on multi-scale block LBP operator | |
CN113537037A (en) | Pavement disease identification method, system, electronic device and storage medium | |
CN113537211B (en) | A deep learning license plate frame location method based on asymmetric IOU | |
CN115082776A (en) | Electric energy meter automatic detection system and method based on image recognition | |
CN114445620A (en) | An Object Segmentation Method to Improve Mask R-CNN | |
CN111709305B (en) | A Face Age Recognition Method Based on Partial Image Blocks | |
CN117197064A (en) | Automatic non-contact eye red degree analysis method | |
CN116883868A (en) | UAV intelligent cruise detection method based on adaptive image defogging | |
CN116363064A (en) | Defect identification method and device integrating target detection model and image segmentation model | |
CN111046866A (en) | Method for detecting RMB crown word number region by combining CTPN and SVM | |
CN108491883A (en) | A kind of conspicuousness inspection optimization method based on condition random field | |
CN118392891A (en) | Chip packaging defect detection method and system based on machine vision |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |