Diaretdb0: Evaluation database and methodology for diabetic retinopathy algorithms

DIARETDB0: Evaluation Database and Methodology for Diabetic Retinopathy Algorithms Tomi Kauppi1 , Valentina Kalesnykiene2, Joni-Kristian Kamarainen1 , Lasse Lensu1 , Iiris Sorri2 , Hannu Uusitalo2, Heikki Kälviäinen1 and Juhani Pietilä3 1 Machine Vision and Pattern Recognition Research Group Lappeenranta University of Technology, Finland 2 Department of Ophthamology Faculty of Medicine, University of Kuopio, Finland 3 Perimetria Ltd., Helsinki, Finland Abstract For a particularly long time, automatic diagnosis of diabetic retinopathy from digital fundus images has been an active research topic in the medical image processing community. The research interest is justified by the excessive potential for new products in the medical industry and possible reductions in healthcare costs. However, the maturity of state-of-the-art algorithms cannot be judged due to the lack of commonly accepted and representative image database with a verified ground truth and strict evaluation protocol. In this study, problems and issues related to the database are discussed from medical, image processing, and security perspectives. Based on the discussion, an evaluation methodology is proposed and a prototype image database with the ground truth is described. The database is made publicly available for benchmarking diagnosis algorithms. By utilizing the proposed database, it is now possible compare different algorithms, and correspondingly, analyze their maturity for technology transfer from the research laboratories to the medical practice. 1 Introduction Diabetes has become one of the rapidly increasing health threats worldwide [19]. Only in Finland there are 30 000 people diagnosed to the type 1 maturity onset diabetes in the young, and 200 000 people diagnosed to the type 2 latent autoimmune diabetes in adults [1]. In addition, the current estimate predicts that there are 200 000 undiagnosed patients [1]. 1 Proper and early treatment of diabetes is cost effective since implications of poor or late treatment are very expensive. In Finland, only 10% of the total health care costs of diabetes arise from 70% of early diagnosed patients while the remaining 90% arise from the patients having poor treatment (30%) [2]. This fact promotes the study of automatic diagnosis methods for screening over larger populations. Fundus imaging has an important role in diabetes monitoring since occurrences of retinal abnormalities are common and consequences serious. However, since the eye fundus seems to be sensitive to vascular diseases, fundus imaging is considered as a candidate for non-invasive screening of diabetes. The success rate of screening depends on accurate fundus image capturing and especially on accurate and reliable image processing algorithms for detecting the abnormalities. Various algorithms have been proposed by many research groups for this purpose. However, it is impossible to judge the accuracy and reliability of the approaches because of the lack of commonly accepted and representative fundus image database and evaluation protocol. The commonly accepted protocol could evaluate the maturity of state-of-the-art methods, i.e., produce the achieved sensitivity and selectivity rates. This would finally allow the technology transfer from research laboratories to practice. The main contribution of this work is a framework for evaluating methods for an automatic diabetic retinopathy diagnosis. A prototype database with the ground truth and evaluation protocol are proposed. Experimental results for a method from the literature are reported. This study provides the means for the trustable evaluation of automatic diabetic retinopathy. The idea for this approach originates from the strict regulations for the valuation of biometric authentication method, e.g., the FERET protocol for face recognition methods [13]. 2 Diabetic retinopathy In the type 1 diabetes, the insulin production is permanently damaged, whereas in the type 2 diabetes, the person is suffering from increased resistance to insulin. The type 2 diabetes is a familial disease, but also related to limited physical activity and lifestyle [19]. The diabetes may cause abnormalities in the retina (diabetic retinopathy), kidneys (diabetic nefropathy), and nervous system (diabetic neuropathy) [11]. The diabetes is also a major risk factor in cardiovascular diseases [11]. The diabetic retinopathy is a complication of diabetes, causing abnormalities in the retina, and in the worst case, blindness. Typically there are no salient symptoms in the early stages of diabetes, but the number and severity predominantly increase during the time. The diabetic retinopathy typically begins as small changes in the retinal capillary. The first detectable abnormalities are mircroaneyrysms (Fig. 1(a)) which are local distensions of the retinal capillary and which cause intra-retinal haemorrhage (Fig. 1(b)) when ruptured [6]. In time, the retinal edema and hard exudates (Fig. 1(c)) are followed by the increased permeability of the capillary walls [6]. The hard exudates are lipid formations leaking from these weakened blood vessels. This state of the retinopathy is called non-proliferative diabetic retinopathy. However, if the abovementioned abnormalities appear in the central vision area (macula), it is called diabetic maculopathy [18]. As the retinopathy advances, the blood vessels become obstructed which causes microinfarcts in the retina. These microinfarcts are called soft exudates 2 (Fig. 1(d)). When a significant number of soft exudates (> 6), or intraretinal microvascular abnormalities are encountered, the state of the retinopathy is defined as preproliferative diabetic retinopathy. The preproliferative diabetic retinopathy can quickly turn into proliferative diabetic retinopathy when extensive lack of oxygen causes the development of new fragile vessels. This is called as neovascularization (Fig. 1(e)) which is a serious state threatening eye sight. The field of eye sight can be obstructed by a haemorrhage to the vitreous body which is a common cause of blindness for the type 1 diabetes. The neovascularization can tear retina, and when it is located near the center of macula, it can cause the loss of eye sight [18]. Due to the progressive nature of the retinopathy, regular monitoring is needed after diagnosis. However, broad screenings cannot be performed due to the fact that the fundus image examination requires attention of medical experts. For the screening, automatic image processing methods must be developed. (a) (b) (c) (d) (e) Figure 1: Abnormal findings in the eye fundus caused by the diabetic retinopathy: (a) microaneyryms (marked with an arrow), (b) haemorrhages, (c) hard exudates, (d) soft exudate (marked with an arrow), and (e) neovascularization. 3 2.1 Automatic methods As mentioned previously, the diagnosis of diabetic retinopathy can be divided into the following two categories: 1. Screening of the diabetic retinopathy 2. Monitoring of the diabetic retinopathy Most automatic systems approach the detection directly using shape, color, and domain knowledge of diabetic retinopathy findings, but the abnormalities can also be found indirectly by detecting changes between two fundus images taken from the same eye in different time moment [8, 14]. The direct approach contributes to screening of the disease, where indirect approach contributes to both screening and monitoring of the diabetic retinopathy. Both approaches use roughly the following stages for finding abnormalities in fundus images: 1) image enhancement 2) candidate diabetic retinopathy finding detection 3) classification to correct diabetic retinopathy category (or hypothesis rejection). Some of the main features distinguishing between the different findings and normal fundus parts are the color and brightness. The same features have been verified also by ophthalmologists. Unsurprisingly these features dominate in the automatic methods, and therefore will be shortly reviewed in our brief surveys for different type of findings in Section 2.1.1 and Section 2.1.2. Most of the automatic methods also detect normal fundus parts, such as optic disk, blood vessels, and macula. The automatic methods either use the vital domain information provided by the normal fundus parts or remove them due to their similar color and shape appearance with abnormal fundus findings. The detection of normal fundus parts is not considered in this study. 2.1.1 Microaneurysms and hemorrhages Image enhancement methods: Niemeijer et al. [10] estimated non-uniform background intensity of fundus image by applying median filtering to the green channel of the fundus image. Shade correction was generated by subtracting the result from the original green channel. Fleming et al. [3] had similar approach for microaneurysms, but the green channel of the original fundus image was divided with the background intensity image. In addition, the shade corrected image was normalized for global image contrast by dividing with its standard deviation. Multiple local contrast enhancement methods were tested to improve detection accuracy. In hemorrhage detection, Zhang and Chutape [22] used histogram specification applied to each individual RGB color component to normalize the colors between different fundus images. Sinthayothin et al. [16] used local contrast enhancement to equalize the intensity variation in fundus images. The fundus images were transformed from RGB color model to IHS color model and the local contrast enhancement was applied to the intensity component of the image. 4 Detection and classification methods: Niemeijer et al. [10] extracted the candidate finding areas by assigning posterior probability of being red finding for every pixel using Gaussian filter and its derivates as features for k-nearest neighbor clustering. Shape and intensity properties of the candidate areas were used for more accurate abnormal red finding and normal red finding classification. Fleming et al. [3] segmented candidate microaneurysm areas by applying region growing to image enhanced with morphological top-hat operation and thresholding. The result candidate areas were classified with k-nearest neighbor clustering using the shape and intensity information. Zhang and Chutape [22,23] used hemorrhage areas restricted by finite window in training images as input for support vector machine. To detect different sized hemorrhages a pyramid of images was generated by changing the resolution of fundus image. The local minima of the support vector machine provided evidence map were selected as hemorrhage locations. The principal component analysis was used to reduce the complexity of feature space. Sinthanayothin et al. [16] sharpened the edges of red finding regions by applying moat operator to green channel of the contrast enhanced image. From the result image, red findings were extracted with recursive region growing and thresholding. 2.1.2 Hard and soft exudates Image enhancement methods: Narasimha-iyer et al. [8] used normal retinal findings (vasculature, optic disk, fovea, and abnormal findings) to estimate the illumination component using iterative robust homographic surface fitting to compensate the nonuniform illumination in fundus images. In detection of bright diabetic retinopathy areas from fundus images, Zhang and Chutape [24] applied adaptive local contrast enhancement to sub-image areas using the local mean and standard deviation of intensities. The same approach was used by Osareh et al. [12] after color normalization between fundus images using histogram specification. Wang et al. [21] adjusted the image brightness using brightness transform function similar to gamma correction. Detection and classification methods: Hsu et al. [4] determined abnormal and normal finding areas using intensity properties for dynamic clustering. From the result abnormal areas, hard exudates were separated from soft exudates and drusen using intensity contrast information between abnormal areas and immediate background. The domain knowledge of retinal blood vessels were used to remove false artifacts. Walter et al. [20] eliminated the vessels by applying morphological closing to the luminance component of the fundus image. From the result, within a sliding window local standard variation image was calculated and thresholded into coarse exudate areas. More accurate countours were acquired by thresholding difference between original image and morphologically reconstructed image. Sànchez et al. [15] used yellowish color and sharp edges to distinguish hard exudates from the fundus images. The image pixels were classified into background and yellowish objects using minimum distance discrimination, where the countour pixels of extracted optic disk were used as background color reference and pixels inside the contour were used as yellowish object color reference. The segmented yellowish areas and 5 their edge information extracted with Kirsch’s mask were combined to hard exudate areas using boolean operator. Zhang and Chutape [24] located the bright abnormal regions in fundus images by applying fuzzy c-means clustering in LUV color space. The result areas were classified to hard exudates, soft exudates, and normal findings using support vector machine. Osareh et al. [12] searched the coarse hard exudate areas using fuzzy c-means clustering with Gaussian smoothed histograms of each color band of the fundus image. The segmented areas were classified to exudate and non-exudate regions using neural networks. Color, region size, mean and standard deviation of intensity, and edge strength were used as features. Li and Chutape [7] segmented exudates with combination of Canny edge detection and region growing in LUV color space. Gradient, mean pixel value, and seed pixel value were used as criteria in region growing. Niemeijer et al. [9] used a similar approach for bright abnormal region detection as they used for finding abnormal red regions in [10]. In addition to the previous work, the prior knowledge of optic disk and vascular arch were used to improve detection accuracy. Sinthanayothin et al. [16] clustered similar pixels using intensity difference as criteria for recursive region growing. The region with the most pixels were considered as background and defined the threshold value for hard exudate areas. Wang et al. [21] used spherical color coordinates as features for the classification of fundus image pixels to background and bright abnormal findings using minimum distance discriminant. The abnormal findings were verified using local-window-based method. 3 Evaluation database A necessary tool for the reliable evaluation and comparison of medical image processing algorithms is a database including a selected set of high-quality medical images which are representatives of the diabetic retinopathy and have been verified by experts. In addition to the images, also information about the medical findings must be available. This information of findings is called the ground truth. An accurate algorithm should take an image as input, and output a result or description which is consistent with the ground truth. In the evaluation, the consistency is measured and compared between the algorithms. In the following, we describe images and ground truth for DIARETDB0. 3.1 Fundus images The current database consists of 130 color fundus images of which 20 are normal and 110 contain signs of the diabetic retinopathy. The images were taken in the Kuopio university hospital. The images were dedicatedly selected, but their distribution does not correspond to any typical population, i.e., the data is biased – no a priori information can be devised from it. Images were captured with few 50 degree fieldof-view digital fundus cameras with unknown camera settings (flash intensity, shutter 6 speed, aperture, gain) (Fig. 2). The images contain an unknown amount of imaging noise and optical aberrations (dispersion, transverse and lateral chromatic, spherical, field curvature, coma, astigmatism, distortion), and unknown accuracy of photometric information (color or intensity). Variance over the visual appearance of different retinopathy findings can, thus, be considered as maximal. However, The data correspond to practical situations, and can be used to evaluate the general performance of diagnosis methods. The general performance corresponds to the situation where no calibration is performed (no correspondence to the real world measurements), but where images correspond to commonly used imaging conditions, i.e., the conditions typically encountered in hospitals. This data set is referred to as “calibration level 0 fundus images”. (a) (b) (c) Figure 2: Fundus images: (a) normal fundus, (b) abnormal fundus, and (c) abnormal fundus after treatment by photo-coagulation. 3.2 Ground truth The most important accuracy measures for medical diagnosis methods are sensitivity and specificity (see Sec. 4.2 for the definitions). Sensitivity and specificity are defined on the image basis – an image either contains a specific finding or not despite the fact that the diabetic retinopathy findings do have spatial locations in the fundus . For the computer vision researchers, it is important to ensure that the automatically extracted diabetic retinopathy findings also spatially correspond the findings marked by experts, that is, they appear at the same location in the image. Thus, the more detailed expert ground truth contains also the description of visual appearance of diabetic retinopathy findings. 7 For every fundus image there is a corresponding ground truth file. A ground truth file contains all finding types found in the specific image file. An example ground truth file contains e.g: redsmalldots hemorrhages hardexudates softexudates neovascularisation If a certain finding type is not found in the image, it is marked as not available (n/a): redsmalldots hemorrhages hardexudates n/a neovascularisation 3.2.1 Marking visual findings The image groundtrtuh is based on expert-selected findings related to the diabetic retinopathy and normal fundus structures (see Fig. 3). A person with a medical education (M.D.) and specialization to ophthalmology is considered as an expert. blood vessels fovea Optic disc (papilla) macula Figure 3: Structural elements of a normal fundus. A special software tool was provided for the experts to inspect the fundus images and annotate the findings. The user interface of the current software version is shown in Fig. 4. It should be noted that the workstation displays were not calibrated. Therefore, the diabetic retinopathy findings were not equally visible on all displays. However, the situation corresponds to the current best practices. The ground truth tool provided two graphical marking directives at the time of marking and the usage was not instructed for the medical experts. This freedom of choice was allowed to prevent a biased scheme, and the medical experts were allowed construct their own best practices to mark different findings. It was possible to use different combinations of the graphical directives for the same type of findings. Currently, The following graphical directives are: 8 Figure 4: Graphical tool for gathering knowledge from medical experts. 1. Centroid (Fig. 5(a)), 2. Polygon region (Fig. 5(b)), 3. Circle region (Fig. 5(c)), 4. Ellipse region (Fig. 5(c)), and 5. Representative point (Fig. 5(e)). In addition to the graphical directives, the image annotation tool provided a gamma correction tool and semi-automatic tool for more accurate definition of the finding areas (Fig. 5(e)). The semi-automatic tool used the color information provided by the representative point. In gathering the expert knowledge, the semi-automatic tool was not used. 9 Red_small_dots[1] Hard_exudates[3] (a) (b) Disc[4] Macula[5] (c) (d) (e) Figure 5: Graphical directives for marking the visual findings: (a) centroid; (b) polygon region; and (c) circle region; d) semi-automatic region cropping tool and representative point. 3.2.2 Data format The expert knowledge gathered with the groundtrtuh tool is stored to a text file. Each line in the text file corresponds to a visual finding marked with the groundtrtuh tool. The data format for visual finding is defined as CENTROID [POLYGON_REGION] [CIRCLE/ELLIPSE_REGION] RP_POINT REGION_TOOL ... CONFIDENCE_LEVEL FINDING_TYPE Each marking in image is defined by the centroid and one other graphical directive. The same graphical directive should be consistently used for every medical finding type over the image set. The directive attributes are visualized in Fig. 6 and defined as follows: CENTROID POLYGON_REGION CIRCLE_REGION ELLIPSE REGION RP_POINT REGION_TOOL CONFIDENCE_LEVEL FINDING_TYPE X Y [X_1 X_2 X_3 ...] [Y_1 Y_2 Y_3 ...] [RADIUS] [RADIUS1] [RADIUS2] [ANGLE] (X,Y) [METHOD,VALUE] HIGH, MEDIUM, LOW RED_SMALL_DOTS, HARD_EXUDATES ... where CONFIDENCE and FINDING TYPE values are preset by the experts and stored, but which the user can define. An example file produced by the groundtrtuh tool based on the findings in Fig. 5 is as follows: 233 354 543 899 945 461 104 348 [242,251,245,213,206,] [] [] [] [921,949,971,967,938,] [] [] [] [] [102] [41] [] 10 [] [94] [] [] [] [117] [] [] (231,944) (413,466) (543,104) (899,348) [HS,25] [HS,65] [HS,39] [,] High High High Medium Haemorrhages Disc Soft exudates Red small dots (X_7, Y_7) (X_6, Y_6) (X_8, Y_8) (X_5, Y_5) (X_4, Y_4) (X_9, Y_9) (X_3, Y_3) (X_10, Y_10) (X, Y) (X_2, Y_2) (X_11, Y_11) (X_1, Y_1) (X, Y) (X_12, Y_12) (a) (b) RADIUS1 RADIUS ANGLE (X, Y) (X, Y) RADIUS2 (c) (d) Figure 6: Graphical directives for marking visual findings: (a) a centroid defined by the coordinate pair; (b) a polygon defined by the point coordinates and centroid; (c) a circle defined by the radius and centroid; (d) an ellipse defined by the orientation(angle) and length of major-axis(radius1) and minor-axis(radius2) with centroid. different 3.3 Verifying image quality Representative quality of the database images varies due to the uncalibrated imaging environment. The vaguely defined practices in imaging and challenging imaging geometry of the fundus cause partial over and under exposures in digital images which can still be used for the diagnosis. A method for evaluating the partial image quality was proposed by Kuivalainen [5]. The method results a binary mask (Fig. 7 ), which represents whether a certain location is applicable for the analysis or not. DIARETDB0 includes a binary mask for every fundus image. The mask areas define where the distortion level caused by the imaging is sufficiently small, and only acceptable areas are used in the evaluation. 3.4 Security The patient information from the database images were removed by converting the images into raw data. The raw data images were converted to lossless portable network images and marked to belong DIARETDB0 by redefining the comment field to ”DIARETDB0 image filename 1500x1152 DirectClassRGB”. 11 (a) (b) (c) Figure 7: Image area with sufficient image quality: (a) orginal image; (b) binary mask; and (c) area used by the algorithms. 3.5 Training and test sets The set of 130 images was divided into 5 image categories, and a fixed number of randomly selected images were taken from each category to form the training set. The rest of the images compose the test set. The image categories were formed to confirm that each diabetic retinopathy finding type is included in the both training and test sets. The diabetic retinopathy finding types that each image group contains are the following: 1. Red Small dots, haemorrhages, hard exudates. 2. Red Small dots, haemorrhages, hard exudates, soft exudates. 3. Red Small dots, haemorrhages, hard exudates, soft exudates, neovascularisation. 4. Red small dots, haemorrhages, soft exudates, neovascularisation. 5. Normal. The categories represent the typical progress of the diabetic retinopathy [17]. 12 4 Evaluation methodology for diabetic retinopathy The proposed database and evaluation protocol are publicly available for all research groups who wish to evaluate their automatic diagnosis methods. By using the protocol and images defined in this document, the evaluation results are comparable between different studies. 4.1 Retrieving data The essential documentation and resources of DIARETDB0 diabetic retinopathy image database are available via the web-page at http://www.it.lut.fi/project/ imageret/diaretdb0/. The data and documents are arranged as shown in Table 1 Table 1: Structure of DIARETDB0 files. Documentation for DIARETDB0 resources/ All the needed resources for DIARETDB0 resources/evalscript/ MATLAB Evaluation script Example ouput files for evaluation resources/example evalresults/ resources/html/ Html files for the diaretdb0.htm resources/html/html images/ Thumbnail images for diaretdb0.htm resources/traindatasets/ Predefined training set files resources/testdatasets/ Predefined test set files resources/images/ DIARETDB0 images and ground truth data DIARETDB0 ground truth for color resources/images/diaretdb0 groundtruths/ fundus images DIARETDB0 binary masks for color resources/images/diaretdb0 binary masks/ fundus images resources/images/diaretdb0 fundus images/ DIARETDB0 color fundus images resources/images/example images/ Example images with diabetic retinopathy findings doc/ 4.2 Evaluation protocol Methods used for automatic detection of diabetic retinopathy are evaluated by using sensitivity and specificity per image basis. Sensitivity is the percentage of abnormal funduses classified as abnormal by the screening method and specificity is the percentage of normal fundus classified as normal by the screening method. The higher the sensitivity and specificity values, the better the method. Sensitivity and specificity values are calculated for three diabetic retinopathy finding classes: exudates (soft and hard), haemorrhages and red small dots(microaneurysm). In the current database the neovascularisation is not included due to its rarity. Sensitivity and specificity can be computed as [20]: 13 sensitivity = TP TP + FN (1) specificity = TN TN + FP (2) where TP is the number of abnormal fundus images found as abnormal, T N is the number of normal fundus images found as normal, F P is the number of normal fundus images found as abnormal (false positives) and F N is the number of abnormal fundus images found as normal (false negatives). For every image the area is given where the representative image quality is acceptable, and only the findings from those areas should be used. The results should be stored to a text file using the same structure as was defined for the ground truth files in Sec. 3.2. The text files should be named as follows: image filename.res, e.g., Image001.res The provided evaluation script reads the result text files and calculates the sensitivity and specificity values over all images by comparing the results to the ground truth files. 5 Example on algorithm evaluation The method proposed by Kuivalainen was evaluated using the database and the evaluation methodology [5]. It turned out, however, that the method could not be evaluated by the proposed evaluation protocol since the method assumed that the optic disk, papilla and blood vessels were perfectly detected and the findings from those areas were manually removed - an assumption which is very difficult to perform in practice, but which of ten appears in the studies. This is an example of “hidden” assumptions which actually prevent the true evaluation between the methods. Furthermore, it was only possible to evaluate the exudate detection which includes hard and sof exudates. The evaluation result of the exudates is given in Table 2. Table 2: Evaluation result for the method in [5] . Exudates Sensitivity % 79 Spesificity % 58 Table 3 contains the evaluation results for the case where the diabetic retinopathy findings located near the blood vessels were manually removed. 14 Table 3: Sensitivity and specificity achieved by the method. in [5] Red small dots Haemorrhages Exudates Sensitivity % 73 92 77 Specificity % 70 75 50 6 Discussion and future research The development of image processing methods to a mature level where the results can be transferred from the research laboratories to practice requires the following: accepted and applied protocols for evaluating the methods, protocols that are similar to the strict regulations in the medical treatment, and medicine research. Medical image processing is not different from the medical practice in that sense. We proposed the first step for a standardized evaluation of methods for detecting findings of diabetic retinopathy. DIARETDB0 is in many ways a difficult database, but it corresponds to the situation in practice: the images are uncalibrated, expert evaluation is free form and the displays used to view the images are uncalibrated. In the future, however, we will continue to develop the database and evaluation methodology. The following development steps will be taken: 1. The fundus camera and optics are calibrated due to deficiencies of imaging (as the results, optical distortions are known and photometric information is the same between images). Calibration level 1 achieved. 2. A predefined set of directives for different kinds of findings is provided to the experts. The directives prevent the free form description, and thus, allow control over subjective interpretations. 3. Findings are classified based on the confidence level (high, medium, low) given by the expert. All findings are independently verified by several experts. 4. The effect of display calibration for the experts will be evaluated. 5. Sensitivity and specificity measures will be improved (sensitivity/specificity function). 6. Location of normal findings will be added to the data and a protocol for evaluating also their localization accuracy 7 Conclusions An image database, ground truth and evaluation methodology was proposed for evaluating and comparing methods for automatic detection of diabetic retinopathy. The DIARETDB0 provides a unified framework for benchmarking the methods, but also points out clear deficiencies in the current practice in the method development. The work will continue and the research group’s main objective is to publish an ultimate 15 tool for the evaluation of diabetic retinopathy detection methods to provide accurate and reliable information of their performance, and maturity for technology transfer from the research laboratories to practice and industry. References [1] Finnish Diabetes Association. Development programme for the prevention and care of diabetes, 2001. ISBN 952 5301-13-3. [2] Finnish Diabetes Association. Programme for the prevention of type 2 diabetes in Finland, 2003. ISBN 952-5301-36-2. [3] Alan D. Fleming, Sam Philip, Keith A. Goatman, John A. Olson, and Peter F. Sharp. Automated microaneurysm detection using local contrast normalization and local vessel detection. IEEE Transactions in Medical Imaging, 25(9):1223– 1232, September 2006. [4] Wynne Hsu, P. M. D. S Pallawa, M. Li Lee, and Au Eong K.-G. The role of domain knowledege in the detection of retinal hard exudates. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 2, pages 246–251, Kauai, Hi, USA, December 2001. [5] M. Kuivalainen. Retinal image analysis using machine vision. Master’s thesis, Lappeenranta University of technology, 2005, Finland. [6] L. Laatikainen. Diabeettinen retinopatia, kuvasto silm änpohjal öydösten tutkintaan. Finnish Diabetes Association, 2002. ISBN 952-5301-28-1. [7] Huiqi Li and Opas Chutape. Automated feature extraction in color retinmal images by a model based approach. IEEE Transactions on Biomedical Engineering, 51(2):246–254, February 2004. [8] Harihar Narasimha-Iyer, Ali Can, Bandrinath Roysam, Charles V. Stewart, Howard L.Tanenbau, Anna Majerovics, and Hanumant Singh. Robust detection and classfication of longitudinal changes in color retinalfundus images for monitoring diabetic retinopathy. IEEE Transactions on Biomedical Engineering, 53(6):1084–1098, June 2006. [9] M. Niemeijer, M. D Abràmoff, and B. van Ginneken. Automatic detection of the presence of bright lesions in color fundus photgraphs. In Proceedings of IFMBE the 3rd European Medical and Biological Engineering Confrence, volume 11 of 1, pages 1823–2839, Prague and Czech Republic, November 2005. [10] M. Niemeijer, B. van Ginneken, J. Staal, M. S. A. Suttorp-Schulten, and N. D. Abramoff. Automatic detection of red lesion in digital color fundus photographs. IEEE Transactions on Medical Imaging, 24(5):584–592, May 2005. [11] M. Niemi and K. Winell. Diabetes in Finland, prevalence and variation in quality of care. Kirjapaino Hermes Oy, Tampre, Finland, 2006. [12] Alireza Osareh, Majid Mirmehdi, Barry Thomas, and Richard Markham. Classification and localization of diabetic-related eye disease. In Proc. of 7th European Conference on Computer Vision (ECCV), pages 502–516, 2002. 16 [13] P.J. Phillips, H. Moon, S.A. Rizvi, and P.J. Rauss. The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(10), 2000. [14] Richard J. Radke, Srinivas Andra, Omar Al-Kofahi, and Badrinath Roysam. Image change detection algorithms: a systematic survey. IEEE Transactions on Image Processing, 14(3):294–307, March 2005. [15] C. I. Sànchez, R. Hornero, M. I .Lòpez, and J. Poza. Retinal image analysis to detect and quantify lesions associated with diabetic retinopathy. In Proceedings of the 26th Annual International Confrence of the IEEE Engineering in Medicine and Biology Society (EMBS), pages 1624–1627, San Francisco, CA, USA, September 2004. [16] C. Sinthayothin, J. F. Boyce, T. H. Williamson, E. Mensah, S.Lal, and D. Usher. Automated detection of diabetic retinopathy on digital fundus images. Diabetic Medicine, 19:105–112, 2002. [17] P. Summanen, V. Kallioniemi, J. Komulainen, H. Lamminen, H.Levänen, E. Orhanen, S. Tulokas, T. Virtamo, and G. von Wendt. Diabeettinen retinopatia. Duodecim, 11:1387–1400, 2006. [18] Suomen diabetesliiton retinopatia ryhmä. Diabeettinen retinopatia seuranta, hoito ja näkövammautuneen diabeetikon kuntoutus. Diabetes-lehden l ääkäriliite, 1992. [19] Gunvor von Wendt. Screening for diabetic retinopathy: Aspects of photographic methods. PhD thesis, Karolinska Institutet, 2005. [20] T. Walter, J.-C. Klein, P. Massin, and A. Erginay. A contribution of image processing to the diagnosis of diabetic retinopathy - detection of exudates in color fundus images of the human retina. IEEE Transactions on Medical Imaging, 21:1236–1243, October 2002. [21] Huan Wang, Wynne Hsu, Kheng Guan Goh, and Mong Li Lee. An effective approach to detect lesions in color retinal images. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 181–186, 2000. [22] Xiaohoui Zhang and Opas Chutape. A SVM approach for detection of hemorrhages in background diabetic retinopathy. In Proceedings of International Joint Confrence on Neural Networks, pages 2435–2440, Montreal and Canada, July 2005. [23] Xiaohoui Zhang and Opas Chutape. Top-down and bottom-up strategies in lesion deection of background diabetic retinpathy. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), volume 2, pages 422–428, San diego, CA, USA, July 2005. [24] Xiaohui Zhang and O. Chutatape. Detection and classification of bright lesions in colour fundus images. In Proceedings of IEEE Internationl Confrence on Image Processing (ICIP), volume 1, pages 139–142, October 2004. 17