JP2010035756A - Diagnosis support apparatus and diagnosis support method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent the oversight of an area requiring special attention and to reduce the mental load of a doctor who is a diagnosing person even in the diagnosis over a long period of time of a medical diagnostic image imaged by a capsule endoscope or the like. <P>SOLUTION: A diagnostic image is divided into the prescribed number of areas and an image feature amount for each area is calculated. Also, when the diagnosing person observes the diagnostic image, the observation time in each area of the diagnostic image is detected using line-of-sight detection. From the relation of the observation time and image feature amount of each area, the threshold Th_p of the image feature amount is calculated. From all the divided areas of the entire diagnostic image, the area wherein the image feature amount is equal to or more than the threshold Th_p and observation time is shorter than a threshold Th_k is extracted, and an image number including the area and an area number are notified to an HMD 10. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a diagnosis support apparatus and a diagnosis support method, and more particularly, to a diagnosis support apparatus and a diagnosis support method for preventing oversight of a caution area of a medical diagnosis image taken with a capsule endoscope or the like.

  A capsule endoscope is known in which a battery, a photographing lens, an image sensor, an illumination light source, and the like are housed inside a tablet capsule-shaped housing. The capsule endoscope is inserted into a body cavity when the subject swallows the capsule endoscope, and images the affected part and the like at a frame rate of, for example, twice per second. The captured image data is transmitted outside the body by the transmission means in the capsule endoscope, and is received by the reception device outside the body.

  Japanese Patent Application Laid-Open No. 2004-228561 describes a technique for detecting the movement of a capsule in a human body using an acceleration sensor, a pressure sensor, or the like and changing a photographing frame rate or a display frame rate. According to this technology, when the capsule endoscope is moving slowly and an excessive number of frames are the same, the display rate is accelerated by eliminating one or more identical frames. Can do.

Further, in Patent Document 2, when a viewer determining unit that determines whether or not a viewer is watching TV determines that the viewer is not watching TV, the image recording unit receives the TV. The technology to record the video is recorded. According to this technology, even if the viewer misses the video displayed on the display screen of the television, the missed video can be viewed later.
JP-T-2004-521626 JP 2007-318431 A

  However, although the technique of Patent Document 1 can reduce the observation time, there is a problem that a psychological burden is particularly great for a doctor who makes a diagnosis because all images must not be overlooked. Further, the technique of Patent Document 2 has a drawback that it can only determine whether or not the entire screen is viewed, and cannot determine the observation status of each part in the screen.

  The present invention has been made in view of such circumstances, and it is an object of the present invention to provide a diagnosis support apparatus and a diagnosis support method for preventing oversight of a caution area of a medical diagnosis image taken with a capsule endoscope or the like. To do.

  In order to achieve the object, the diagnosis support apparatus according to claim 1 is a diagnosis support apparatus for a diagnosis person to observe and diagnose a plurality of diagnosis images continuously reproduced and displayed on a display unit. Extraction means for extracting a region requiring attention from an image for diagnosis, and detecting the gaze of the diagnostician when the diagnostician observes the diagnostic image reproduced and displayed on the display means, and based on the detected gaze Means for calculating an observation time for each of the divided areas of the diagnostic image, and whether or not there is an oversight of the attention area based on the extracted attention area and the calculated observation time for each area. And a discriminating means for discriminating whether or not.

  According to the present invention, the attention area is extracted from the diagnostic image, and the observation time for each divided area of the diagnostic image is calculated based on the line of sight when the diagnostician observes the diagnostic image, and the extracted attention area Based on the calculated observation time of each area, it is determined whether or not there is an oversight of the area requiring attention, so even in the diagnosis over a long time of medical diagnostic images taken with a capsule endoscope etc. The oversight of the area requiring attention can be prevented, and the psychological burden on the doctor who is the diagnostician can be reduced.

  According to a second aspect of the present invention, in the diagnosis support apparatus according to the first aspect, the extraction unit includes a feature amount calculation unit that calculates an image feature amount of the diagnostic image, and the extraction unit includes the feature amount calculation unit based on the calculated image feature amount It is characterized by extracting a region requiring attention.

  Thereby, it is possible to appropriately extract a region requiring attention.

  According to a third aspect of the present invention, in the diagnosis support apparatus according to the second aspect, the feature amount calculating unit calculates an image feature amount for each of the divided regions.

  Thereby, it is possible to appropriately extract a region requiring attention.

  According to a fourth aspect of the present invention, in the diagnosis support apparatus according to the third aspect, the extraction unit extracts, as the attention area, an area in which the image feature amount is equal to or more than a first threshold among the plurality of divided areas. It is characterized by that.

  Thereby, it is possible to appropriately extract a region requiring attention.

  As described in claim 5, in the diagnosis support apparatus according to claim 4, the determination means includes an oversight of the area requiring attention in an area where the observation time is less than a second threshold among the areas requiring attention. It is characterized by distinguishing.

  Thereby, it can be determined that there is an oversight of the area requiring attention.

  As described in claim 6, in the diagnosis support apparatus according to claim 4 or 5, the first threshold value is set based on a characteristic indicating a relationship between an image feature amount and an observation time for each of the divided areas. It is characterized by having a determination means for determining.

  Thereby, it can be determined that there is an oversight of the area requiring attention.

  The diagnosis support apparatus according to claim 6, wherein the determination unit includes an approximate curve or an approximate line from a data group including a combination of an image feature amount and an observation time for each of the divided areas. And the first threshold value is determined from the relationship between the preset second threshold value and the generated approximate curve or approximate line.

  Thereby, a 1st threshold value can be determined appropriately.

  According to an eighth aspect of the present invention, in the diagnosis support apparatus according to the sixth or seventh aspect, the determining unit determines the first threshold value for each body part taken by the diagnostic image.

  Thereby, the suitable 1st threshold value according to the body region which the diagnostic image image | photographed can be determined.

  The diagnosis support apparatus according to any one of claims 6 to 8, wherein the feature amount calculation unit calculates a plurality of image feature amounts, and the determination unit includes the plurality of image feature amounts. The first threshold value is determined every time, and the extracting unit sets a region in which at least one of the plurality of image feature amounts is equal to or more than the first threshold value among the plurality of divided regions as a region requiring attention. It is characterized by extracting.

  Thereby, it is possible to appropriately extract a region requiring attention.

  According to a tenth aspect of the present invention, in the diagnosis support apparatus according to any one of the first to ninth aspects, a notification means for notifying a determination result of the determination means is provided.

  Thereby, the diagnostician can know the oversight of the area requiring attention.

  11. The diagnosis support apparatus according to claim 10, wherein the notifying unit displays information indicating the attention area and information indicating a diagnostic image from which the attention area is extracted on the display section. And is notified.

  Thus, the diagnostician can know the missed attention area and the diagnostic image from which the attention area has been extracted.

  The diagnosis support apparatus according to any one of claims 1 to 11, further comprising means for synthesizing auxiliary information indicating the extracted attention area in the diagnosis image. .

  Thereby, the diagnostician can know the area requiring attention at the time of diagnosis.

  The diagnosis support apparatus according to any one of claims 1 to 12, further comprising changing means for changing a time for reproducing and displaying the diagnostic image from which the attention area is extracted. To do.

  Thereby, the diagnostician can observe in detail the diagnostic image from which the attention area is extracted.

  According to a fourteenth aspect of the present invention, in the diagnosis support apparatus according to the thirteenth aspect of the invention, the changing unit increases the time for reproducing and displaying the diagnostic image as the area of the attention area increases.

  Thereby, the diagnostician can observe in detail the diagnostic image with the larger area of the area requiring attention.

  The diagnosis support apparatus according to any one of claims 1 to 14, wherein the image feature amount includes an average by density / color, a histogram, a frequency distribution, an edge intensity distribution by edge direction, It is one of the fractal dimensions.

  Thereby, it is possible to appropriately extract a region requiring attention.

  The diagnosis support apparatus according to any one of claims 1 to 15, wherein the display means is a head mounted display provided with a camera for detecting a line of sight of the diagnostician, or the diagnosis. It is a display in which a camera for detecting a person's line of sight is provided in close proximity.

  As a result, it is possible to appropriately display a diagnostic image and further detect the line of sight of the diagnostician.

  As shown in Claim 17, in the diagnosis support apparatus according to any one of Claims 1 to 16, the diagnosis image is an image photographed by a capsule endoscope.

  Thereby, it can apply to many images image | photographed with the capsule endoscope.

  In order to achieve the above object, the diagnosis support method according to claim 18, wherein the diagnosis person observes and diagnoses a plurality of diagnosis images continuously reproduced and displayed on the display means. Extracting a region requiring attention from an image for diagnosis, and detecting the gaze of the diagnostician when the diagnostician observes the diagnostic image reproduced and displayed on the display means, and based on the detected gaze Whether or not there is an oversight of the attention area based on the step of calculating the observation time for each of the divided regions of the diagnostic image, and the extracted attention area and the calculated observation time for each area And a step of discriminating between.

  Thereby, an oversight of the area requiring attention can be prevented, and the psychological load of the doctor who is the diagnostician can be reduced.

  According to the present invention, the attention area is extracted from the diagnostic image, and the observation time for each divided area of the diagnostic image is calculated based on the line of sight when the diagnostician observes the diagnostic image, and the extracted attention area Based on the calculated observation time of each area, it is determined whether or not there is an oversight of the area requiring attention, so even in the diagnosis over a long time of medical diagnostic images taken with a capsule endoscope etc. The oversight of the area requiring attention can be prevented, and the psychological burden on the doctor who is the diagnostician can be reduced.

  The best mode for carrying out the present invention will be described below.

<First Embodiment>
FIG. 1 is a schematic diagram showing an embodiment of a diagnosis support system according to the present invention, and shows a diagnosis support system applied when a diagnosis is performed by reproducing and displaying an image taken by capsule endoscopy. Yes. As shown in FIG. 1, the diagnosis support system includes a head mounted display (HMD) 10 and a server 20.

  The server 20 stores a large number of images taken by capsule endoscopy. In the capsule endoscopy, about 50000 endoscopic images are taken per examination. The server 20 reads out these diagnostic images in time series and transmits them to the HMD 10.

  The HMD 10 that has received the diagnostic image transmitted from the server 20 displays the received diagnostic image on the display unit 11 (described later). As shown in FIG. 1, the HMD 10 is attached to the head of the diagnostician, and the diagnostician wearing the HMD 10 makes a diagnosis by observing a diagnostic image displayed on the display unit 11.

  FIG. 2 is a block diagram showing an embodiment of the diagnosis support system. As shown in FIG. 2, the HMD 10 includes a display unit 11, a CPU 12, a line-of-sight detection unit 13, a communication unit 14, and an antenna 15.

  The CPU 12 performs overall control of each unit of the HMD 10 and converts data for transmission / reception into a predetermined format.

  The display unit 11 is a publicly known device that can see what is actually in front of the user through a half mirror, for example, and can view a display screen such as a liquid crystal monitor. The image signal transmitted from the server 20 and received by the communication unit 14 via the antenna 15 is subjected to predetermined processing in the CPU 12 and displayed on the display unit 11.

  The line-of-sight detection unit 13 includes, for example, a head-mounted small camera that shoots an eyeball from the front via a half mirror, and tracks the position of the pupil of the eyeball of the diagnostician wearing the HMD 10, thereby enabling the diagnostician Are sequentially detected. Therefore, with respect to the image displayed on the display unit 11, it is possible to determine to which part the diagnostician has directed the line of sight. Information on the line-of-sight direction of the diagnostician detected by the line-of-sight detection unit 13 is transmitted to the server 20 via the communication unit 14 in real time.

  In addition, the server 20 includes a CPU 21 that controls the operation of each component, a memory 22 that is a work area during program execution, an image processing unit 23 that performs area division of a diagnostic image, which will be described later, a processing program according to the present invention, and the like. It comprises a hard disk 24 for storing diagnostic images and storing it, a communication unit 25, and an antenna 26.

  In the hard disk 24, a large number of diagnostic images photographed by capsule endoscopy are recorded for each patient ID. The diagnostic images recorded on the hard disk 24 are read in the order of photographing based on the control of the CPU 21 and transmitted from the communication unit 25 to the HMD 10 via the antenna 26. Further, the communication unit 25 receives the line-of-sight information of the diagnostician transmitted from the HMD 10 via the antenna 26.

  The image processing unit 23 divides the diagnostic image into a plurality of areas, and calculates an image feature amount for each divided area. Details of the image feature amount will be described later.

  Note that although the example in which the HMD 10 and the server 20 communicate wirelessly has been described, they may communicate by wire.

  Next, the operation of the diagnosis support system will be described with reference to FIG.

  A diagnostician wears the HMD 10, designates a patient ID using an operation unit (not shown), and selects an endoscopic image to be diagnosed. The operation unit of the server 20 may be operated for selection. The server 20 sequentially reads the selected diagnostic images from the hard disk 24 in time series, and transmits them to the HMD 10 by the communication unit 25 (step S1).

  The HMD 10 receives the diagnostic images transmitted from the server 20 in order by the communication unit 14, and reproduces and displays them on the display unit 11 (step S11).

  The diagnostician observes the diagnostic image displayed on the display unit 11. At this time, the line-of-sight detection unit 13 detects the line of sight of the diagnostician (step S12). Based on the line of sight detected by the line-of-sight detection unit 13 and the positional relationship between the display unit 11 and the line-of-sight detection unit 13, the CPU 12 coordinates the viewpoint of the display unit 11, that is, the observation position of the image displayed on the display unit 11. Can be calculated.

  The viewpoint information calculated by the CPU 12 is transmitted to the server 20 via the communication unit 14 (step S13). The HDM 10 repeats the processing from step S11 to step S13 each time a diagnostic image is transmitted from the server 20.

  The server 20 that has received the viewpoint information divides the image being displayed on the HMD 10 by the image processing unit 23 into a plurality of areas, and based on the received viewpoint information, the observation time for each area in the image (the presence time of the line of sight) Is calculated (step S2).

  Further, the image processing unit 23 calculates an image feature amount for each divided area (step S3). Examples of the image feature amount include an average by density / color, a histogram, a frequency distribution, an edge intensity distribution by edge direction, and a fractal dimension. Here, the fractal dimension is used. The fractal dimension is a quantification of the level of complexity of a figure. The smaller the fractal dimension, the lower the level of complexity of the figure, and the larger the fractal dimension, the higher the level of complexity of the figure. . Therefore, for example, the complexity of blood vessels can be expressed. The image processing unit 23 calculates a fractal dimension for each divided area by a known calculation method.

  The thus calculated observation time and image feature amount for every divided area of all diagnostic images reproduced and displayed on the HMD 10 are recorded in the hard disk 24 in association with the diagnostic images.

  When the diagnostician finishes observing all the diagnostic images to be diagnosed, the CPU 21 calculates the relationship between the observation time for each area and the image feature amount calculated by the image processing unit 23. FIG. 4 is a diagram showing an approximate curve generated from a data group serving as coordinate points by plotting the image feature amount for each area on the horizontal axis and the observation time on the vertical axis.

  Furthermore, a threshold value Th_p of the image feature amount of an area that has been an observation target for a predetermined observation time Th_k or more is calculated from this approximate curve (step S4). In the example of FIG. 4, an intersection point between the characteristic approximate curve and a broken line (parallel to the feature amount axis) indicating a predetermined observation time Th_k is obtained, and the feature amount value at the intersection point is set as a threshold value Th_p of the image feature amount. The predetermined observation time Th_k is a predetermined value and is recorded on the hard disk 24. The observation time Th_k may be set freely by the diagnostician. Note that the threshold value Th_p of the image feature amount may be calculated at the time when the observation of a predetermined number of images or a predetermined amount of images is completed, not after the observation of all the diagnostic images.

  Next, an area having an image feature amount equal to or greater than the threshold Th_p and less than the observation time Th_k is extracted from all divided areas of all diagnostic images (step S5). FIG. 5 is a diagram illustrating an area where the feature amount is equal to or greater than the threshold Th_p. The horizontal axis indicates the time during which the diagnostic image is displayed on the display unit 11, and the vertical axis indicates the observation time of the area.

  As shown in FIG. 5, when there is an area where the image feature amount is equal to or greater than the threshold Th_p and the observation time is less than Th_k, there is a possibility that an important subject imaged in this area is missed. The server 20 notifies the HMD 10 of the image number including the area and the area number (step S6). Here, as shown in FIG. 6, for all the endoscopic images photographed in one capsule endoscopy, the numbers given in the photographing order are image numbers, and a plurality of areas divided by the image processing unit 23. The numbers assigned in order are called area numbers. In the case of FIG. 6, one image is divided into 16 areas.

  Receiving this, the HMD 10 displays the received image number and area number on the display unit 11 (step S14). By this display, the diagnostician can know the possibility of oversight and can reconfirm the displayed image. Instead of displaying the received image number and area number on the display unit 11, the HMD 10 may output a warning by a voice output unit (not shown) and give a warning. Further, instead of notifying the image number and the area number, the corresponding image may be retransmitted from the server 20 to the HMD 10. In this case, the HMD 10 may be provided with a mode for reproducing and displaying only the retransmitted image.

  In this embodiment, the fractal dimension is used as the image feature quantity, but other image feature quantities as described above may be used. In addition, different threshold values Th_p may be used or different image feature amounts may be used depending on the body part such as the esophagus, stomach, small intestine, and large intestine where the endoscopic image is taken. If the capsule endoscope used is equipped with an accelerometer, the internal part of the body is identified by the cumulative value of acceleration, the radio wave reception strength of the receiving antennas at multiple locations that receive endoscopic images from the capsule endoscope, and after swallowing Is estimated by the diagnostician based on the elapsed time of the image, the progress of the captured image, and the like. These pieces of information may be recorded in the server 20 together with the captured image.

  Further, the determination may be made using a plurality of image feature amounts. When a plurality of image feature amounts are used, it is preferable to set a threshold value Th_p for each image feature amount and notify the HMD 10 of the image number including the corresponding area for each image feature amount and the area number.

  The threshold value Th_p of the image feature amount may be created in advance from another diagnosis example, recorded in the hard disk 24 of the server 20, and the like may be used. For example, a sample diagnosis of a veteran doctor may be registered, threshold values may be prepared for each part of the body, or the threshold value of the previous diagnosis may be used when revisiting the same patient.

  Further, in this embodiment, the observation area of the image of the diagnostician is specified using the HMD 10, but the diagnostic image is displayed on a display such as a liquid crystal, and the eye of the diagnostician is provided with a camera provided close to the display. The coordinates of the viewpoint on the display, that is, the observation area of the image displayed on the display may be detected by detecting the line of sight.

  Note that it is also possible to specify the observation area of the image of the diagnostician using a combination of a known wearable line-of-sight detection unit (including line-of-sight detection calibration processing) and a display such as a normal liquid crystal. In this case, it is only necessary to include means for detecting which position of the image displayed on the display is the point of the detected line of sight (gaze position).

<Second Embodiment>
In the diagnosis support system according to the second embodiment, when an image including a caution area is displayed at the time of diagnosis, auxiliary information is displayed for the caution area and alerts the diagnostician.

  FIG. 7 is a flowchart illustrating the operation of the diagnosis support apparatus according to the second embodiment. In addition, the same code | symbol is attached | subjected to the part which is common in the flowchart shown in FIG. 3, and the detailed description is abbreviate | omitted.

  First, as shown in FIG. 8, the threshold value Th_p of the image feature amount is calculated using a predetermined number of diagnostic images (step S21). The image feature amount threshold Th_p is calculated for each area of each image by displaying each image transmitted from the server 20 on the display unit 11 and detecting the line of sight of the diagnostician, as in the first embodiment. Is calculated from the approximate curve of the relationship with the image feature amount for each area. In the present embodiment, the threshold Th_p is calculated not only for the fractal dimension but also for each image feature amount such as an average by density / color, a histogram, a frequency distribution, and an edge intensity distribution by edge direction. Different threshold values Th_p may be used depending on the body part.

  Even after calculating the threshold value Th_p, the server 20 continues to sequentially read the diagnostic images (step S22). The image processing unit 23 divides the diagnostic image into a plurality of areas (step S23), and calculates an image feature amount for each divided area (step S24). The image feature amount here is also calculated for all image feature amounts for which the threshold value Th_p has been calculated.

  Here, it is determined whether or not the calculated image feature amount is larger than a previously calculated threshold value Th_p (step S25). This determination is also performed for all the image feature amounts for which the threshold value Th_p has been calculated. If there is even one corresponding image feature amount, the determination in step S25 is Yes. If it is larger than the threshold Th_p (Yes in step S25), the auxiliary information is combined with the corresponding area of the corresponding diagnostic image with the corresponding area as the area requiring attention (step S26). The auxiliary information may be information for changing the color of the area requiring attention or a thick frame surrounding the area requiring attention.

  After the determination for all areas is completed, the synthesized diagnostic image (if there is no area where the image feature amount is larger than the threshold Th_p, the read diagnostic image as it is) is transmitted to the HMD 10 by the communication unit 25 (step S27). The server 20 repeats the processing from step S22 to step S27 for all diagnostic images.

  As in the past, the HMD 10 receives the diagnostic images transmitted from the server 20 in order by the communication unit 14 and reproduces and displays them on the display unit 11 (step S11). FIG. 9 is a diagram showing a display on the display unit 11 at this time. Auxiliary information is synthesized in the caution area 17 of the diagnostic image 16 so that the diagnostician can recognize that the area should be checked with particular care.

  In addition, feature amount information 18 indicating which image feature amount is larger than the threshold Th_p is displayed on the display unit 11 at the same time as the diagnostic image 16. When a plurality of image feature amounts are larger than the threshold value Th_p, all the image feature amounts are displayed.

  Further, the HMD 10 detects the line of sight of the diagnostician at the line-of-sight detection unit 13 (step S12), and transmits viewpoint information based on the detected line of sight to the server 20 via the communication unit 14 (step S13). The HDM 10 repeats the processing from step S11 to step S13 each time a diagnostic image is transmitted from the server 20.

  The server 20 that has received the viewpoint information calculates the observation time for each area in the image (step S2).

  When the diagnostician finishes observing all the diagnostic images, an approximate curve of these relationships is calculated from the calculated observation time and the image feature amount calculated in advance, and further, a predetermined observation time is calculated from the approximate curve. The threshold value Th_p of the image feature amount of the area that is the observation target for Th_k or more is calculated (step S4). Also in this case, the calculation is performed for all image feature amounts used in step S21. Note that the threshold Th_p is calculated every time observation of a predetermined number of diagnostic images or a predetermined imaging time is completed, not after all the diagnostic images are observed, and the threshold Th_p used in step S25 is updated. May be. For example, the threshold Th_p is calculated and updated for each 30 minute diagnostic image.

  Next, an area having an image feature amount equal to or greater than the threshold Th_p and an observation time less than Th_k is extracted from all divided areas of all diagnostic images (step S5). In the area where the observation time is less than Th_k, if at least one of the plurality of image feature amounts has an image feature amount equal to or greater than the threshold Th_p, the area is extracted. The server 20 notifies the HMD 10 of the image number including the extracted area and the area number (step S6).

  Receiving this, the HMD 10 displays the received image number and area number on the display unit 11 (step S14). The display allows the diagnostician to know the possibility of oversight.

  In the present embodiment, auxiliary information is displayed for the area requiring attention, but the reproduction rate (display time) of the diagnostic image including the area requiring attention may be increased. Furthermore, the playback rate may be increased as the number of areas requiring attention increases. Conversely, the playback rate of diagnostic images that do not require attention areas may be shortened. You may comprise so that it may not transfer to the display of the following image until the observation time of the area requiring attention exceeds threshold value Th_p.

  In calculating the image feature amount, three colors of RGB may be used as the color of the image data, or a weighted average of the three colors may be used as the gray density or luminance. In addition, image data may be converted into lightness / saturation / hue, and an image feature amount may be calculated for the converted image data.

FIG. 1 is a schematic diagram showing an embodiment of a diagnosis support system according to the present invention. FIG. 2 is a block diagram showing an embodiment of the diagnosis support system. FIG. 3 is a flowchart showing the operation of the diagnosis support system. FIG. 4 is a diagram showing an approximate curve generated from the relationship between the image feature amount and the observation time for each area. FIG. 5 is a diagram illustrating an area where the feature amount is equal to or greater than the threshold Th_p. FIG. 6 is a diagram showing the image number and area number of the diagnostic image. FIG. 7 is a flowchart illustrating the operation of the diagnosis support apparatus according to the second embodiment. FIG. 8 is a diagram illustrating timing for calculating the threshold value Th_p of the image feature amount. FIG. 9 is a diagram illustrating a state in which an image transmitted from the server 20 is displayed on the display unit 11.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Head mounted display (HMD), 11 ... Display part, 13 ... Eye-gaze detection part, 16 ... Diagnostic image, 17 ... Area of caution, 18 ... Feature-value information, 20 ... Server, 23 ... Image processing part, 24 ... Hard disk , 25 ... Communication part

Claims (18)

In a diagnosis support apparatus for a diagnostician to observe and diagnose a plurality of diagnostic images that are continuously reproduced and displayed on a display means,
Extraction means for extracting a region requiring attention from the diagnostic image for diagnosis;
The line of sight of the diagnostician when the diagnostician observes the diagnostic image reproduced and displayed on the display means is detected, and the observation time for each of the divided areas of the diagnostic image is calculated based on the detected line of sight Means,
Discriminating means for determining whether or not there is an oversight of the attention area based on the extracted attention area and the calculated observation time for each area;
A diagnostic support apparatus comprising: The extraction unit includes a feature amount calculation unit that calculates an image feature amount of the diagnostic image,
The diagnosis support apparatus according to claim 1, wherein the region requiring attention is extracted based on the calculated image feature amount.   The diagnosis support apparatus according to claim 2, wherein the feature amount calculation unit calculates an image feature amount for each of the plurality of divided regions.   The diagnosis support apparatus according to claim 3, wherein the extraction unit extracts a region having the image feature amount equal to or greater than a first threshold among the plurality of divided regions as a region requiring attention.   The diagnosis support apparatus according to claim 4, wherein the determination unit determines that an area having the observation time less than a second threshold among the attention areas is overlooked.   5. The apparatus according to claim 4, further comprising a determining unit that determines the first threshold based on a characteristic indicating a relationship between an image feature amount and an observation time for each of the plurality of divided areas. Or the diagnostic assistance apparatus of 5.   The determination unit generates an approximate curve or an approximate line from a data group including a combination of an image feature amount and an observation time for each of the plurality of divided areas, and sets the second threshold set in advance and the generated approximation. The diagnosis support apparatus according to claim 6, wherein the first threshold is determined from a relationship with a curve or an approximate straight line.   The diagnosis support apparatus according to claim 6 or 7, wherein the determination unit determines the first threshold value for each body part taken by the diagnostic image. The feature amount calculating means calculates a plurality of image feature amounts,
The determining means determines the first threshold value for each of the plurality of image feature amounts,
The extraction unit extracts, as the area requiring attention, an area in which at least one of the plurality of image feature amounts is equal to or more than the first threshold among the plurality of divided areas. The diagnosis support apparatus according to any one of 8.   The diagnosis support apparatus according to claim 1, further comprising a notification unit that notifies a determination result of the determination unit.   The diagnosis support apparatus according to claim 10, wherein the notification unit displays information indicating the attention area and information indicating a diagnostic image from which the attention area is extracted on the display unit. .   12. The diagnosis support apparatus according to claim 1, further comprising means for synthesizing auxiliary information indicating the extracted attention area in the diagnosis image.   13. The diagnosis support apparatus according to claim 1, further comprising changing means for changing a time for reproducing and displaying the diagnostic image from which the attention area is extracted.   The diagnosis support apparatus according to claim 13, wherein the changing unit increases the time for reproducing and displaying the diagnostic image as the area of the attention area increases.   The image feature amount is any one of an average by density / color, a histogram, a frequency distribution, an edge intensity distribution by edge direction, and a fractal dimension. Diagnostic support device.   The display means is a head mounted display provided with a camera for detecting the line of sight of the diagnostician or a display provided with a camera close to the line for detecting the line of sight of the diagnostician. The diagnosis support apparatus according to any one of claims 1 to 15.   The diagnosis support apparatus according to any one of claims 1 to 16, wherein the diagnosis image is an image photographed by a capsule endoscope. In a diagnostic support method for a diagnostician to observe and diagnose a plurality of diagnostic images continuously reproduced and displayed on a display means,
Extracting a region requiring attention from the diagnostic image for diagnosis; and
The line of sight of the diagnostician when the diagnostician observes the diagnostic image reproduced and displayed on the display means is detected, and the observation time for each of the divided areas of the diagnostic image is calculated based on the detected line of sight Process,
Determining whether or not there is an oversight of the attention area based on the extracted attention area and the calculated observation time for each area;
A diagnostic support method characterized by comprising:

Images