JP7066385B2 - Information processing methods, information processing equipment, information processing systems and programs - Google Patents

Description

The disclosure of this specification relates to an information processing method, an information processing apparatus, an information processing system and a program.

There is known a transfer learning technique that efficiently performs machine learning that is effective for other data by using the result of machine learning using certain data (Non-Patent Document 1).

Patent Document 1 describes a technique for transfer learning to a classifier that classifies a second label based on a classifier that has been trained to classify the first label using different labels attached to the same data. Is described.

Japanese Unexamined Patent Publication No. 2017-084320

Pan SJ and Yang Q, "A survive on transfer learning", IEEE Transitions on Knowledge and Data Engineering, 22 (10), pp. 1345-1349, 2010

As in the medical field, the amount of labels given may differ depending on the field of data. For example, if the first label is the age and the second label is the blood test value, the first label is given to all cases, but the second label is given only to the cases where the blood test is performed. Patent Document 1 does not disclose the case where the number of data differs depending on the type of label.

The information processing method according to the embodiment of the present invention includes a first medical data group to which only the first label is attached among the first label and the second label, and the first label and the above. Of the second label, the acquisition step of acquiring the second medical data group to which only the second label is attached, and the first machine learning based on the first medical data group are performed. It is characterized by having one learning step and a second learning step of performing a second machine learning based on the parameters in the first machine learning and the second medical data group.

According to the information processing method according to the embodiment of the present invention, even if different types of labels are assigned to each data group, the second label is based on the result of machine learning related to the first label. Machine learning related to labels can be performed appropriately.

It is a figure which shows an example of the functional structure of an information processing apparatus. It is a figure which shows an example of the hardware composition of an information processing apparatus. It is a flowchart which shows an example of the processing of an information processing apparatus. It is a figure which shows the example of the 1st machine learning and the 2nd machine learning. It is a figure which shows another example of the functional structure of an information processing apparatus. It is a flowchart which shows another example of the processing of an information processing apparatus.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

<Embodiment 1>
In the medical field, development of a diagnostic support device that infers and presents information that assists a doctor's diagnosis from medical images is underway. Machine learning based on medical images and their labels may be performed for this reasoning. Although a large amount of learning data is required for machine learning, it may not be possible to obtain a large amount of data including labels of information to be inferred among various information used for diagnosis. The first embodiment aims to enable accurate machine learning even when there are a small number of data including labels of information to be inferred.

The information processing apparatus in the first embodiment transfers a classifier that classifies the first label constructed from a large number of data, and constructs a classifier that classifies the second label using a small number of data.

In the following, the medical image used for learning will be a chest X-ray CT image, and medical information will be used as the first label and the second label. The first label and the second label are medical information indicating the state of the subject in the medical image, respectively. More specifically, the diagnostic name of the lung nodule is used as the first label, and the imaging findings of the lung nodule are used as the second label.

FIG. 1 is a diagram showing an example of the functional configuration of the information processing apparatus 100 according to the first embodiment. The information processing device 100 is an example of a device that executes the information processing method according to the embodiment of the present invention. In the information processing apparatus 100 of the present embodiment, the storage unit 200 holds a medical image used for learning and medical information such as a diagnosis name and image findings related to the medical image as a label. The storage unit 200 holds information extracted from the PACS, the electronic medical record, and the interpretation report. Alternatively, the storage unit 200 may be a PACS, an electronic medical record, or an image interpretation report, and the storage unit 200 outputs necessary information to the information processing device 100 in accordance with a request from the information processing device 100.

The information processing apparatus 100 includes an acquisition unit 102, a selection unit 104, a first machine learning unit 106, and a second machine learning unit 108. The acquisition unit 102 makes a request to the storage unit 200, and obtains a first medical data group having a plurality of sets of medical images and diagnosis names (first label), and a set of medical images and image findings (second label). Acquire a second group of medical data having a plurality of pieces. The selection unit 104 selects at least a part of the data to which the first label and the second label are attached as the third medical data group. The first machine learning unit 106 performs the first machine learning that classifies the diagnosis names based on the first medical data group. The second machine learning unit 108 performs a second machine learning that classifies image findings based on a second medical data group and a third medical data group based on the result of the first machine learning.

At least a part of each part of the information processing apparatus 100 shown in FIG. 1 may be realized as an independent apparatus. Further, it may be realized as software that realizes each function. In this embodiment, it is assumed that each part is realized by software.

FIG. 2 is a diagram showing an example of the hardware configuration of the information processing apparatus 100. The CPU 1001 mainly controls the operation of each component. The main memory 1002 stores a control program executed by the CPU 1001 and provides a work area when the program is executed by the CPU 1001. The magnetic disk 1003 stores a program for realizing various application software including an operating system (OS), a device driver of a peripheral device, and a program for performing processing described later. By executing the program stored in the main memory 1002, the magnetic disk 1003, etc., the CPU 1001 realizes the function (software) of the information processing apparatus 100 shown in FIG. 1 and the processing in the flowchart described later.

The display memory 1004 temporarily stores display data. The monitor 1005 is, for example, a CRT monitor, a liquid crystal monitor, or the like, and displays an image, a text, or the like based on the data from the display memory 1004. The mouse 1006 and the keyboard 1007 each perform pointing input and character input by the user. Each of the above components is communicably connected to each other by a common bus 1008.

The CPU 1001 is an example of a processor. The information processing apparatus 100 may have at least one of a GPU and an FPGA (Field-Programmable Gate Array) in addition to the CPU 1001. The main memory 1002 and the magnetic disk 1003 are examples of memory. The information processing apparatus 100 may have an SSD (Solid State Drive) as a memory.

Next, the entire process performed by the information processing apparatus 100 will be described with reference to the flowchart of FIG. FIG. 3 is a flowchart showing an example of processing performed by the information processing apparatus 100. In the present embodiment, the process shown in FIG. 3 is realized by the CPU 1001 executing a program that realizes the functions of each part stored in the main memory 1002.

In this embodiment, the data used in the first machine learning (first medical data group) and the data used in the second machine learning (sum of the second medical data group and the third medical data group) are used. Avoid duplication of data. This is to avoid the influence on machine learning due to duplication of data because the relationship between the first machine learning and the second machine learning is not independent. In this embodiment, a case where the learning data is not duplicated between the first machine learning and the second machine learning will be described as an example, but as shown in the modification described later, there is duplication. You may.

In step S310, the acquisition unit 102 makes a request to the storage unit 200 to acquire the first medical data group and the second medical data group. In the present embodiment, a plurality of medical images to which a diagnosis name is given as a first medical data group and a plurality of medical images to which an overall shape, which is one of the image findings, are given as a second medical data group are acquired. do. However, in the present embodiment, the first medical data group does not include the data to which the image findings are added. A medical image to which both a diagnosis name and an overall shape are given is referred to as a third medical data group described later. Specifically, the acquisition unit 102 requests the storage unit 200 to have a plurality of medical images to which a diagnosis name is given and a plurality of medical images to which image findings of the entire shape are given. Among these data groups acquired from the storage unit 200, the acquisition unit 102 acquires a plurality of medical images to which the diagnosis name is given and the image findings of the entire shape are not given as the first medical data group. Among these data groups acquired from the storage unit 200, the acquisition unit 102 acquires a plurality of medical images to which the image findings of the entire shape are given and to which the diagnosis name is not given, as the second medical data group. That is, of the first label (for example, diagnosis name) and the second label (for example, image findings), the data group to which only the first label is attached is the first medical data group. Of the first label and the second label, the data group to which only the second label is attached is the second medical data group. Step S310 is an example of an acquisition step of acquiring a first medical data group and a second medical data group.

Here, it is assumed that any one of three diagnosis names is given: primary lung cancer, lung metastasis of cancer, and benign nodule. Image findings express the characteristics of medical images, and are expressed in words, for example. It is assumed that any of four shapes, spherical, wedge-shaped, irregular, and planar, is given as the overall shape. Further, it is assumed that the number of data in the first medical data group is larger than the number of data in the second medical data group (for example, there are five times or more).

The quantities shown for the first medical data group and the second medical data group are examples, and it is not always necessary to satisfy this condition. An example of the quantity of the first medical data group and the second medical data group is shown, but the sum of the first medical data group and the third medical data group, the second medical data group and the third medical data group are shown. It may be regarded as the number of data with the sum of medical data groups. Further, the relationship between the number of data having the first label and the number of data having the second label may be, for example, a small number of the former and a large number of the latter, or both may have the same number. There may be.

In step S320, the selection unit 104 selects, among the data groups acquired from the storage unit 200 in step S310, a plurality of medical images to which the diagnosis name and the image findings of the overall shape are given as the third medical data group. .. In the present embodiment, all the data to which the diagnosis name and the image finding of the whole shape are given are selected as the third medical data group. That is, the data group to which the first label (for example, the diagnosis name) and the second label (for example, the image findings) are attached is the third medical data group. The acquisition unit 102 may integrate the processing of step S320 and the processing of step S310.

In step S330, the first machine learning unit 106 performs the first machine learning to classify the diagnosis name based on the first medical data group acquired in step S310. The first machine learning is to learn a deep convolutional neural network (DCNN). The DCNN is generally composed of an input layer, a plurality of convolution layers, a fully connected layer, and an output layer. In the following, the total number of layers is indicated by N, and the output layer is referred to as the Nth layer. Step S330 is an example of a first learning step in which the first machine learning is performed based on the first medical data group.

In step S340, the second machine learning unit 108 performs the second machine learning based on the second medical data group acquired in step S310 and the third medical data group selected in step S320. The second machine learning is machine learning that classifies image findings based on the result of the first machine learning performed in step S330. The first machine learning result is, for example, a parameter learned in the first machine learning or an output value calculated based on the parameter, and is hereinafter referred to as a parameter in the first machine learning. In the present embodiment, the second machine learning is performed by fine-tuning the first machine learning result using the data of the sum of the second medical data group and the third medical data group. That is, the second machine learning unit 108 learns DCNN in the second machine learning. Fine-tuning in DCNN means that the output layer of the model trained with the data having the first label is replaced so as to match the second label, and the parameter of the trained model is used as the initial value, and the second label is used. It refers to a method of performing re-learning with the data that it has. By using this method, it is possible to achieve the same performance with a small number of data, while learning is performed without initial values. Step S340 is an example of a second learning step in which the second machine learning is performed based on the parameters in the first machine learning and the second medical data group.

The machine learning method is not limited to DCNN. The first machine learning unit 106 and the second machine learning unit 108 may perform machine learning using a method such as a Bayesian network, a support vector machine, or a random forest.

FIG. 4 is a diagram showing an example of the relationship between data groups and fine-tuning in the present embodiment. The two circles in the total data represent a set of a plurality of medical images with a diagnosis name (left) and a set of a plurality of medical images with the image findings of the whole shape (right), respectively. The first medical data group 410 (dark color portion in FIG. 4) and the second medical data group 420 (light color portion in FIG. 4) are schematic views of the respective data groups acquired in step S310. The third medical data group 430 is a data group selected in step S320, and is a data group of a plurality of medical images to which a diagnosis name and an image finding of the whole shape are given.

The first machine learning unit 106 performs the first machine learning on the first medical data group 410 (dark color portion in FIG. 4) (step S330). The second machine learning unit 108 has the same model structure as the first machine learning unit 106 except for the output layer (Nth layer), and from the first layer to the N-1 layer in the first machine learning result. It has the parameter 440 of the above as an initial value. At this time, the output layer 450 of the first machine learning unit has a diagnostic name (first label), whereas the output layer 460 of the second machine learning unit has an overall shape (second label). It has been replaced. Then, the second machine is the data of the sum of the second medical data group and the third medical data group (the light-colored portion in FIG. 4 and the shaded portion in FIG. 4, that is, the data group represented by the circle on the right side). Learning is performed (step S340). Since the second machine learning is performed based on the first machine learning result, the number of data required for the effective second machine learning can be suppressed.

In the first embodiment, a case where the second machine learning is performed using the data group to which the image findings of the whole shape are given will be described as an example, but the second machine learning is performed for each different image finding. good. That is, a plurality of transfer learnings (second machine learning) may be performed using the parameters in one machine learning (first machine learning). As described above, in the medical data group, while many types of image findings are used, the number of data groups to which each image finding is given may be small. Performing a plurality of transfer learnings based on one machine learning parameter is particularly effective when performing machine learning using a medical data group.

For example, consider a case where the number of data in the first medical data group is 9000, the number of data in the second medical data group is 500, and the number of data in the third medical data group is 1000. If the data with overlapping labels (third medical data group) is not used for machine learning, the number of training data used for the first machine learning is 9000, and the number of training data used for the second machine learning is 9000. The number of data is 500. On the other hand, when the processing of the present embodiment is performed, the number of learning data used for the first machine learning does not change to 9000, and the number of learning data used for the second machine learning becomes 1500. Therefore, the number of learning data used for the second machine learning is three times as large as that in the case of excluding the data whose label overlaps with the learning data used for the first machine learning. In general, in machine learning, the larger the number of data, the higher the accuracy of learning, so it can be expected that the accuracy of the second machine learning will be positively affected compared to the case where duplicated data is not used for both machine learning. ..

In the present embodiment, the first machine learning is performed on the first medical data group to which only the first label of the first label and the second label is attached. Further, a second medical data group to which only the second label of the first label and the second label is attached, and a third medical data to which the first label and the second label are attached. Perform a second machine learning with the group. By using the data with duplicate labels (first label, second label) only for the second machine learning, the number of data for the second machine learning can be increased, and the first based on the first machine learning. The machine learning of 2 can be performed appropriately. The method of the present embodiment is particularly effective when the number of data in the second medical data group is small.

(Modification 1 of Embodiment 1)
In the first embodiment, in step S320, among the data acquired from the storage unit 200, the data to which the diagnosis name and the image finding of the whole shape are given are selected as the third medical data group. In the first modification, a third medical data group may be created by attaching a second label to a part of the first medical data group. Alternatively, a second label may be attached to a part of the first medical data group and added to the third medical data group. This process is performed by the information processing apparatus 100 having an addition unit 112 (not shown), and the addition unit 112 attaches a second label to a part of the first medical data group. In yet another example, the selection unit 104 may select a part of the data group of the data group to which both the first label and the second label are attached as the third medical data group.

For example, consider a case where the number of data in the first medical data group is 9500, the number of data in the second medical data group is 5000, and the number of data in the initial third medical data group is 500. In this case, when the process of the present embodiment is applied, the number of data of the training data used for the first machine learning is 9,500, and the number of data of the learning data used for the second machine learning is 1000. Here, consider a case where 500 data are selected from the data of the initial first medical data group 9500, a second label is given, and the data is added to the third medical data group. In this case, the number of data of the training data used for the first machine learning is 9000, and the number of data of the training data used for the second machine learning is 1500. That is, the number of data of the training data used for the first machine learning is about 95% and the number of data of the learning data used for the second machine learning is 150% as compared with before the processing. In machine learning, it is considered that there is almost no negative effect even if the number of learning data reaches 95%, whereas it can be expected to have a large positive effect when it reaches 150%.

Therefore, according to this modification, even when the number of data in the second medical data group is small, the number of data for the second machine learning can be secured by increasing the data, so that the first machine learning can be performed. The second machine learning based on it can be done appropriately.

(Modification 2 of Embodiment 1)
In the first embodiment, in step S340, the second machine learning is performed by transferring the first machine learning result to fine-tuning. However, when the number of data in the data group to which the second label is attached is larger than a predetermined value, normal machine learning may be more accurate than transferring the first machine learning result. , Ordinary machine learning may be performed without transfer. That is, the first machine learning and the second machine learning may be performed independently.

For example, if the ratio of the number of data in the second labeled data group to the number of data in the first labeled data group exceeds 0.5, the first machine learning result is transferred. Also judges that normal machine learning will be highly accurate. Of course, this ratio is an example and may be another value. Moreover, you may judge by another method which does not use a ratio.

According to this modification, if it is determined that normal machine learning is more accurate than transferring the first machine learning result, the second machine learning can be performed more appropriately by not performing the transfer. can. This modification is particularly effective when collecting data continuously.

(Modification 3 of Embodiment 1)
In the first embodiment, in step S330 and step S340, machine learning was performed using the same method in the first machine learning and the second machine learning, but in the first machine learning and the second machine learning. Different methods may be used. For example, the first machine learning may use DCNN, and the second machine learning may learn a support vector machine having an intermediate output of DCNN (first machine learning result) as an input. Of course, these are just examples, and other methods may be used.

According to this modification, the second machine learning based on the first machine learning can be performed more appropriately by using the optimum methods for each of the first machine learning and the second machine learning.

(Modified Example 4 of Embodiment 1)
In the first embodiment, the third medical data group was selected in step S320. However, the present invention is not limited to this, and when it is found that there is no overlapping data between the data group to which the first label is attached and the data group to which the second label is attached, in step S320. It is not necessary to select the third group of medical data. Then, learning in steps S330 and S340 may be performed only with the first medical data group and the second medical data group.

According to this modification, the second machine learning based on the first machine learning can be appropriately performed even when there is no overlapping data in the first medical data group and the second medical data group. In particular, it is effective when the first medical data group and the second medical data group are acquired from different data sources.

<Embodiment 2>
The information processing apparatus 500 in the second embodiment determines whether or not the third medical data group is used for learning in the first machine learning and the second machine learning based on the number of data of the third medical data group.

FIG. 5 is a diagram showing an example of the functional configuration of the information processing apparatus 500 according to the second embodiment. It should be noted that only the differences from the first embodiment will be described for the components to which the same reference numerals are given as those in FIG.

The information processing apparatus 500 includes an acquisition unit 102, a selection unit 104, a determination unit 510, a first machine learning unit 506, and a second machine learning unit 508. The determination unit 510 determines whether or not to use the third medical data group for learning in the first machine learning unit 506 and the second machine learning unit 508 based on the number of data of the third medical data group. The first machine learning unit 506 performs the first machine learning that classifies the diagnosis name based on the data determined based on the determination. The second machine learning unit 508 performs the second machine learning that classifies the image findings with the data determined based on the determination based on the result of the first machine learning.

The hardware configuration of the information processing apparatus 500 in this embodiment is the same as that in FIG. 2 in the first embodiment.

Next, the entire process performed by the information processing apparatus 500 will be described with reference to the flowchart of FIG. It should be noted that only the difference from the first embodiment will be described for the process to which the same reference numerals are given as in FIG.

The processing of steps S610 and S620 is the same as the processing of steps S310 and S320 in the first embodiment.

In step S625, the determination unit 510 determines the third medical data in the first machine learning and the second machine learning of steps S630 and S640 based on the number of data of the third medical data group selected in step S620. Determine if to use the group.

In the present embodiment, the determination unit 510 determines that the third medical data group is not used in the first machine learning when the number of data in the third medical data group is larger than a predetermined value, and the second machine learning. Then, it is determined that the third medical data group is used. That is, in step S630 and step S640, the same processing as in the first embodiment is performed.

When the number of data in the third medical data group is equal to or less than a predetermined value, the determination unit 510 determines that the third medical data group is not used in steps S630 and S640. In this case, the third medical data group is used as evaluation data for the first machine learning means and the second machine learning means. By making the evaluation data the same, machine learning can be performed in consideration of the accuracy of the first machine learning and the accuracy of the second machine learning at the same time.

In step S630, the first machine learning unit 506 performs the first machine learning for classifying the diagnosis name based on the first medical data group acquired in step S610. Note that machine learning is performed by DCNN as in the first embodiment, and the description thereof will be omitted.

In step S640, the second machine learning unit 508 is the first machine performed in step S630 based on the second medical data group acquired in step S610 and the third medical data group selected in step S620. A second machine learning is performed to classify the image findings based on the learning result. In the present embodiment, when it is determined in step S625 that the third medical data group is used, learning is performed using the sum data of the second medical data group and the third medical data group as in the first embodiment. .. On the other hand, when it is determined in step S625 that the third medical data group is not used, learning is performed using the second medical data group. The second machine learning is performed by fine-tuning the first machine learning result using DCNN as in the first embodiment, and the description thereof will be omitted.

In the present embodiment, it is determined whether or not the third medical data group is used for learning in the first machine learning and the second machine learning based on the number of data of the third medical data group. Since the data used for learning can be flexibly handled according to the state of the third medical data group, the second machine learning based on the first machine learning can be appropriately performed.

(Modification 1 of Embodiment 2)
In the second embodiment, in step S625, it is determined whether or not the third medical data group is used for learning in the first machine learning and the second machine learning based on the number of data of the third medical data group. .. However, the number of data in the first medical data group and the number of data in the second medical data group may also be considered. More specifically, the number of data in the first medical data group (| D1 |), the number of data in the second medical data group (| D2 |), and the number of data in the third medical data group (| D3 |). Judgment is made based on the ratio of.

For example, when | D3 | is equal to or less than the first predetermined value, the determination unit 510 determines that the third medical data group is not used in the first machine learning and the second machine learning.

When | D3 | is larger than the first predetermined value, the determination unit 510 makes a determination based on the ratio as follows. When | D3 | / | D1 | is equal to or less than the second predetermined value and | D3 | / | D2 | is larger than the second predetermined value, the ratio of the third medical data group to the first medical data group. Is small, indicating that the ratio of the third medical data group to the second medical data group is large. In this case, it is considered that the influence on the first machine learning by not using the third medical data group is small. On the other hand, it is considered that learning is performed only by the second medical data group without using the third medical data group, which has a great influence on the second machine learning. The determination unit 510 determines that the third medical data group is not used in the first machine learning, and determines that the third medical data group is used in the second machine learning. In this case, the number of data having the second label can be increased, and the second machine learning can be performed more appropriately.

When | D3 | / | D1 | is equal to or less than the second predetermined value and | D3 | / | D2 | is equal to or less than the second predetermined value, the first medical data group and the second medical data group are obtained. It shows that the ratio of the medical data group of 3 is small. In this case, it is considered that the influence on the first machine learning and the second machine learning by not using the third medical data group is small. Therefore, the determination unit 510 determines that the third medical data group is not used in both the first machine learning and the second machine learning. At this time, as described above, the third medical data group may be used as the evaluation data.

When | D3 | / | D1 | is larger than the second predetermined value and | D3 | / | D2 | is larger than the second predetermined value, the third for the first medical data group and the second medical data group. It shows that the proportion of the medical data group of is large. In this case, it is considered that the influence on the first machine learning and the second machine learning by not using the third medical data group is large. Here, each of them is due to the fact that the number of data used for learning is lower than the influence on each machine learning by learning using the overlapping data in the first machine learning and the second machine learning. It is thought that the impact on machine learning is large. Therefore, the determination unit 510 determines that the third medical data group is used in both the first machine learning and the second machine learning.

When | D3 | / | D1 | is larger than the second predetermined value and | D3 | / | D2 | is equal to or less than the second predetermined value, the ratio of the third medical data group to the first medical data group is It is large and indicates that the ratio of the third medical data group to the second medical data group is small. That is, since | D1 | << | D2 |, the determination unit 510 uses the third medical data group in both the first machine learning and the second machine learning, and similarly to the second modification of the first embodiment. It is determined that the second machine learning does not transfer the result of the first machine learning and performs normal machine learning.

That is, step S625 determines whether or not to use the third medical data group for at least one of the first machine learning and the second machine learning based on the number of data of the third medical data group. This is an example of the process.

According to this modification, the data used for learning can be determined according to the states of the first medical data group and the second medical data group in addition to the third medical data group, so that the first machine can be determined. The second machine learning based on learning can be performed more appropriately.

(Other embodiments)
In the above-described embodiment, the case of learning about the lung nodule in the chest X-ray CT image has been described as an example, but the present invention is not limited to this. The target medical images include a CT device, a digital radiography, an MRI (Magnetic Resonance Imaging) device, a SPECT (Single Photon Emission CT) device, a PET (Positron Emission Tomography) device, an ultrasonic diagnostic device, an ultrasonic diagnostic device, and an ultrasonic diagnostic device. It may be a medical image acquired by using at least one of an imaging device such as an acoustic device. The target lesion is not limited to the lung nodule shadow, and may be a lesion at any site of the subject. The target of learning is not limited to medical treatment. For example, the image used for learning is an image taken by a camera, the first label is an image scene, and the second label is an object (sky, tree, etc.) in the image. It may be in a state.

In the above-described embodiment, an example in which the image findings of the whole shape are used as the second label has been described, but the present invention is not limited to this. For example, the second label may be any image finding. Imaging findings are, for example, overall shape showing the overall shape of the lesion, findings showing the size of the lesion, findings showing the state of the anatomical structure (for example, findings related to the bronchial fluoroscopy), and findings showing the detailed shape of the lesion (for example, lesion). Findings regarding the shape of the notch, findings regarding the spinous process), etc. can be mentioned.

The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The information processing device in each of the above-described embodiments may be realized as a single device, or may be a form in which a plurality of devices are combined so as to be able to communicate with each other to execute the above-mentioned processing, both of which are embodiments of the present invention. include. The above processing may be executed by a common server device or a group of servers. The information processing device and the plurality of devices constituting the information processing system need not be present in the same facility or in the same country as long as they can communicate at a predetermined communication rate.

In the embodiment of the present invention, a software program that realizes the functions of the above-described embodiment is supplied to the system or device, and the computer of the system or device reads and executes the code of the supplied program. Including morphology.

Therefore, in order to realize the processing according to the embodiment on the computer, the program code itself installed on the computer is also one of the embodiments of the present invention. Further, based on the instruction included in the program read by the computer, the OS or the like running on the computer performs a part or all of the actual processing, and the function of the above-described embodiment can be realized by the processing. ..

An embodiment in which the above-described embodiments are appropriately combined is also included in the embodiment of the present invention.

100 Information processing device 102 Acquisition unit 104 Selection unit 106 First machine learning unit 108 Second machine learning unit 510 Judgment unit 512 Granting unit

Claims (22)

Of the first label and the second label, the first medical data group to which only the first label is attached, and the second of the first label and the second label. An acquisition step of acquiring a second medical data group to which only a label is attached and a third medical data group to which the first label and the second label are attached .
A first learning step of performing a first machine learning based on the first medical data group, and
The second learning step of performing the second machine learning based on the parameters in the first machine learning and the second medical data group, and the second learning step based on the number of data of the third medical data group. A step of using the medical data group of 3 for at least one of the first machine learning and the second machine learning, and
An information processing method characterized by having. The information processing method according to claim 1 , wherein the second learning step performs the second machine learning based on the number of data in the third medical data group. The information according to claim 1 or claim 2 , wherein the first learning step performs the first machine learning based on the number of data of the third medical data group. Processing method. The acquisition step is characterized in that the third medical data group is acquired by attaching the second label to the data included in the first medical data group. The information processing method according to any one of the following items. The second learning step is characterized in that when the number of data in the third medical data group is larger than the first predetermined value, the third medical data group is used for the second machine learning. The information processing method according to claim 2 . In the second learning step, the number of data in the third medical data group is larger than the first predetermined value, and the number of data in the third medical data group is larger than the number of data in the first medical data group. When the ratio is equal to or less than the second predetermined value and the ratio of the number of data in the third medical data group to the number of data in the second medical data group is larger than the second predetermined value, the third medical data. The information processing method according to claim 5, wherein the group is used for the second machine learning . The number of data in the third medical data group is larger than the first predetermined value,
The ratio of the number of data in the third medical data group to the number of data in the first medical data group and the ratio of the number of data in the third medical data group to the number of data in the second medical data group are second. The information processing method according to claim 5, wherein when the value is larger than the predetermined value, the third medical data group is used in the first learning step and the second learning step . A determination step of determining whether or not the third medical data group is used for at least one of the first machine learning and the second machine learning based on the number of data of the third medical data group. The information processing method according to any one of claims 1 to 7 , further comprising. In the determination step, the ratio of the number of data in the third medical data group to the number of data in the first medical data group and the data in the third medical data group to the number of data in the second medical data group. 8. Claim 8 is characterized in that it is determined whether or not the third medical data group is used for at least one of the first machine learning and the second machine learning based on the ratio of the numbers. Information processing method described in. When it is determined in the determination step that the third medical data group is not used in either the first machine learning or the second machine learning, the first learning step and the second learning step The information processing method according to claim 8 , wherein the third medical data group is used as evaluation data for machine learning in at least one of the steps. Of the first label and the second label, the first medical data group to which only the first label is attached, and the second of the first label and the second label. An acquisition step of acquiring a second medical data group to which only a label is attached and a third medical data group to which the first label and the second label are attached .
A determination step for determining whether or not the number of data in the second medical data group is larger than a predetermined number, and
A first learning step of performing a first machine learning based on the first medical data group, and
When the number of the second data is larger than the predetermined number, the second machine learning is performed based on the second medical data group.
A second learning step in which the second machine learning is performed based on the parameters in the first machine learning and the second medical data group when the number of the second data is a predetermined number or less.
A step of using the third medical data group for at least one of the first machine learning and the second machine learning based on the number of data of the third medical data group.
An information processing method characterized by having. The acquisition step further acquires a third medical data group to which the first label and the second label are attached.
In the determination step, the ratio of the number of data in the third medical data group to the number of data in the first medical data group and the data in the third medical data group to the number of data in the second medical data group. 11. It is characterized in that it is determined whether or not the third medical data group is used for at least one of the first machine learning and the second machine learning based on the ratio of the numbers. Information processing method described in. The information processing according to any one of claims 1 to 12, wherein each of the first label and the second label is medical information indicating the state of a subject in a medical image. Method. The information according to any one of claims 1 to 13, wherein the first label is a diagnosis name, and the second label is an image finding representing a feature of a medical image. Processing method. Of the first label and the second label, the first medical data group to which only the first label is attached, and the second of the first label and the second label. An acquisition means for acquiring a second medical data group to which only a label is attached, a third medical data group to which the first label and the second label are attached, and an acquisition means.
A first learning means for performing a first machine learning based on the first medical data group,
It has a second learning means for performing a second machine learning based on the parameters in the first machine learning and the second medical data group .
An information processing apparatus characterized in that the third medical data group is further used for at least one of the first machine learning and the second machine learning based on the number of data of the third medical data group. .. The information processing apparatus according to claim 15 , wherein the second learning means further uses the third medical data group to perform the second machine learning. A first inference means for making inferences about the first label based on the first inference device generated by the first learning means, and a first inference means.
A second inference means that makes inferences about the second label based on the second inference device generated by the second learning means .
The information processing apparatus according to claim 15 or 16 . The second inference device is a claim that is a machine-learned inference device using a third medical data group to which the first label and the second label are attached as learning data. The information processing apparatus according to 17 . The information processing apparatus according to claim 17 or 18 , wherein the first label is a diagnostic name, and the second label is an image finding representing a feature of a medical image. Of the first label and the second label, the first medical data group to which only the first label is attached, and the second of the first label and the second label. An acquisition means for acquiring a second medical data group to which only a label is attached, a third medical data group to which the first label and the second label are attached, and an acquisition means.
A first learning means for performing a first machine learning based on the first medical data group,
It has a second learning means for performing a second machine learning based on the parameters in the first machine learning and the second medical data group .
An information processing system characterized in that the third medical data group is further used for at least one of the first machine learning and the second machine learning based on the number of data of the third medical data group. .. A storage means for storing data including a medical image and medical information given to the medical image, and a storage means.
The first machine learning based on the first medical data group which is the stored data and is the data to which only the first label is given among the first label and the second label. The first means of learning to do
The second medical data group, which is the stored data and is the data to which only the second label is given among the first label and the second label, and the first one. It has a second learning means that performs a second machine learning based on parameters in machine learning.
Based on the number of data in the third medical data group to which the first label and the second label are attached, the third medical data group is further added to the first machine learning and the second machine learning. An information processing system characterized in that it is used for at least one of them . A program for causing a computer to execute the information processing method according to any one of claims 1 to 14.

Images