JP7544567B2 - Medical data processing device and medical data processing method - Google Patents

Description

The embodiments disclosed in this specification and the drawings relate to a medical data processing device and a medical data processing method.

Conventionally, in the medical field, software that grows by collecting learning data and performing machine learning is known as software for implementing or supporting diagnosis, treatment planning, evaluation of treatment effectiveness, etc. using medical data. Here, in machine learning, algorithms are updated by updating the learning model using learning data.

However, in software that uses machine learning, if the software does not collect learning data based on appropriate conditions for the task, the learning model will not be updated appropriately, and as a result, the performance of the software may decline.

Note that this situation does not occur only in the updating process of a learning model in machine learning, but can also occur in various types of information processing using medical data.

Special table 2019-533870 publication JP 2015-66318 A JP 2012-157683 A

Nakagomi K, Shimizu A, Kobatake H, Yakami M, Fujimoto K, Togashi K, “Multi-shape graph cuts with neighbor prior constraints and its application to lung segmentation from a chest CT volume”, Med Image Anal. 2013, 17(1): 62-77, doi: 10. 1016/j. media. 2012.08.002

One of the problems that the embodiments disclosed in this specification and the drawings attempt to solve is to enable more appropriate information processing using medical data. However, the problems that the embodiments disclosed in this specification and the drawings attempt to solve are not limited to the above problem. Problems that correspond to the effects of each configuration shown in the embodiments described below can also be considered as other problems.

The medical data processing device according to the embodiment includes an acquisition unit, an identification unit, and a selection unit. The acquisition unit acquires medical data of multiple time phases. The identification unit identifies the motion state of biological tissue in medical data of a specific time phase included in the medical data of multiple time phases based on the medical data of the specific time phase and data related to a time phase other than the specific time phase. The selection unit selects the medical data based on the motion state.

FIG. 1 is a diagram showing the configuration of a medical data processing apparatus according to the first embodiment. FIG. 2 is a diagram for explaining an example of specifying an operation state performed by a specific function according to the first embodiment. FIG. 3 is a flowchart showing a processing procedure performed by the processing circuit of the medical data processing device according to the first embodiment. FIG. 4 is a diagram for explaining the second modification of the first embodiment. FIG. 5 is a diagram showing the configuration of a medical data processing apparatus according to the second embodiment. FIG. 6 is a flowchart showing a processing procedure performed by the processing circuit of the medical data processing device according to the second embodiment.

Below, an embodiment of a medical data processing device and a medical data processing method will be described in detail with reference to the drawings.

First Embodiment
FIG. 1 is a diagram showing the configuration of a medical data processing apparatus according to the first embodiment.

For example, as shown in FIG. 1, the medical data processing device 100 according to the first embodiment is communicatively connected to an X-ray CT (Computed Tomography) device 1 and a medical image storage device 2 via a network 3.

In addition to the X-ray CT device 1, the medical data processing device 100 may be further connected to other medical image diagnostic devices such as a Magnetic Resonance Imaging (MRI) device, an ultrasound diagnostic device, a PET (Positron Emission Tomography) device, and a SPECT (Single Photon Emission Computed Tomography) device.

The X-ray CT device 1 generates a CT image of a subject. Specifically, the X-ray CT device 1 collects projection data representing the distribution of X-rays that have passed through the subject by rotating an X-ray tube and an X-ray detector on a circular orbit that surrounds the subject. The X-ray CT device 1 then generates a CT image based on the collected projection data.

The medical image storage device 2 stores various medical images related to subjects. Specifically, the medical image storage device 2 acquires CT images from the X-ray CT device 1 via the network 3, and stores the CT images in a memory circuit within the device. For example, the medical image storage device 2 is realized by a computer device such as a server or a workstation. Also, for example, the medical image storage device 2 is realized by a PACS (Picture Archiving and Communication System) or the like, and stores CT images in a format that complies with DICOM (Digital Imaging and Communications in Medicine).

The medical data processing device 100 processes various medical data related to the subject. Specifically, the medical data processing device 100 acquires CT images from the X-ray CT device 1 or the medical image storage device 2 via the network 3, and processes the CT images. For example, the medical data processing device 100 is realized by computer equipment such as a server or a workstation.

For example, the medical data processing device 100 includes a network (NetWork: NW) interface 110, a memory circuit 120, an input interface 130, a display 140, and a processing circuit 150.

The NW interface 110 controls the transmission and communication of various data sent and received between the medical data processing device 100 and other devices connected via the network 3. Specifically, the NW interface 110 is connected to the processing circuit 150, and transmits data received from other devices to the processing circuit 150, or transmits data received from the processing circuit 150 to other devices. For example, the NW interface 110 is realized by a network card, a network adapter, a NIC (Network Interface Controller), etc.

The memory circuitry 120 stores various data and programs. Specifically, the memory circuitry 120 is connected to the processing circuitry 150, and stores data received from the processing circuitry 150, or reads out stored data and transmits it to the processing circuitry 150. For example, the memory circuitry 120 is realized by a semiconductor memory element such as a random access memory (RAM) or a flash memory, a hard disk, an optical disk, etc.

The input interface 130 accepts input operations of various instructions and various information from the user. Specifically, the input interface 130 is connected to the processing circuit 150, converts the input operations received from the user into electrical signals, and transmits them to the processing circuit 150. For example, the input interface 130 is realized by a trackball, a switch button, a mouse, a keyboard, a touchpad that performs input operations by touching the operation surface, a touch screen in which a display screen and a touchpad are integrated, a non-contact input interface using an optical sensor, and a voice input interface. Note that in this specification, the input interface 130 is not limited to only those that have physical operation parts such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and transmits this electrical signal to a control circuit is also included as an example of the input interface 130.

The display 140 displays various information and data. Specifically, the display 140 is connected to the processing circuit 150, and displays various information and data received from the processing circuit 150. For example, the display 140 is realized by a liquid crystal monitor, a CRT (Cathode Ray Tube) monitor, a touch panel, etc.

The processing circuitry 150 controls the entire medical data processing device 100. For example, the processing circuitry 150 performs various processes in response to input operations received from a user via the input interface 130. Also, for example, the processing circuitry 150 receives data transmitted by another device from the NW interface 110 and stores the received data in the memory circuitry 120. Also, for example, the processing circuitry 150 transmits the data received from the memory circuitry 120 to the NW interface 110, thereby transmitting the data to another device. Also, for example, the processing circuitry 150 displays the data received from the memory circuitry 120 on the display 140.

The configuration of the medical data processing device 100 according to this embodiment has been described above. With this configuration, the medical data processing device 100 according to this embodiment has a function of performing predetermined information processing using medical data.

For example, in the medical field, software that grows by collecting learning data and performing machine learning is known as software for implementing or supporting diagnosis, treatment planning, and evaluation of treatment effectiveness using medical images. Here, in machine learning, algorithms are updated by updating the learning model using learning data.

However, in software that uses machine learning, if the software does not collect learning data based on appropriate conditions for the task, the learning model will not be updated appropriately, and as a result, the performance of the software may decline.

Specifically, it is known that moving biological tissues exhibit characteristic shapes and properties for each different operating state. For example, it is known that heart valves (aortic valve, mitral valve, tricuspid valve, pulmonary valve) behave differently in the period from when they transition from a completely closed state to a completely open state, and in the period from when they transition from a completely open state to a completely closed state. Therefore, in software targeting such moving biological tissues, if the collected learning data is biased toward data that indicates a specific operating state of the biological tissue, the learning model will be updated through biased learning, and as a result, performance for operating states other than the specific operating state may be degraded.

Note that this situation does not occur only in the updating process of a learning model in machine learning, but can also occur in various types of information processing using medical data.

For these reasons, the medical data processing device 100 according to this embodiment is configured to more appropriately perform information processing using medical data.

Specifically, the medical data processing device 100 has an acquisition function 151, a specification function 152, a selection function 153, and an information processing function 154 as processing functions possessed by the processing circuit 150. The acquisition function 151 is an example of an acquisition unit. The specification function 152 is an example of a specification unit. The selection function 153 is an example of a selection unit. The information processing function 154 is an example of an information processing unit.

Here, the acquisition function 151 acquires medical data. Furthermore, the identification function 152 acquires the motion state of the biological tissue in the medical data acquired by the acquisition function 151. Furthermore, the selection function 153 selects the medical data based on the motion state acquired by the identification function 152. Then, the information processing function 154 performs information processing using the medical data selected by the selection function 153.

In this way, in this embodiment, by selecting medical data taking into account the operating state of the biological tissue, information processing using the medical data can be performed more appropriately.

Below, each function of the medical data processing device 100 according to this embodiment will be described in detail. In the following, an example will be described in which the medical data processing device 100 performs information processing using medical data, using CT images captured by the X-ray CT device 1 as learning data to perform an update process of a learning model in machine learning. In addition, in the following, an example will be described in which the target biological tissue is a heart valve, more specifically, the target biological tissue is an aortic valve.

The acquisition function 151 acquires CT images including the aortic valve from the X-ray CT device 1 or the medical image storage device 2 via the NW interface 110.

In this embodiment, the acquisition function 151 acquires CT images of multiple time phases including the aortic valve from the X-ray CT device 1 or the medical image storage device 2.

For example, the acquisition function 151 acquires a CT image when a start instruction is received from a user via the input interface 130. Alternatively, the acquisition function 151 may monitor the X-ray CT device 1 or the medical image storage device 2, and automatically acquire a new CT image when the new CT image is captured.

Here, an example in which a CT image is used as medical data will be described, but the embodiment is not limited to this. For example, the medical data targeted by this embodiment may be any type of medical data as long as it is capable of acquiring or predicting information regarding the movement of the target biological tissue. For example, the medical data may be medical images captured by other medical image diagnostic devices, such as ultrasound images, MRI images, X-ray images, Angio images, PET images, and SPECT images. In this case, the medical images may be two-dimensional medical images, three-dimensional medical images, or four-dimensional images obtained by capturing multiple images of these in the time direction. Alternatively, the medical data may be a time and/or spatial distribution map of one-dimensional measurement values of electricity, magnetism, near-infrared rays, etc., obtained by an electrocardiograph, an electroencephalograph, a magnetoencephalograph, a magnetocardiograph, a NIRS (Near Infrared Spectroscopy) electroencephalograph, etc.

In addition, although an example in which the target biological tissue is an aortic valve will be described here, the embodiment is not limited to this. For example, the target biological tissue of this embodiment may be other heart valves such as the mitral valve, tricuspid valve, or pulmonary valve.

The identification function 152 identifies the operating state of the aortic valve in the CT image acquired by the acquisition function 151.

For example, the identification function 152 classifies the open/closed state of the aortic valve into a plurality of states in advance. The identification function 152 then determines which of the plurality of states the open/closed state of the aortic valve in the target CT image corresponds to, and identifies the determined state as the operating state of the aortic valve in the CT image.

It is generally known that the aortic valve opens mainly between early systole and early diastole, and closes between early diastole and end diastole.

Therefore, for example, the identification function 152 classifies the open/closed state of the aortic valve into two states: an open state and a closed state. Here, the open state is the period from when the aortic valve transitions from a completely closed state to a completely open state, and the closed state is the period from when the aortic valve transitions from a completely open state to a completely closed state. The identification function 152 then determines which of the two states the open/closed state of the aortic valve in the target CT image corresponds to, and identifies the determined state as the operating state of the aortic valve in the CT image.

As a result, in this embodiment, even if the CT images show the aortic valve in the same open and closed states at first glance, the operating state may be identified as being in the open phase or in the closed phase.

Figure 2 is a diagram illustrating an example of the identification of an operating state performed by the identification function 152 according to the first embodiment.

For example, as shown in FIG. 2, the identification function 152 identifies the operating state of the aortic valve as being in the open phase for CT images taken between t1, when the aortic valve is completely closed, and t4, when the aortic valve is completely open, among the multiple time phases t1 to t8 included in one cardiac cycle. The identification function 152 also identifies the operating state of the aortic valve as being in the closed phase for CT images taken between t4, when the aortic valve is completely open, and t8, when the aortic valve is completely closed. To give one example, in the example shown in FIG. 2, the CT image taken at t5 between t4 and t8 is identified as being in the closed phase.

Here, various methods can be used by the identification function 152 to identify the operating state of the aortic valve.

In this embodiment, the identification function 152 identifies the operating state of the aortic valve in a CT image of a specific time phase included in the CT images of multiple time phases acquired by the acquisition function 151, based on the CT image of the specific time phase and data related to time phases other than the specific time phase.

Note that the data relating to a time phase other than the specified time phase may be the same type of data as the CT image of the specified time phase, or may be a different type of data. In other words, the data relating to a time phase other than the specified time phase may be a CT image, or may be data other than a CT image.

For example, the identification function 152 identifies the operating state of the aortic valve in a CT image of a specific time phase using data other than the CT image collected when CT images of multiple time phases are captured.

Generally, when CT images of the heart are taken, ECG-synchronized imaging is performed to synchronize the imaging with the movement of the heart, and the ECG information measured during imaging is often stored in the DICOM header of the CT image.

Therefore, for example, the identification function 152 may acquire electrocardiogram information from the DICOM header of each of the CT images of multiple time phases, and identify the operating state of the aortic valve in the CT image of a specific time phase from the acquired electrocardiogram information. Alternatively, the identification function 152 may acquire cardiac sound information corresponding to the CT images of multiple time phases measured when imaging is performed, and identify the operating state of the aortic valve in the CT image of a specific time phase from the acquired cardiac sound information.

Alternatively, the identification function 152 may use CT images of multiple time phases to identify the operating state of the aortic valve in a CT image of a specified time phase.

Generally, when multiple cardiac phases are targeted in a single examination, CT images of multiple time phases corresponding to the multiple cardiac phases are reconstructed.

Therefore, for example, the identification function 152 may use CT images of multiple time phases reconstructed in a single examination to estimate the direction of movement of each leaflet of the aortic valve from the position of each leaflet of the aortic valve in the CT image of the cardiac phase to be identified for its operating state and the position of each leaflet of the aortic valve in the CT image of the cardiac phase adjacent to that cardiac phase, and identify the operating state of the aortic valve in the CT image of the cardiac phase to be identified based on the result of this estimation.

Here, various techniques can be used as a method for the identification function 152 to acquire the position of each leaflet of the aortic valve. For example, the identification function 152 may acquire the position of each leaflet using a technique for identifying an organ region called organ segmentation, or may acquire the position of each leaflet through machine learning techniques using a learning model that has learned the positions of the organ.

Alternatively, the identification function 152 may identify the operating state of the aortic valve using information on another organ other than the aortic valve. For example, the identification function 152 may identify the operating state of the aortic valve using the volume of the left ventricle. Because the volume of the left ventricle changes due to the movement of the heart, the cardiac phase can be estimated from the volume of the left ventricle or its change, and the operating state of the aortic valve can be identified from the estimation result.

The selection function 153 selects CT images based on the operating state identified by the identification function 152.

In this embodiment, the selection function 153 selects CT images to be used as learning data in the update process of the learning model in machine learning, based on the operating state identified by the identification function 152.

For example, the selection function 153 compares the operating state identified by the identification function 152 with a predetermined condition, and selects CT images whose operating state satisfies the condition. Here, the predetermined condition is, for example, a condition that "the operating state of the aortic valve is in a closed state." In the case of this condition, for example, in the example shown in FIG. 2, the image at t5, which is identified as the operating state of the aortic valve being in a closed state, does not satisfy the condition, and is therefore not selected.

The conditions used here may be automatically set based on the number of learning data already selected and stored and the proportion of operating states. For example, when most of the aortic valves in the CT images already selected and stored are in the open phase, the selection function 153 determines that there are few images in the closed phase, and sets the above-mentioned condition that "the operating state of the aortic valve is in the closed phase."

Alternatively, the selection function 153 may set a ratio of the number of learning data between the open and closed states in advance, and automatically update the conditions at a specific time to achieve that ratio. For example, when the ratio of the open and closed states is set to 1:1, the selection function 153 initially selects all CT images without setting conditions for the operating state. After that, when a predetermined period of time has elapsed, the selection function 153 counts the number of stored learning data for each operating state. Then, when the result of the counting is that the number of learning data for the closed state is less than a predetermined number (e.g., half the number of learning data for the open state), the selection function 153 sets a condition to select data for the closed state. Alternatively, the selection function 153 may set a condition not to select data for the open state when the number of learning data for the closed state is less than a predetermined number.

Then, the selection function 153 stores the selected CT images as learning data.

For example, the selection function 153 copies and stores the selected CT image itself as learning data in the memory circuitry 120 or the medical image storage device 2. Alternatively, the selection function 153 may store only information (address) indicating the storage location of the selected CT image as learning data information, and store the CT image itself in the medical image storage device 2. Alternatively, the selection function 153 may directly set a mark (flag) in the DICOM header (e.g., a private tag, etc.) of the selected CT image to indicate that it will be used as learning data.

The information processing function 154 performs information processing using the CT images selected by the selection function 153.

In this embodiment, the information processing function 154 uses the CT images selected by the selection function 153 as learning data to perform update processing of the learning model in machine learning.

For example, the information processing function 154 determines whether or not to perform learning based on a specified condition.

For example, the information processing function 154 may set in advance conditions for determining whether or not to execute learning, and may determine whether or not to execute learning by determining whether or not the conditions are met. For example, the information processing function 154 may manage the number of pieces of learning data selected and stored by the selection function 153, and may determine to execute learning when the number of pieces of learning data exceeds a predetermined number. In this case, the number of pieces of learning data may be the total number, or may be the number of pieces of learning data corresponding to a particular operating state.

Alternatively, the information processing function 154 may receive an instruction to perform learning from a user, and when the instruction is received, determine to perform learning. At this time, for example, the information processing function 154 may receive an instruction to perform learning from a user via a predetermined user interface that is defined in advance.

Alternatively, the information processing function 154 may determine whether or not to execute learning based on the date and time. For example, the information processing function 154 may be set so that an instruction to execute learning is given on a predetermined day, day of the week, or time of each month, week, or day, and may determine to execute learning when the instruction is received. Also, for example, the information processing function 154 may have the user set the timing and conditions for executing the next learning at the time when it is determined that learning is to be executed or at the time when the update process for the learning model described below is completed.

Then, when the information processing function 154 determines to perform learning, it updates the algorithm by updating the learning model using the learning data stored by the selection function 153.

For example, as described above, if the selection function 153 writes a mark (flag) in the DICOM header of a CT image indicating that the image will be used as learning data, the information processing function 154 identifies the learning data by searching the medical image storage device 20 for CT images that have a mark set in the DICOM header.

The above describes each processing function possessed by the processing circuitry 150 of the medical data processing device 100. Here, for example, the processing circuitry 150 is realized by a processor. In this case, for example, each of the above-mentioned processing functions is stored in the memory circuitry 120 in the form of a program executable by a computer. Then, the processing circuitry 150 realizes the function corresponding to each program by reading and executing each program stored in the memory circuitry 120. In other words, when each program is read, the processing circuitry 150 has each processing function shown in FIG. 1.

Figure 3 is a flowchart showing the processing procedure performed by the processing circuit 150 of the medical data processing device 100 according to the first embodiment.

For example, as shown in FIG. 3, first, the processing circuitry 150 acquires a CT image including the aortic valve from the X-ray CT device 1 or the medical image storage device 2 (step S11). This step corresponds to the acquisition function 151. For example, the processing circuitry 150 executes this step by reading out a program corresponding to the acquisition function 151 from the storage circuitry 120 and executing it.

Next, the processing circuitry 150 identifies the operating state of the aortic valve in the acquired CT image (step S12). This step corresponds to the specific function 152. For example, the processing circuitry 150 executes this step by reading out a program corresponding to the specific function 152 from the memory circuitry 120 and executing it.

Then, the processing circuitry 150 selects a CT image based on the identified operating state (step S13, Yes) and stores the selected CT image as learning data (step S14). On the other hand, if a CT image is not selected (step S13, No), the processing circuitry 150 returns to step S11 and acquires the next CT image. These steps correspond to the selection function 153. For example, the processing circuitry 150 executes these steps by reading a program corresponding to the selection function 153 from the memory circuitry 120 and executing it.

Then, the processing circuitry 150 determines whether or not to execute learning based on a predetermined condition (step S15), and if it is determined that learning is to be executed (step S15, Yes), it updates the learning model using the stored learning data (step S16). On the other hand, if it is determined that learning is not to be executed (step S15, No), the processing circuitry 150 returns to step S11 and acquires the next CT image. These steps correspond to the information processing function 154. For example, the processing circuitry 150 executes these steps by reading a program corresponding to the information processing function 154 from the memory circuitry 120 and executing it.

As described above, in the first embodiment, the acquisition function 151 acquires CT images. Furthermore, the identification function 152 identifies the operating state of the aortic valve in the CT images acquired by the acquisition function 151. Furthermore, the selection function 153 selects CT images based on the operating state identified by the identification function 152. Then, the information processing function 154 uses the CT images selected by the selection function 153 as learning data to perform an update process of the learning model in machine learning.

Specifically, in the first embodiment, the acquisition function 151 acquires CT images of multiple time phases. The identification function 152 identifies the motion state of biological tissue in a CT image of a specific time phase included in the CT images of multiple time phases acquired by the acquisition function 151, based on the CT image of the specific time phase and data related to time phases other than the specific time phase. The selection function 153 selects CT images based on the motion state identified by the identification function 152.

According to this configuration, by selecting CT images taking into consideration the operating state of the aortic valve, it is possible to prevent bias in the CT images collected as learning data. Therefore, according to the first embodiment, it becomes possible to more appropriately perform the update process of the learning model in machine learning. As a result, for example, software using machine learning can learn and grow with an optimal amount of information, and it becomes possible to more accurately perform or support diagnosis, treatment planning, and treatment effect evaluation using medical images.

(Modification 1 of the first embodiment)
In the above-described first embodiment, an example has been described in which the identification function 152 identifies the operating state of the aortic valve in a CT image of a specified time phase based on the CT image of the specified time phase and data relating to time phases other than the specified time phase; however, the embodiment is not limited to this.

For example, the identification function 152 may identify the operating state of the aortic valve in a CT image by using only the CT image of a specific time phase acquired by the acquisition function 151.

In this case, for example, the acquisition function 151 acquires a CT image of a specified time phase including the aortic valve from the X-ray CT device 1 or the medical image storage device 2.

For example, the identification function 152 classifies the open/closed state of the aortic valve into two states: an open state and a closed state. The identification function 152 then determines which of the two states the open/closed state of the aortic valve in the CT image acquired by the acquisition function 151 corresponds to, and identifies the determined state as the operating state of the aortic valve in the CT image.

(Modification 2 of the First Embodiment)
In the above-described first embodiment, the target biological tissue is a heart valve, but the embodiment is not limited thereto. For example, the target biological tissue of the present embodiment may be any biological tissue, such as an organ or a body fluid, that has a movement or flow in the time direction.

For example, the above-described embodiment can also be applied when the target biological tissue is the lung. In this case, for example, the identification function 152 classifies the movement of the lung into two states: an inhalation state and an exhalation state. Here, the inhalation state is the state during the period when the lungs change from a deflated state to an expanded state due to breathing, and the exhalation state is the state during the period when the lungs change from an expanded state to a deflated state due to breathing. The identification function 152 then determines which of the two states the movement of the lungs in the target CT image corresponds to, and identifies the determined state as the movement state of the lungs in the CT image.

Here, various methods can be used as a method for the identification function 152 to identify the operating state of the lungs. For example, the identification function 152 may acquire respiratory information of multiple time phases from an instrument used to monitor the respiratory state of the subject during imaging, and identify the operating state of the lungs in a CT image of a specific time phase from the acquired respiratory information. Alternatively, the identification function 152 may calculate the volume of the lungs from each of the CT images of multiple time phases using a known lung area extraction technique, and identify the operating state of the lungs based on the change in the volume.

For example, the above-described embodiment can also be applied to a video swallowing contrast examination using X-rays. Here, the swallowing contrast examination is an examination of the swallowing function by having the subject (patient) swallow food mixed with a contrast agent under X-ray fluoroscopy. In this case, for example, the identification function 152 identifies the subject's motion state in the target X-ray fluoroscopy image by classifying the subject's motion during imaging in the swallowing contrast examination into three states: oral phase state, pharyngeal phase state, and esophageal phase state. In this case, for example, the identification function 152 may classify the subject's motion state based on the relative positional relationship between the position of the food mixed with the contrast agent and the position of the esophagus.

For example, the above-described embodiment can also be applied to examinations that obtain images of the operation of an examination device (such as an endoscope) rather than an organ, such as an endoscopic examination of the stomach. In this case, for example, the identification function 152 identifies the operating state of the target examination device by classifying the operation of the examination device, such as an endoscope, into an esophageal phase state, a gastric phase state, and a duodenal phase state based on the type of organ depicted in the image. In this case, for example, the identification function 152 may identify the type of organ by using known image processing technology or machine learning technology.

(Modification 3 of the first embodiment)
In the first embodiment, an example has been described in which the specification function 152 classifies the open/closed state of the aortic valve into two states, an open state and a closed state, but the embodiment is not limited to this.

Figure 4 is a diagram for explaining variant 2 of the first embodiment.

For example, as shown in Figure 4, it is known that the aortic valve does not move linearly in the opening or closing phase, but moves randomly in the opening and closing directions to transition between operating states. In other words, the aortic valve moves in such a way that it appears to flutter wildly in the opening and closing phases, respectively.

Therefore, for example, the identification function 152 may identify the operating state of the aortic valve in the target CT image by classifying the open/closed state of the aortic valve into a state during which the aortic valve is moving in the opening direction and a state during which the aortic valve is moving in the closing direction.

For example, the identification function 152 extracts the mitral valve region from each of the multiple time-phase CT images reconstructed in one examination, and deforms and aligns the annulus of the mitral valve in adjacent time phases. The identification function 152 can then calculate the movement direction and amount of movement of the leaflets by comparing the positional relationship of each leaflet in each time phase after deformation and alignment. Here, the deformation and alignment can be achieved by using a known method. For example, the affine transformation method, the FFD (Free Form Deformation) method, the LDDMM (Large Deformation Diffeomorphic Metric Mapping), etc. can be applied.

For example, the identification function 152 uses CT images of multiple time phases reconstructed in one examination to calculate the amount of movement of each leaflet of the aortic valve in each CT image from a completely closed state, thereby calculating the amount of movement of the aorta in the opening direction in each time phase. The identification function 152 then statistically analyzes the amount of movement calculated for each time phase to identify periods in which the amount of movement of the aortic valve in the opening direction tends to increase and periods in which the amount of movement tends to decrease. The identification function 152 then identifies CT images of periods in which the amount of movement of the aortic valve in the opening direction tends to increase as being in a state in which the operating state of the aortic valve is moving in the opening direction. The identification function 152 also identifies CT images of periods in which the amount of movement of the aortic valve in the opening direction tends to decrease as being in a state in which the operating state of the aortic valve is moving in the closing direction.

For example, the identification function 152 may classify the open/closed state of the aortic valve into three states: an open state, a closed state, and a closed state. In other words, since the aortic valve is in a closed state for a certain period of time, the open state is defined as the period from "the point at which the aortic valve transitions from a closed state to an open state (i.e., the point at which it opens very slightly)" to "the point at which it is in a completely open state", the closed state is defined as the period from "the point at which the aortic valve transitions from an open state to a closed state (i.e., the point at which it opens very slightly and closes the next moment)", and the closed state is defined as the period of the closed state. Then, the identification function 152 determines which of the three states the open/closed state of the aortic valve in the target CT image corresponds to, and identifies the determined state as the operating state of the aortic valve in the CT image.

(Fourth Modification of the First Embodiment)
In the above-described first embodiment, the selection function 153 selects CT images based only on the operating conditions, but the embodiment is not limited to this. For example, the selection function 153 may select CT images using conditions other than the operating conditions.

In this case, for example, the acquisition function 151 further acquires at least one of the imaging conditions, image conditions, and image quality of a CT image of a specified time phase.

Then, the selection function 153 selects CT images based on at least one of the imaging conditions, image conditions, and image quality acquired by the acquisition function 151 in addition to the operating state acquired by the identification function 152.

For example, the selection function 153 may use an imaging condition of a matrix size of 1024 x 1024 or more. This makes it possible to select only CT images whose operating state meets the condition and has high spatial resolution.

Alternatively, the selection function 153 may use the specified contrast phase as an image condition. It is generally known that the imaging ability of various organs varies greatly depending on the time that has elapsed since the contrast agent was injected into the human body. Therefore, by adding the contrast phase as a condition, it is possible to maintain a constant imaging ability of various organs in the selected CT images.

Alternatively, the selection function 153 may set conditions for selecting CT images by combining multiple such imaging conditions and image conditions.

Alternatively, the selection function 153 may use image quality as a condition. For example, the selection function 153 determines the image quality of the CT images using known image processing techniques, and selects only CT images with image quality above a certain level. This makes it possible to exclude CT images from which sufficient information about organs cannot be obtained due to noise or artifacts.

Alternatively, the selection function 153 may use information about the subject (patient) as a condition. For example, the selection function 153 sets conditions related to the subject's condition, such as age and gender. This makes it possible to construct a more unbiased learning dataset.

Alternatively, the selection function 153 may use conditions related to the pathology. Generally, most software has a target disease and is adjusted to operate appropriately for organs with that disease. However, among all patients who visit the hospital, there are more patients who do not have the disease that the software targets. Therefore, if conditions related to the pathology are not set, the software will operate with high accuracy for normal organs, but the accuracy may decrease for organs with the target disease.

Therefore, for example, the selection function 153 sets the condition that the condition is aortic valve regurgitation. In this case, the selection function 153 may obtain information on the pathology from an electronic medical record system connected via the network 3 and make a judgment based on the condition, or may automatically identify and make a judgment on the state of the aortic valve depicted in the CT image using known image processing technology.

The above various conditions may be set in any combination, or may be set by the user using a predefined user interface. In any case, this embodiment can be applied to any combination of conditions other than the operating state, and is not limited by the combination of conditions.

Fifth Modification of the First Embodiment
In addition, in the above-described first embodiment, an example has been described in which the selection function 153 sets the ratio of the number of learning data items between the open and closed states to 1:1 when targeting the aortic valve; however, the embodiment is not limited to this.

For example, the selection function 153 may change the ratio of the number of pieces of learning data based on the complexity of the operating state of the aortic valve. For example, the selection function 153 may set the ratio of the number of pieces of learning data between the open and closed states so that there is a large difference, such as 100:1.

In this case, for example, the selection function 153 may calculate the complexity of the operating state of the aortic valve and select CT images based on the calculated complexity.

It is generally known that the complexity of the movement of the aortic valve differs between the opening and closing phases, with the closing phase exhibiting more complex movements than the opening phase. Furthermore, with machine learning, the more complex the movements, the more difficult it is to learn, and the more data is required.

Therefore, for example, the selection function 153 may set the ratio of the number of learning data based on the complexity of the operation in each operating state. Specifically, when the aortic valve is the target, the selection function 153 sets the ratio of each state so that the number of learning data for the closed state is greater than the number of learning data for the open state.

Alternatively, the selection function 153 may calculate the complexity of the movement of biological tissue in the CT image using known image processing technology, and automatically set a ratio according to the calculated degree of complexity. In this case, for example, the selection function 153 extracts the target organ from the CT image, and calculates the complexity of the motion state by frequency analysis of the movement of the extracted organ.

Alternatively, the selection function 153 may use CT images of multiple target operating states to perform simple calculations (e.g., automatic calculation of diameter) through machine learning while changing the number of pieces of learning data to calculate the number of pieces of data until performance stabilizes (generally, for a target with a complex operating state, a large number of pieces of data are required until performance stabilizes), and set the ratio of the number of pieces of learning data according to the ratio of the number of pieces of data until performance stabilizes in each operating state.

(Sixth Modification of the First Embodiment)
In the above-described first embodiment, an example in which the information processing function 154 performs an update process of a learning model in machine learning has been described, but the embodiment is not limited to this. For example, the above-described embodiment is not limited to an update process of a learning model in machine learning, and can be applied to various information processing using medical data.

For example, when image processing is performed, some constraints or conditions may be set for the image processing based on statistical information of data collected in advance.

For example, the technology described in Non-Patent Document 1 enables more accurate extraction of lung regions by introducing a statistical shape model constructed from pre-collected data into the regularization term of the graph cuts algorithm.

In this way, in image processing that is performed based on previously collected data, the above-described embodiment can be applied when automatically collecting data and updating the algorithm model.

Second Embodiment
Next, a second embodiment will be described. In the following, the second embodiment will be described mainly with respect to the differences from the first embodiment, and detailed description of the contents common to the first embodiment will be omitted.

In the second embodiment, an example of augmenting learning data used in machine learning is described.

Figure 5 is a diagram showing the configuration of a medical data processing device according to the second embodiment.

For example, as shown in FIG. 5, the medical data processing device 200 according to the second embodiment includes a network interface 110, a memory circuit 120, an input interface 130, a display 140, and a processing circuit 250.

In this embodiment, the medical data processing device 200 has an acquisition function 151, a specification function 152, a selection function 153, an enhancement function 255, and an information processing function 254 as processing functions possessed by the processing circuitry 250. The enhancement function 255 is an example of an enhancement section. The information processing function 254 is an example of an information processing section.

The enhancement function 255 enhances the learning data using the CT images selected by the selection function 153 based on the operating state identified by the identification function 152.

For example, the enhancement function 255 determines whether or not to enhance the learning data for the CT images selected by the selection function 153 based on a predetermined condition related to the operating state. For example, the enhancement function 255 pre-sets an operating state that is rarely imaged in the hospital as a condition, and determines to enhance the learning data when the operating state identified by the identification function 152 matches the condition.

When it is determined that the training data should be augmented, the augmentation function 255 augments the training data by data augmentation using the CT images selected by the selection function 153. Here, data augmentation is a process of artificially increasing the training data by performing image processing on the training data, and is generally known as a method for compensating for a lack of data in machine learning. For example, methods using data augmentation include a method of increasing the training data by rotating or translating the collected images to create multiple images in which the depiction position of the organ is pseudo-changed, a method of increasing the training data by extracting only multiple images of a specific range from one image, and a method of increasing the training data by creating multiple images in which the pixel values are changed or inverted.

Note that the method by which the enhancement function 255 enhances the CT image is not limited to the above-mentioned method, and various other methods can be used.

Then, the augmentation function 255 stores the augmented learning data.

For example, the enhancement function 255 copies and stores the augmented learning data in the memory circuitry 120 or the medical image storage device 2. Alternatively, the enhancement function 255 may store only information (address) indicating the storage location of the augmented learning data, and store the learning data itself in the medical image storage device 2. Alternatively, the enhancement function 255 may directly set a mark (flag) indicating that the data will be used as learning data in the DICOM header (e.g., a private tag, etc.) of the CT image that will be the learning data.

The information processing function 254 uses the learning data augmented by the augmentation function 255 to perform update processing of the learning model.

In this embodiment, the information processing function 254 uses the learning data augmented by the augmentation function 255 to perform update processing of the learning model in machine learning, similar to the information processing function 154 described in the first embodiment.

The above describes each processing function possessed by the processing circuitry 250 of the medical data processing device 200. Here, for example, the processing circuitry 250 is realized by a processor. In this case, for example, each of the above-mentioned processing functions is stored in the memory circuitry 120 in the form of a program executable by a computer. Then, the processing circuitry 250 realizes the function corresponding to each program by reading and executing each program stored in the memory circuitry 120. In other words, when each program is read, the processing circuitry 250 has each processing function shown in FIG. 5.

Figure 6 is a flowchart showing the processing procedure performed by the processing circuit 250 of the medical data processing device 200 according to the second embodiment.

For example, as shown in FIG. 6, the processing circuit 250 performs the same processing as steps S11 to S14 described with reference to FIG. 3 in the first embodiment (steps 21 to S24).

Next, the processing circuit 150 determines whether or not to enhance the learning data based on a predetermined condition related to the operating state (step S25). If the processing circuit 250 determines that the learning data is to be enhanced (step S25, Yes), the processing circuit 250 enhances the learning data using the selected CT image (step S26) and stores the enhanced learning data (step S27). On the other hand, if the processing circuit 250 determines that the learning data is not to be enhanced (step S25, No), the processing circuit 150 proceeds to step S28 without enhancing the learning data. These steps correspond to the enhancement function 255. For example, the processing circuit 250 executes these steps by reading a program corresponding to the enhancement function 255 from the memory circuit 120 and executing it.

Then, the processing circuitry 250 determines whether or not to execute learning based on a predetermined condition (step S28), and if it is determined that learning is to be executed (step S28, Yes), it updates the learning model using the stored learning data (step S29). On the other hand, if it is determined that learning is not to be executed (step S28, No), the processing circuitry 250 returns to step S21 and acquires the next CT image. These steps correspond to the information processing function 254. For example, the processing circuitry 250 executes these steps by reading a program corresponding to the information processing function 254 from the memory circuitry 120 and executing it.

As described above, in the second embodiment, the enhancement function 255 enhances the learning data using the CT images selected by the selection function 153 based on the operating state identified by the identification function 152. In addition, the information processing function 254 uses the learning data enhanced by the enhancement function 255 to perform an update process of the learning model in machine learning.

With this configuration, for example, it is possible to increase the amount of learning data for operating states that are rarely imaged in the hospital, and to collect appropriate learning data according to the operating state.

In addition, by combining the above-mentioned variant 5 of the first embodiment with the second embodiment, it is possible to increase the amount of learning data for states with complex movements, enabling effective learning for targets that include multiple motion states with different degrees of complexity.

(Variation 1 of the second embodiment)
In the above-mentioned second embodiment, an example was described in which the enhancement function 255 determines whether or not to enhance the learning data based on certain conditions related to the operating state, but the embodiment is not limited to this.

For example, the enhancement function 255 may change the enhancement method for enhancing the learning data depending on the operating state identified by the identification function 152.

In this case, for example, the enhancement function 255 may enhance the learning data using a predetermined enhancement method (e.g., image inversion) for CT images with a certain range of operating states, and enhance the learning data using a different enhancement method (e.g., pixel value change) for data with a different range of operating states.

Note that the types of augmentation and the number of categories when the augmentation function 255 augments the learning data are not limited to the above example, and various types of augmentation and the number of categories can be used.

(Modification 2 of the second embodiment)
In addition, in the above-mentioned second embodiment, an example was described in which the enhancement function 255 enhances learning data using CT images selected by the selection function 153, but the embodiment is not limited to this.

For example, the enhancement function 255 may enhance the learning data using the CT images acquired by the acquisition function 151, instead of the CT images selected by the selection function 153, based on the operating state identified by the identification function 152. In this case, if the acquisition function 151 stores the acquired CT images as learning data, the processing circuit 250 does not need to have the selection function 153.

The configuration of the medical data processing device described in the above-mentioned embodiment and modified example may be realized by a server via a network such as the cloud. The configuration of the medical data processing device described in the above-mentioned embodiment and modified example may also be applied to a console device of a medical image diagnostic device such as an X-ray CT device or a medical image storage device. In this case, functions equivalent to the above-mentioned acquisition function, identification function, selection function, information processing function, and enhancement function are implemented in a processing circuit included in the console device of the medical image diagnostic device or a processing circuit included in the medical image storage device.

In the above-described embodiment and modified examples, the processing circuit is not limited to being realized by a single processor, but may be configured by combining multiple independent processors, with each processor executing a program to realize each processing function. Each processing function of the processing circuit may be realized by being appropriately distributed or integrated into a single or multiple processing circuits. Each processing function of the processing circuit may be realized by a mixture of hardware and software such as circuits. Here, an example has been described in which a program corresponding to each processing function is stored in a single storage circuit, but the embodiment is not limited to this. For example, a configuration in which a program corresponding to each processing function is distributed and stored in multiple storage circuits, and a processing circuit reads out and executes each program from each storage circuit may be used.

In addition, in the above-mentioned embodiment and modified example, examples have been described in which the acquisition unit, identification unit, selection unit, information processing unit, and enhancement unit in this specification are realized by the acquisition function, identification function, selection function, information processing function, and enhancement function of the processing circuit, respectively, but the embodiment is not limited to this. For example, in addition to being realized by the acquisition function, identification function, selection function, information processing function, and enhancement function described in the embodiment, the acquisition unit, identification unit, selection unit, information processing unit, and enhancement unit in this specification may also realize the same functions by hardware only, software only, or a combination of hardware and software.

The term "processor" used in the description of the above-mentioned embodiment means, for example, a circuit such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), or a programmable logic device (for example, a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). Here, instead of storing a program in a memory circuit, the program may be directly built into the circuit of the processor. In this case, the processor realizes its function by reading and executing the program built into the circuit. In addition, each processor in this embodiment is not limited to being configured as a single circuit for each processor, but may be configured as a single processor by combining multiple independent circuits to realize its function.

Here, the program executed by the processor is provided in advance in a ROM (Read Only Memory) or a storage circuit. The program may be provided in a format that can be installed in these devices or in a format that can be executed, recorded on a non-transient storage medium that can be read by a computer, such as a CD (Compact Disk)-ROM, a FD (Flexible Disk), a CD-R (Recordable), or a DVD (Digital Versatile Disk). The program may also be provided or distributed by being stored on a computer connected to a network such as the Internet and downloaded via the network. For example, the program is composed of modules including each of the above-mentioned processing functions. In terms of actual hardware, the CPU reads and executes the program from a storage medium such as a ROM, and each module is loaded onto the main storage device and generated on the main storage device.

In addition, in the above-mentioned embodiment and modified examples, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution or integration of each device is not limited to that shown in the figure, and all or part of it can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

Furthermore, among the processes described in the above-mentioned embodiments and variations, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, control procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

The various data discussed in this specification are typically digital data.

According to at least one of the embodiments described above, it is possible to more appropriately perform information processing using medical data.

Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are within the scope of the invention and its equivalents as set forth in the claims, as well as the scope and spirit of the invention.

100, 200 Medical data processing device 150, 250 Processing circuit 151 Acquisition function 152 Identification function 153 Selection function 154, 254 Information processing function 255 Amplification function

Claims (20)

An acquisition unit that acquires medical data of multiple time phases;
an identification unit that identifies a motion state of a living tissue in medical data of a predetermined time phase included in the medical data of the plurality of time phases based on the medical data of the predetermined time phase and data related to a time phase other than the predetermined time phase;
a selection unit that selects the medical data based on the motion state identified by the identification unit so as not to place too much weight on a specific motion state ;
An information processing unit that uses the medical data selected by the selection unit as learning data to perform an update process of a learning model in machine learning;
A medical data processing device comprising: and an augmenting unit that augments the learning data using the medical data selected by the selecting unit based on the operating state.
The information processing unit performs an update process of the learning model by using the learning data augmented by the augmentation unit.
The medical data processing device according to claim 1 . the biological tissue is a heart valve;
The operating state is an open/closed state of the heart valve.
3. The medical data processing device according to claim 1 or 2 . the identification unit classifies an open/closed state of the cardiac valve into two states, an open state and a closed state, determines which of the two states the open/closed state of the cardiac valve in the medical data corresponds to, and identifies the determined state as the operating state of the cardiac valve in the medical data.
The medical data processing device according to claim 3 . the identification unit classifies the open/closed state of the cardiac valve into three states, an open state, a closed state, and a closed state, determines which of the three states the open/closed state of the cardiac valve in the medical data corresponds to, and identifies the determined state as the operating state of the cardiac valve in the medical data.
The medical data processing device according to claim 3 . the biological tissue is lung;
the identification unit classifies lung motion into two states, an inhalation state and an exhalation state, determines which of the two states the lung motion in the medical data corresponds to, and identifies the determined state as the lung motion state in the medical data.
3. The medical data processing device according to claim 1 or 2 . the selection unit changes a ratio of the number of medical data to be selected between the first motion state and the second motion state such that the number of medical data in a second motion state, which is a more complicated motion than the first motion state, is greater than the number of medical data in the first motion state;
The medical data processing device according to any one of claims 1 to 6 . the selection unit uses medical data of the biological tissue in a plurality of motion states as learning data, and performs a predetermined calculation by machine learning while changing the number of the learning data to calculate the number of data until the performance is stabilized, and changes the ratio of the number of the medical data to be selected between each motion state according to the ratio of the number of data until the performance is stabilized in each motion state ;
7. The medical data processing device according to claim 1 . The acquisition unit further acquires at least one of imaging conditions, image conditions, and image quality of the medical data of the predetermined time phase;
The selection unit selects the medical data based on at least one of the imaging conditions, the image conditions, and the image quality in addition to the operating state.
The medical data processing device according to any one of claims 1 to 8 . The selection unit uses a designated contrast phase as the image condition.
The medical data processing device according to claim 9 . The selection unit selects the medical data by further using information on the subject as a condition in addition to the operating state.
The medical data processing device according to claim 9 . The selection unit selects the medical data by using conditions related to a pathological condition in addition to the operating state.
The medical data processing device according to claim 9 . The data relating to a time phase other than the predetermined time phase is data acquired by the same modality as the modality used to acquire the medical data of the predetermined time phase.
The medical data processing device according to any one of claims 1 to 12 . the identification unit uses the medical data of the multiple time phases to estimate a moving direction of the biological tissue from a position of the biological tissue in the medical data of the predetermined time phase and a position of the biological tissue in the medical data of a time phase adjacent to the predetermined time phase, and identifies a motion state of the biological tissue in the medical data of the predetermined time phase based on a result of the estimation.
The medical data processing device according to claim 13 . The data relating to a time phase other than the predetermined time phase is data acquired by a means other than the modality that acquired the medical data of the predetermined time phase.
The medical data processing device according to any one of claims 1 to 12 . the identifying unit identifies a motion state of the biological tissue in the medical data of the predetermined time phase based on electrocardiogram information or phonocardiogram information collected when the medical data of the multiple time phases is collected as data related to a time phase other than the predetermined time phase;
The medical data processing device according to claim 15 . the identification unit identifies a motion state of the biological tissue in the medical data of the predetermined time phase based on information of biological tissue other than the biological tissue in the medical data of the predetermined time phase as data related to a time phase other than the predetermined time phase;
The medical data processing device according to claim 15 . An acquisition unit for acquiring medical data;
An identification unit that identifies a motion state of biological tissue in the medical data;
a selection unit that selects the medical data based on the motion state identified by the identification unit so as not to place too much weight on a specific motion state ;
An information processing unit that uses the medical data selected by the selection unit as learning data to perform an update process of a learning model in machine learning ;
A medical data processing device comprising: the biological tissue is a heart valve;
the identification unit classifies an open/closed state of the cardiac valve into two states, an open state and a closed state, determines which of the two states the open/closed state of the cardiac valve in the medical data corresponds to, and identifies the determined state as the operating state of the cardiac valve in the medical data.
20. The medical data processing device according to claim 18. Acquire medical data from multiple time phases,
Identifying a motion state of living tissue in medical data of a predetermined time phase included in the medical data of the plurality of time phases based on the medical data of the predetermined time phase and data relating to a time phase other than the predetermined time phase;
Selecting the medical data based on the identified operating state so as not to give weight to a specific operating state ;
The selected medical data is used as learning data to update the learning model in machine learning.
A medical data processing method comprising:

Images