KR20220060127A - Method for predicting prognosis based on deep-learning and apparatus thereof - Google Patents

Abstract

Provided are a method for predicting the prognosis based on deep learning and a device thereof. According to some embodiments of the present disclosure, the method for predicting the prognosis based on deep learning includes the steps of: acquiring a pathological image of a patient with cancer; performing semantic segmentation on the pathological image using a deep learning model, wherein the semantic segmentation class includes a cancer class and a stroma class; and predicting the prognosis or prognosis factors of the patient based on the result of the semantic segmentation.

Description

Deep learning-based prognosis prediction method and device {METHOD FOR PREDICTING PROGNOSIS BASED ON DEEP-LEARNING AND APPARATUS THEREOF}

The present disclosure relates to a deep learning-based prognostic prediction method and an apparatus therefor. More particularly, it relates to a method capable of predicting a prognosis and/or a prognostic factor of a target cancer using a deep learning model, and an apparatus for performing the method.

Colorectal cancer (CRC) is the third most common cancer worldwide and the second leading cause of cancer-related death.

In colorectal cancer, lymph node metastasis (LNM) is an important prognostic factor for predicting a patient's disease recurrence, survival, and treatment methods. For example, the presence or absence of lymph node metastasis can determine whether early stage patients need additional surgery after local endoscopic treatment, and whether adjuvant chemotherapy is needed after surgical resection.

However, current clinicians are making important treatment decisions by estimating lymph node metastasis based on the evaluation of nodule status based on limited tests such as ultrasound and computed tomography (CT), light microscopic examination results, and subjective experience. However, it is difficult to accurately predict whether or not lymph node metastasis in a colorectal cancer patient is achieved only by such pathological examination. For example, it is almost impossible to predict micro-scale lymph node metastasis through pathological examination. Moreover, inexperienced clinicians may make erroneous treatment decisions due to the difficulty of prediction.

Korean Patent Publication No. 10-2019-0021471 (published on March 5, 19)

A technical problem to be solved through some embodiments of the present disclosure is to provide a method capable of predicting a prognosis and/or a prognostic factor of a target cancer using a deep learning model, and an apparatus for performing the method.

Another technical problem to be solved through some embodiments of the present disclosure is to provide an index capable of accurately predicting a prognosis and/or a prognostic factor of a target cancer.

The technical problems of the present disclosure are not limited to the above-mentioned technical problems, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above technical problem, a deep learning-based prognosis prediction method according to some embodiments of the present disclosure is a method for predicting the prognosis of a target cancer based on deep learning in a computing device, and the pathology of the target cancer patient Acquiring an image, performing semantic segmentation on the pathological image using a deep learning model, wherein the segmentation class includes an arm class and a stroma class, and based on the result of the semantic segmentation to predict the patient's prognosis or prognostic factors.

In some embodiments, the deep learning model may include an encoder that performs downsampling and a decoder that performs upsampling.

In some embodiments, further comprising the step of training the deep learning model using an image patch for training, wherein the training includes whether the size of the image patch for training meets the input condition of the deep learning model. Determining, in response to the determination that does not match, performing mirror padding on the image patch for training and training the deep learning model using the mirror-padded image patch for training. can

In some embodiments, the predicting of the prognosis or prognostic factor of the patient may include determining a dark region and a stromal region from the result image of the semantic segmentation, and a predictive index based on a ratio of the dark region and the stromal region. It may include calculating a score and predicting a prognosis or prognostic factor of the patient based on the predictive index score.

In some embodiments, calculating the prediction index score may include calculating the prediction index score based on a ratio of the number of pixels in the dark region to the number of pixels in the stroma region.

In some embodiments, the determining of the dark region and the stroma region may include extracting a region corresponding to the dark class and the stroma class from the result image, performing an erosion operation on the extracted region, and It may include performing a morphology operation including a dilation operation and determining the dark region and the stroma region based on a result of the morphology operation.

In some embodiments, the prognostic factor may include whether lymph node metastasis.

In some embodiments, the target cancer may be colorectal cancer.

Prognosis prediction apparatus according to some embodiments of the present disclosure for solving the above technical problem, a memory for storing one or more instructions, and a pathological image for a patient of target cancer by executing the one or more instructions obtain, perform semantic segmentation on the pathological image using a deep learning model, and predict a prognosis or prognostic factor of the patient based on the result of the semantic segmentation. It may include a processor. In this case, the target class of the semantic segmentation may include an arm class and a stroma class.

A computer program according to some embodiments of the present disclosure for solving the above-described technical problem is combined with a computing device, obtaining a pathological image of a patient of target cancer, and using a deep learning model to In order to perform semantic segmentation on the patient, the segmentation class includes a cancer class and a stroma class, and predicting a prognosis or prognostic factor of the patient based on the result of the semantic segmentation. It may be stored in a computer-readable recording medium.

According to some embodiments of the present disclosure described above, the ratio of the cancer region to the stromal region may be used as an objective predictive index for the prognosis and/or prognostic factors of target cancer. These predictive indicators can be effectively used to predict whether lymph node metastasis, which is a major prognostic factor of target cancer, etc., and thus provide valuable and objective diagnostic auxiliary information to clinicians of cancer patients.

In addition, by using a deep learning model that performs semantic segmentation, the score regarding the predictive index can be accurately calculated from the patient's pathological image in an automated manner.

Effects according to the technical spirit of the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is an exemplary diagram for describing an apparatus for predicting a prognosis and input/output data thereof according to some embodiments of the present disclosure.
2 is an exemplary flowchart illustrating a deep learning-based prognosis prediction method according to some embodiments of the present disclosure.
3 is an exemplary diagram for explaining the concept of semantic segmentation that may be referred to in some embodiments of the present disclosure.
4 and 5 are exemplary views for explaining the structure of a deep learning model according to some embodiments of the present disclosure.
6 is an exemplary diagram for describing a mirror padding technique that may be referred to in some embodiments of the present disclosure.
7 illustrates a pathological image that may be referenced in some embodiments of the present disclosure.
8 illustrates the result of semantic segmentation for the pathological image illustrated in FIG. 7 .
9 is an exemplary flowchart illustrating a detailed process of step S400 of FIG. 2 .
FIG. 10 is an exemplary diagram for further explaining step S420 of FIG. 9 .
11 is an exemplary diagram for explaining a method for calculating a prediction index score according to a first embodiment of the present disclosure.
12 is an exemplary view for explaining a method of calculating a prediction index score according to a second embodiment of the present disclosure.
13 illustrates an exemplary computing device that may implement a prognosis prediction apparatus according to some embodiments of the present disclosure.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Advantages and features of the present disclosure, and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the technical spirit of the present disclosure is not limited to the following embodiments, but may be implemented in various different forms, and only the following embodiments complete the technical spirit of the present disclosure, and in the technical field to which the present disclosure belongs It is provided to fully inform those of ordinary skill in the art of the scope of the present disclosure, and the technical spirit of the present disclosure is only defined by the scope of the claims.

In adding reference numerals to the components of each drawing, it should be noted that the same components are given the same reference numerals as much as possible even though they are indicated on different drawings. In addition, in describing the present disclosure, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present disclosure, the detailed description thereof will be omitted.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which this disclosure belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly defined in particular. The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present disclosure. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase.

In addition, in describing the components of the present disclosure, terms such as first, second, A, B, (a), (b), etc. may be used. These terms are only for distinguishing the elements from other elements, and the essence, order, or order of the elements are not limited by the terms. When it is described that a component is “connected”, “coupled” or “connected” to another component, the component may be directly connected or connected to the other component, but another component is formed between each component. It should be understood that elements may also be “connected,” “coupled,” or “connected.”

As used herein, “comprises” and/or “comprising” refers to a referenced component, step, operation and/or element of one or more other components, steps, operations and/or elements. The presence or addition is not excluded.

Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

1 is an exemplary diagram for explaining a prognosis prediction apparatus 10 and input/output data thereof according to some embodiments of the present disclosure.

As shown in FIG. 1 , the prognosis prediction apparatus 10 is a computing device, which receives patient information 1 of a target cancer and predicts the prognostic factors 2 and 3 of the corresponding patient. can be output (provided). In addition, the prognosis prediction apparatus 10 may determine and output (provide) the treatment information 4 for the patient. Hereinafter, for convenience of description, the prognosis prediction apparatus 10 will be abbreviated as "prediction apparatus 10".

The computing device may be, for example, a notebook computer, a desktop computer, a laptop computer, and the like, but is not limited thereto and may include any type of device having a computing function. Refer to FIG. 13 for an example of a computing device.

The target cancer may be, for example, colorectal cancer. However, the scope of the present disclosure is not limited thereto, and the target cancer may further include other types of cancer, such as breast cancer.

The patient information 1 may include, for example, demographic information of a patient, a pathological image of the patient, a disease history of the patient, and the like. However, the present invention is not limited thereto. The pathology image may include, for example, a whole slide image (e.g. hematoxylin & eosin (H&E) staining image) of a cancer (tumor) site (tissue) and/or a patch of the whole slide image. However, the present invention is not limited thereto.

The prognostic factor information 12 may include, for example, whether lymph node metastasis, extramural tumor deposits (EMTD), tumor size, tumor location, tumor invasion depth, and the like. However, the present invention is not limited thereto.

The prognostic information 13 may include, for example, all information related to the prognosis of the target cancer, such as the possibility of recurrence, the possibility of survival, the progression stage of the disease, the disease progression state, and the like.

The treatment information 14 may include, for example, information on an appropriate treatment method according to the prognosis, desirable living rules, and the like. However, the present invention is not limited thereto.

In various embodiments of the present disclosure, the prediction device 10 may predict a prognosis and/or a prognostic factor of a patient with target cancer by using a deep learning model. Specifically, the prediction device 10 analyzes the patient's pathological image through a deep-learning model that performs semantic segmentation to obtain a predictive index score for the patient's prognosis and/or prognostic factors. can be calculated. In addition, the prediction apparatus 10 may predict a patient's prognosis and/or prognostic factors based on the calculated prediction index score. In relation to this embodiment, it will be described in detail with reference to FIG. 2 and the following drawings.

Meanwhile, although FIG. 1 illustrates that the prediction device 10 is implemented as one computing device as an example, the prediction device 10 may be implemented with a plurality of computing devices. For example, a first function of the prediction device 10 may be implemented in a first computing device, and a second function may be implemented in a second computing device. Alternatively, a specific function of the prediction device 10 may be implemented in a plurality of computing devices.

So far, the prediction apparatus 10 according to some embodiments of the present disclosure has been described with reference to FIG. 1 . Hereinafter, a deep learning-based prognostic prediction method that can be performed in the prediction apparatus 10 illustrated in FIG. 1 will be described.

Each step of the prognostic prediction method to be described below may be implemented as one or more instructions that can be executed by the processor of the computing device, and when the subject of a specific step (operation) is omitted, the prediction device 10 It can be understood to be carried out by However, in some cases, some steps of the prognosis prediction method may be performed in another computing device.

2 is an exemplary flowchart illustrating a deep learning-based prognosis prediction method according to some embodiments of the present disclosure. However, this is only a preferred embodiment for achieving the purpose of the present disclosure, and it goes without saying that some steps may be added or deleted as needed.

As shown in FIG. 2 , the prognostic prediction method may be started in step S100 of training a deep learning model that performs semantic segmentation. For convenience of understanding, the concept of semantic segmentation, the structure of a deep learning model, and a learning method will be described in detail with reference to FIGS. 3 to 6 .

First, semantic segmentation may refer to a task of predicting a class in units of pixels, not images. For example, as shown in FIG. 3 , when semantic segmentation is performed on a given image 13, a segmentation map 15 or a segmentation result image 17 that is a pixel-wise class prediction result is obtained. can The segmentation result image 17 may mean representing the segmentation map 15 in the form of an image, but is not limited thereto. Since semantic segmentation is a task of performing dense prediction on a pixel-by-pixel basis, it may be referred to as a dense prediction in the art.

Semantic segmentation is the most essential and difficult task among various tasks in the computer vision field, and can be performed through a deep learning model for high accuracy. For example, as shown in FIG. 4 , semantic segmentation may be performed through a deep learning model composed of an encoder 21 and a decoder 25 . Here, the encoder 21 is a neural network that performs downsampling, and may include, for example, a neural network that performs a convolution operation, but is not limited thereto. In addition, the decoder 25 is a neural network that performs upsampling on the encoding result 23, for example, it may be composed of a neural network that performs an up-convolution (or deconvolution) operation, However, the present invention is not limited thereto.

The structure of a more specific deep learning model is illustrated in FIG. 5 . 5 illustrates a deep learning model composed of an encoder 31 and a decoder 33 and conceptually composed of a U-shaped structure. This deep learning model further utilizes the feature map extracted from the encoder 31 in the decoder 33 (e.g. it is combined with the feature map of the encoder 31 and the feature map of the decoder 33 and utilized during decoding), high segmentation accuracy can guarantee For a more detailed description of the deep learning model illustrated in FIG. 5 , refer to the known technical data about "U-Net".

The above-described deep learning model can be trained using training data (sets) composed of a pathological image (or image patch) for training and a correct answer segmentation map. In this case, the segmentation class may include a cancer class (e.g. cancer epithelial cell/tissue) and a stroma class (e.g. stroma cell/tissue), and may further include other classes. For example, the segmentation class is cancer class, stroma class, normal class (e.g. normal epithelial cells/tissue), muscle class (e.g. myocytes), adipose class (e.g. adipocytes/tissue), lymphoid class (e.g. lymphocytes) ) and the mucus class. Those skilled in the art will be fully familiar with the deep learning model learning method (e.g. error backpropagation technique), so a detailed description of the learning method will be omitted.

In some embodiments, mirror padding may be performed on the pathological image patch for training, and a deep learning model may be trained using the mirror-padded pathological image patch. For example, it may be determined whether the size of the pathological image patch meets the input condition of the deep learning model, and in response to the determination that the size of the pathological image patch does not match, mirror padding may be performed on the pathological image patch. Through this process, even if the size of the pathological image patch for training does not match the deep learning model, learning can be performed smoothly. In addition, since size restrictions on the pathological image patch for training are relaxed, training data (sets) can be more easily secured.

For reference, the mirror padding may refer to a technique of performing padding by mirroring pixel values of the original image 41 as illustrated in FIG. 6 . For example, a pixel value of one region 45 of the mirror-padded image 47 may be filled in such a way that one region 43 of the original image 41 is reflected through a mirror. Mirror padding can be effectively utilized when machine learning images in which similar structures (e.g. shape, shape) appear repeatedly, such as pathological images or biological images. In the art, mirror padding may be used interchangeably with the term reflect padding.

It will be described again with reference to FIG. 2 .

In step S200, a pathological image of the patient may be acquired. As mentioned above, the pathology image may include an entire slide image of the cancer site and/or a patch thereof. Refer to FIG. 7 for an example of a pathological image. 7 illustrates an entire slide image of a cancer site of a patient.

In step S300, semantic segmentation may be performed on the pathological image using the deep learning model. As a result of the semantic segmentation, the pathological image of the patient may be divided into a plurality of regions corresponding to the segmentation class. In this case, the segmentation class may include an arm class and a stroma class as described above. In addition, the segmentation class may further include a normal class, a muscle class, a fat class, a lymph class, and a slime class.

An example of the semantic segmentation result will be referred to FIG. 8 . 8 shows the segmentation result image in which each class area is displayed in different colors. In the illustrated image, purple, green, sky blue, orange, red, blue and yellow are cancer class, stroma class, normal class, and muscle class, respectively. , fat class, lymph class, and mucus class.

In some embodiments, the pathological image may be divided into a plurality of image patches, and semantic segmentation may be performed by applying a deep learning model to each of the plurality of image patches. That is, when the pathological image is a high-resolution image such as an entire slide image, the pathological image may be applied to the deep learning model after the pathological image is divided into a plurality of image patches according to the input size of the deep learning model.

In step S400 , a prognosis and/or a prognostic factor of the patient may be predicted based on the semantic segmentation result. An example of the detailed process of this step is shown in FIG. 9 . Hereinafter, this step will be described in more detail with reference to FIG. 9 .

In step S420, a dark region and a stroma region may be determined (extracted) from the segmentation result image (refer to FIG. 8). For example, the prediction apparatus 10 may extract only regions corresponding to the cancer class and the stroma class from the segmentation result image.

In some embodiments, a morphology operation may be performed on the extracted dark region and stroma region. For example, as shown in FIG. 11 , the prediction apparatus 10 may perform a dilation operation and an erosion operation. 11 illustrates that the dilation operation and the erosion operation are sequentially performed as an example, but the dilation operation and the erosion operation may be performed in an order different from that illustrated in FIG. 11 . In addition, other types of morphological operations may be further performed. According to the present embodiment, noise on the image may be removed through morphological operation, and thus the dark region and the stromal region may be more accurately determined.

Also, in some embodiments, some regions among the extracted dark region and stroma region may be removed. In detail, the prediction apparatus 10 may perform a process of removing pixels having a confidence score of a deep learning model equal to or less than a reference value from the extracted dark region and stroma region. For example, the prediction apparatus 10 may determine the final dark area by removing pixels having a confidence score for the dark class equal to or less than a reference value in the dark area. Alternatively, the prediction device 10 may determine the final stroma region by removing pixels having a confidence score for a stroma class equal to or less than a reference value in the stroma region. Through this removal processing, the effect of segmentation error of the deep learning model on the prognosis of the patient and/or the prediction of prognostic factors can be minimized.

In step S440, a prediction index score may be calculated based on the ratio of the dark region and the stromal region. That is, the ratio of cancer area to stromal area can be used as an indicator to predict prognostic factors such as lymph node metastasis or the prognosis of colorectal cancer.

The reason for using the ratio between the dark area and the stroma area as a predictive index is as follows. In the tumor micro-environment, stroma is located in the periphery of cancer cells (tissues) and is estimated to be partially related to the growth and metastasis of cancer cells (tissues). The inventors of the present disclosure analyzed clinical data of cancer patients based on these estimated facts, and as a result of the analysis, it could be found that the ratio of cancer area to stromal area was closely related to prognostic factors such as lymph node metastasis (see below). See Table 1). It can be understood that the reason for using the ratio of the dark area and the stromal area as a predictive index stems from the results of this study.

In this step S440, a specific method of calculating the prediction index score may vary depending on the embodiment.

In the first embodiment, as shown in FIG. 11 , the prediction index score 50 may be calculated based on the ratio of the number of pixels in the dark region C to the number of pixels in the stroma region S. The results of analyzing clinical data according to this calculation method are shown in Table 1 below. Table 1 below shows the results of calculating the predictive index score (50) using the pathological images of colorectal cancer patients consisting of 98 LNM-negative patients and 66 LNM-positive patients. are displaying Referring to Table 1 below, it can be seen that the average value of the predictive index score (50) is significantly different between the positive and negative patients with lymph node metastasis. Therefore, it can be seen from Table 1 below that the predictive index score 50 can effectively predict whether or not metastasis of colorectal cancer to lymph nodes occurs.

division Average Standard Deviation LNM-negative 0.228 0.160 LNM-positive 0.380 0.285

In the second embodiment, the prediction index score may be calculated by using the confidence score of the deep learning model as a pixel weight. Specifically, the first value for the dark region is calculated by reflecting the first pixel weight to the number of pixels in the dark region, and the second value for the stroma region is calculated by reflecting the second pixel weight to the number of pixels in the stroma region and a prediction index score may be calculated based on the ratio of the first value to the second value. In this case, the first pixel weight may be determined based on the confidence score for the dark class of the corresponding pixel, and the second pixel weight may be determined based on the confidence score for the stroma class of the corresponding pixel. For the convenience of understanding, the present embodiment will be described in more detail with reference to the example shown in FIG. 12. FIG. 12 shows the prediction index score 50 calculated according to the first and second embodiments. , 60), the first prediction index score 50 calculated according to the first embodiment is shown on the left, and the second prediction index score 60 calculated according to the second embodiment is shown on the right is shown in

As shown in FIG. 12 , it is assumed that the number of pixels in the dark region and the stroma region is equal to two, and the confidence score for each class of each pixel is as shown on the right. In this case, the first prediction index score 50 may be calculated as 1 (= 2/2). On the other hand, the second prediction index score 60 can be calculated as 1.3 (= 1 * 0.7 + 1 * 0.6) / 1.7 (= 1 * 0.8 + 1 * 0.9), which is about the segmentation result of the deep learning model. Reliability (= confidence score) may be understood as a result of being reflected as a weight. According to this second embodiment, since the prediction index score is calculated by reflecting the reliability of the segmentation result of the deep learning model, the influence of the segmentation error of the deep learning model on the prediction index score can be effectively limited.

It will be described again with reference to FIG. 9 .

In step S460, a prognosis and/or a prognostic factor of the patient may be predicted based on the predictive index score. For example, the prediction device 10 may predict that lymph node metastasis does not exist in response to the determination that the prediction index score is less than or equal to a reference value. Conversely, when the prediction index score is equal to or greater than the reference value, the prediction apparatus 10 may determine that lymph node metastasis exists.

The reference value may be set to an appropriate value based on the analysis result of clinical data, or may be set to a subdivided numerical value according to the type of target cancer, demographic information of the patient, the type of predictor, and the like.

In some embodiments, a prognosis and/or a prognostic factor of a patient may be predicted based on a change in the predictive index score over time. For example, the prediction device 10 may predict the rate of lymph node metastasis or the rate of progression of a target cancer based on the degree of change (e.g. slope) of the predictive index score over time. As a more specific example, when the slope of change of the prediction index score is equal to or greater than the reference value, the prediction apparatus 10 may predict that the lymph node metastasis rate is fast.

So far, a deep learning-based prognosis prediction method according to some embodiments of the present disclosure has been described with reference to FIGS. 2 to 12 . According to the above-described method, the ratio of the cancer region to the stromal region can be used as an objective predictive index for the prognosis and/or prognostic factors of the target cancer. These predictive indicators can be effectively used to predict whether lymph node metastasis, which is a major prognostic factor of target cancer, etc., and thus provide valuable and objective diagnostic auxiliary information to clinicians of cancer patients. In addition, by using a deep learning model that performs semantic segmentation, the predictive index score can be accurately calculated from the patient's pathological image in an automated manner.

Hereinafter, an exemplary computing device 1000 capable of implementing the prediction device 10 according to some embodiments of the present disclosure will be described.

13 is an exemplary hardware configuration diagram illustrating the computing device 1000 .

As shown in FIG. 13 , the computing device 1000 includes one or more processors 1100 , a bus 1300 , a communication interface 1500 , and a memory for loading a computer program executed by the processor 1100 (load). 1200 ) and a storage 1500 for storing the computer program 1600 . However, only components related to the embodiment of the present disclosure are illustrated in FIG. 13 . Accordingly, those skilled in the art to which the present disclosure pertains can see that other general-purpose components other than the components shown in FIG. 13 may be further included. That is, the computing device 1000 may further include various components in addition to the components illustrated in FIG. 13 . Also, in some cases, the computing device 1000 may be configured in a form in which some of the components illustrated in FIG. 13 are omitted. Hereinafter, each component of the computing device 1000 will be described.

The processor 1100 controls the overall operation of each component of the computing device 1000 . The processor 1100 includes at least one of a central processing unit (CPU), a micro processor unit (MPU), a micro controller unit (MCU), a graphic processing unit (GPU), or any type of processor well known in the art of the present disclosure. may be included. Also, the processor 1100 may perform an operation on at least one application or program for executing the operation/method according to the embodiments of the present disclosure. The computing device 1000 may include one or more processors.

Next, the memory 1200 stores various data, commands and/or information. The memory 1200 may load one or more programs 1600 from the storage 1500 to execute operations/methods according to embodiments of the present disclosure. The memory 1200 may be implemented as a volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

Next, the bus 1300 provides a communication function between components of the computing device 1000 . The bus 1300 may be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

Next, the communication interface 1500 supports wired/wireless Internet communication of the computing device 1000 . Also, the communication interface 1500 may support various communication methods other than Internet communication. To this end, the communication interface 1500 may be configured to include a communication module well-known in the technical field of the present disclosure. In some cases, the communication interface 1500 may be omitted.

Next, the storage 1500 may non-temporarily store the one or more computer programs 1600 . The storage 1500 is a non-volatile memory, such as a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, or well in the art to which the present disclosure pertains. It may be configured to include any known computer-readable recording medium.

Next, the computer program 1600 may include one or more instructions that, when loaded into the memory 1200 , cause the processor 1100 to perform an operation/method according to various embodiments of the present disclosure. That is, the processor 1100 may perform the operation/method according to various embodiments of the present disclosure by executing the one or more instructions.

For example, the computer program 1600 may perform an operation of acquiring a pathological image of a patient of target cancer, an operation of performing semantic segmentation on the pathological image acquired using a deep learning model, and a patient based on the result of the semantic segmentation. It may include instructions including an operation of predicting a prognosis and/or a prognostic factor. In this case, the prediction device 10 according to some embodiments of the present disclosure may be implemented through the computing device 1000 .

The technical idea of the present disclosure described with reference to FIGS. 1 to 13 may be implemented as computer-readable codes on a computer-readable medium. The computer-readable recording medium may be, for example, a removable recording medium (CD, DVD, Blu-ray disk, USB storage device, removable hard disk) or a fixed recording medium (ROM, RAM, computer-equipped hard disk). can The computer program recorded on the computer-readable recording medium may be transmitted to another computing device through a network such as the Internet and installed in the other computing device, thereby being used in the other computing device.

In the above, even though all components constituting the embodiment of the present disclosure are described as being combined or operated in combination, the technical spirit of the present disclosure is not necessarily limited to this embodiment. That is, within the scope of the present disclosure, all of the components may operate by selectively combining one or more.

Although acts are shown in a particular order in the drawings, it should not be understood that the acts must be performed in the specific order or sequential order shown, or that all illustrated acts must be performed to obtain a desired result. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of the various components in the embodiments described above should not be construed as necessarily requiring such separation, and the described program components and systems may generally be integrated together into a single software product or packaged into multiple software products. It should be understood that there is

Although embodiments of the present disclosure have been described above with reference to the accompanying drawings, those of ordinary skill in the art to which the present disclosure pertains may practice the present disclosure in other specific forms without changing the technical spirit or essential features. can understand that there is Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The protection scope of the present disclosure should be interpreted by the following claims, and all technical ideas within the equivalent range should be interpreted as being included in the scope of the technical ideas defined by the present disclosure.

Claims (12)

A method for predicting the prognosis of target cancer based on deep learning in a computing device, the method comprising:
acquiring a pathological image of the patient of the target cancer;
performing semantic segmentation on the pathological image using a deep learning model, wherein the segmentation class includes a cancer class and a stroma class; and
Comprising the step of predicting the prognosis or prognostic factor of the patient based on the result of the semantic segmentation,
Deep learning-based prognostic prediction method. According to claim 1,
The deep learning model includes an encoder performing downsampling and a decoder performing upsampling,
Deep learning-based prognostic prediction method. According to claim 1,
Further comprising the step of learning the deep learning model using the image patch for training,
The learning step is
determining whether the size of the training image patch meets the input condition of the deep learning model;
performing mirror padding on the training image patch in response to the determination that it does not match; and
Including the step of learning the deep learning model using the mirror-padded image patch for training,
Deep learning-based prognostic prediction method. According to claim 1,
The step of performing the semantic segmentation comprises:
segmenting the pathological image into a plurality of image patches; and
Comprising the step of performing the semantic segmentation by applying the plurality of image patches to the deep learning model,
Deep learning-based prognostic prediction method. According to claim 1,
Predicting the prognosis or prognostic factors of the patient comprises:
determining a dark region and a stromal region in the result image of the semantic segmentation;
calculating a prediction index score based on the ratio of the dark region to the stromal region; and
Comprising the step of predicting the prognosis or prognostic factors of the patient based on the predictive index score,
Deep learning-based prognostic prediction method. 6. The method of claim 5,
Calculating the predictive index score comprises:
Comprising the step of calculating the prediction index score based on the ratio of the number of pixels in the dark region to the number of pixels in the stroma region,
Deep learning-based prognostic prediction method. 6. The method of claim 5,
The step of determining the dark region and the stroma region,
extracting a region corresponding to the cancer class and the stroma class from the result image;
performing a morphology operation including an erosion operation and a dilation operation on the extracted region; and
Comprising the step of determining the dark region and the stroma region based on the result of the morphology operation,
Deep learning-based prognostic prediction method. 6. The method of claim 5,
The prognostic factors include whether or not lymph node metastasis,
Deep learning-based prognostic prediction method. 6. The method of claim 5,
Calculating the predictive index score comprises:
calculating a first value for the dark area by reflecting a first pixel weight to the number of pixels in the dark area;
calculating a second value for the stroma region by reflecting a second pixel weight to the number of pixels in the stroma region; and
Comprising the step of calculating the prediction index score based on the ratio of the first value and the second value,
The first pixel weight is determined based on a confidence score for an arm class of the corresponding pixel;
The second pixel weight is determined based on the confidence score for the stroma class of the corresponding pixel,
Deep learning-based prognostic prediction method. According to claim 1,
The target cancer is colon cancer,
Deep learning-based prognostic prediction method. a memory storing one or more instructions;
By executing the one or more instructions,
Obtaining a pathological image of a patient of target cancer,
Performing semantic segmentation on the pathological image using a deep learning model,
A processor for predicting a prognosis or a prognostic factor of the patient based on the result of the semantic segmentation,
The target class of the semantic segmentation includes an arm class and a stroma class,
prognostic device. combined with a computing device,
acquiring a pathological image of a patient of target cancer;
performing semantic segmentation on the pathological image using a deep learning model, wherein the segmentation class includes a cancer class and a stroma class; and
stored in a computer-readable recording medium for executing the step of predicting a prognosis or a prognostic factor of the patient based on the result of the semantic segmentation,
computer program.

Images