CN116344004B

CN116344004B - Image sample data amplification method and device

Info

Publication number: CN116344004B
Application number: CN202310634382.2A
Authority: CN
Inventors: 冯奕乐; 孙步梁
Original assignee: Suzhou Hengrui Hongyuan Medical Technology Co ltd
Current assignee: Suzhou Hengrui Hongyuan Medical Technology Co ltd
Priority date: 2023-05-31
Filing date: 2023-05-31
Publication date: 2023-08-08
Anticipated expiration: 2043-05-31
Also published as: CN116344004A

Abstract

The present invention relates to the field of artificial intelligence image processing technology, and in particular, to a method, an apparatus, a computer device, a storage medium and a computer program product for amplifying image sample data. The method comprises the following steps: acquiring an image dataset; constructing and generating an countermeasure network model; training the generated countermeasure network model to be converged by using the image data set, and extracting a target generation network for synthesizing a first image sample into the second image sample from the generated countermeasure network model; and synthesizing the first image sample in the image data set into a synthesized second image sample by utilizing a target generation network, and matching and splicing the synthesized image sample and the labeling information of the first image sample to obtain a third image sample. By adopting the method, the marked image data and the unmarked image data can be fused in an image synthesis mode, so that the aim of amplifying training sample data is fulfilled, and the accuracy of a finally obtained algorithm model is improved.

Description

Image sample data amplification method and device

Technical Field

The present application relates to the field of artificial intelligence image processing technology, and in particular, to a method, an apparatus, a computer device, a storage medium, and a computer program product for image sample data.

Background

Medical imaging refers to techniques and procedures for non-invasively acquiring images of internal tissues of a human body or a portion of a human body for medical purposes. The medical image can be used for checking and treating various diseases in application, and can provide great convenience for medical staff. Among them, computerized tomography (CT, computed Tomography) and magnetic resonance imaging (MRI, magnetic Resonance Imaging) are important components of medical imaging techniques.

In medical applications, artificial intelligence techniques are often introduced to assist in implementing CT image-based image segmentation techniques in order to improve the analysis efficiency of the image techniques. In order to train a liver segmentation artificial intelligence algorithm model based on flat scan CT images, a large amount of manually marked training data is required to be prepared in the early stage of training. However, under the flat scan CT, the difficulty of manually marking the liver mask is relatively high due to the problems of low liver resolution, unclear texture and the like, and the time and labor costs are far higher than those of other image modes. Accordingly, in the public database, a large number of data sets manually marked by a professional physician, usually data sets based on enhanced CT or magnetic resonance imaging, are almost difficult to find a flat scan CT liver segmented image data set which can be directly used.

Currently, in order to solve the problem of difficulty in data acquisition in liver organ segmentation tasks based on flat scan CT, it is generally necessary to perform amplification processing on the total number of training samples based on existing data. In the field of artificial intelligence image segmentation, conventional training data amplification techniques only perform a series of geometric transformations or numerical transformations on manually labeled images, such as translation, rotation, stretching, clipping, gaussian blurring, contrast adjustment, and the like. The problem of lack of model training samples can be solved to a certain extent by the data amplification technology

However, the current training sample data amplification method has the following technical problems:

the sample data amplification method based on geometric transformation or numerical transformation is still based on the existing data, for example, the initial data volume is too small, so that the training of an algorithm model is difficult to effectively improve, the accuracy of the finally obtained algorithm model is low, and a good effect is difficult to obtain when the method is applied to an actual service scene.

Disclosure of Invention

In view of the foregoing, it is desirable to provide an image sample data amplification method, apparatus, computer device, computer readable storage medium and computer program product that can combine marked image data with unmarked image data in an image synthesis manner, and replace manually marked data, so as to achieve the purpose of training sample data amplification and improve the accuracy of the finally obtained algorithm model.

In a first aspect, the present application provides a method for amplifying image sample data. The method comprises the following steps:

acquiring an image data set, wherein the image data set comprises a first image sample subjected to labeling processing and a second image sample not subjected to labeling processing;

constructing a generated countermeasure network model, wherein the generated countermeasure network model comprises a first-stage network, the first-stage network comprises a generation network and a discrimination network, the generation network is used for realizing mutual generation and conversion between the first image sample and the second image sample, and the discrimination network corresponding to the generation network is used for carrying out true and false discrimination on a result output by the generation network;

training the generated countermeasure network model to be converged by using the image data set, and extracting a target generation network for synthesizing the first image sample into the second image sample from the generated countermeasure network model;

and synthesizing the first image sample in the image data set into a synthesized second image sample by utilizing the target generation network, and matching and splicing the synthesized second image sample and the labeling information of the first image sample to obtain a third image sample.

In one embodiment, the generating the countermeasure network model includes a first generating network, a second generating network, a first discriminating network, and a second discriminating network, and the training the generating the countermeasure network model to converge using the image dataset includes:

generating, by the first generation network, a composite second image sample based on the first image sample and the composite first image sample;

performing true and false discrimination on the synthesized second image sample through the first discrimination network to obtain a first discrimination result;

generating, by the second generation network, the composite first image sample based on the second image sample and the composite second image sample;

performing true and false discrimination on the synthesized first image sample through the second discrimination network to obtain a second discrimination result;

the first generation network iterates according to the first discrimination result, and the second generation network iterates according to the second discrimination result.

In one embodiment, the iterating by the first generating network according to the first discrimination result, and iterating by the second generating network according to the second discrimination result includes:

Obtaining discrimination losses, and updating parameters of the first generation network and the second generation network based on the discrimination losses in iteration, wherein the discrimination losses comprise a first network discrimination loss, a second network discrimination loss, a loop coincidence loss and a global loss.

In one embodiment, the training the generated countermeasure network model to converge using the image dataset includes:

and preprocessing the image samples in the image data set, wherein the preprocessing comprises downsampling processing to obtain a downsampled image data set, and training the generated countermeasure network model to be converged based on the downsampled image data set.

In one embodiment, the generation countermeasure network includes a second stage network including a third generation network and a third discrimination network, the method including:

generating, by the third generation network, a restored second image sample conforming to the second image sample format based on the downsampled second image sample in the downsampled image dataset;

performing true and false discrimination on the restored second image sample through the third discrimination network;

And iterating the third generation network based on the output result of the third discrimination network until convergence.

In one embodiment, the synthesizing the first image sample in the imagery data set into a synthesized second image sample using the target generation network includes:

splicing the target generation network with the converged third generation network to obtain an image synthesis model;

synthesizing the downsampled first image samples in the downsampled image dataset into downsampled synthesized image samples using the target generating network;

synthesizing the downsampled synthesized image samples into the synthesized second image samples conforming to the second image sample format using the third generating network.

In a second aspect, the present application further provides an image labeling method. The method comprises the following steps:

acquiring an image to be marked;

inputting the image to be marked into an image segmentation model, acquiring marking information corresponding to the image to be marked through the image segmentation model, wherein the image segmentation model is obtained by training based on an image sample set, the image sample set comprises a third image sample, and the third image sample is obtained by the image sample data amplification method according to any one of the first aspect.

In a third aspect, the present application further provides an image sample data amplifying device. The device comprises:

the data acquisition module is used for acquiring an image data set, wherein the image data set comprises a first image sample subjected to marking processing and a second image sample not subjected to marking processing;

the network construction module is used for constructing a generated countermeasure network model, the generated countermeasure network model comprises a first stage network, the first stage network comprises generation networks and discrimination networks, the generation networks are used for realizing mutual generation and conversion between the first image sample and the second image sample, and the discrimination networks corresponding to the generation networks are used for carrying out true and false discrimination on a result output by the generation networks;

the network training module is used for training the generated countermeasure network model to be converged by using the image data set, and extracting a target generation network for synthesizing the first image sample into the second image sample from the generated countermeasure network model;

and the sample amplification module is used for synthesizing the first image sample in the image data set into a synthesized second image sample by utilizing the target generation network, and matching and splicing the synthesized second image sample with the labeling information of the first image sample to obtain a third image sample.

In one embodiment, the network training module comprises:

a first generation network module for generating, through the first generation network, a composite second image sample based on the first image sample and a composite first image sample;

the first discrimination network module is used for carrying out true and false discrimination on the synthesized second image sample through the first discrimination network to obtain a first discrimination result;

a second generation network module for generating, by the second generation network, the composite first image sample based on the second image sample and the composite second image sample;

the second discrimination network module is used for carrying out true and false discrimination on the synthesized first image sample through the second discrimination network to obtain a second discrimination result;

the iteration training module is used for iterating the first generation network according to the first discrimination result, and iterating the second generation network according to the second discrimination result.

In one embodiment, the iterative training module comprises:

and the loss function module is used for acquiring discrimination losses, and updating parameters of the first generation network and the second generation network based on the discrimination losses in iteration, wherein the discrimination losses comprise first network discrimination losses, second network discrimination losses, loop coincidence losses and global losses.

In one embodiment, the network training module comprises:

and the image preprocessing module is used for preprocessing the image samples in the image data set, the preprocessing comprises downsampling processing, a downsampled image data set is obtained, and the generation countermeasure network model is trained to be converged based on the downsampled image data set.

In one embodiment, the apparatus further comprises:

a third generation network module, configured to generate, through the third generation network, a restored second image sample that conforms to the second image sample format based on the downsampled second image sample in the downsampled image dataset after the downsampling process;

the third discrimination network module is used for carrying out true and false discrimination on the restored second image sample through the third discrimination network;

and the third network iteration module is used for iterating the third generation network until convergence based on the output result of the third discrimination network.

In one embodiment, the sample amplification module comprises:

the network splicing module is used for splicing the target generation network with the converged third generation network to obtain an image synthesis model;

A stage processing module, configured to synthesize a downsampled first image sample in the downsampled image dataset into a downsampled composite image sample using the target generating network;

and a two-stage processing module for synthesizing the downsampled synthesized image samples into the synthesized second image samples conforming to the second image sample format using the third generating network.

In a fourth aspect, the present application further provides an image labeling apparatus, where the apparatus includes:

the acquisition module is used for acquiring the image to be marked;

the application module is used for inputting the image to be marked into an image segmentation model, obtaining marking information corresponding to the image to be marked through the image segmentation model, wherein the image segmentation model is obtained by training based on an image sample set, the image sample set comprises a third image sample, and the third image sample is obtained by the image sample data amplification method according to any one of the first aspect.

In a fifth aspect, the present application also provides a computer device. The computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in an image sample data amplification method according to any embodiment of the first aspect or an image labeling method according to the second aspect.

In a sixth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of an image sample data amplification method according to any one of the embodiments of the first aspect or an image labeling method according to the second aspect.

In a seventh aspect, the present application also provides a computer program product. The computer program product comprises a computer program which, when executed by a processor, implements the steps of an image sample data amplification method according to any one of the embodiments of the first aspect or an image labeling method according to the second aspect.

The image sample data amplification method, the device, the computer equipment, the storage medium and the computer program product can achieve the following beneficial effects corresponding to the technical problems in the background technology through deducing the technical characteristics in the independent right:

the method comprises the steps of obtaining a marked first image sample and an unmarked second image sample, constructing a generated countermeasure network model, carrying out loop iteration through a pair of generating networks and a judging network, enabling parameters of the generating networks and the judging network to be adjusted in the loop iteration until the image sample synthesized by the generating networks is judged through the judging network, and extracting and generating a target generating network used for synthesizing the first image sample into the second image sample in the countermeasure network model at the moment, so that the first image sample can be used for generating the second image sample through the target generating network, and finally achieving the purpose of amplifying the sample by matching and splicing the generated second image sample with marking information. The finally amplified image sample is obtained based on the marked first image sample, so that a new sample which is not limited to geometric transformation and data transformation can be obtained by means of the first image sample which is widely marked and has larger data volume, the richness of the sample can be improved, and the accuracy of an image model can be improved finally.

Drawings

FIG. 1 is a schematic flow chart of a method for amplifying image sample data according to an embodiment;

FIG. 2 is a schematic diagram showing a second process of an image sample data amplification method according to another embodiment;

FIG. 3 is a flow diagram of data processing in a first stage network in one embodiment;

FIG. 4 is a flow diagram of data processing in a first stage network in one embodiment;

FIG. 5 is a flow diagram of a first stage network in loop iteration in one embodiment;

FIG. 6 is a schematic diagram illustrating a third flow chart of a method for amplifying image sample data according to another embodiment;

FIG. 7 is a flowchart of a method for amplifying image sample data according to another embodiment;

FIG. 8 is a flow chart illustrating preprocessing of an image dataset according to an embodiment;

FIG. 9 is a fifth flowchart of a method for amplifying image sample data according to another embodiment;

FIG. 10 is a flow diagram of data processing in a second stage network in one embodiment;

FIG. 11 is a flow diagram of a second stage network in an iteration in one embodiment;

FIG. 12 is a flowchart of a sixth method for amplifying image sample data according to another embodiment;

FIG. 13 is a block diagram showing an image sample data amplifying apparatus according to an embodiment;

fig. 14 is an internal structural diagram of a computer device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

Currently, in order to solve the problem of difficulty in data acquisition in liver organ segmentation tasks based on flat scan CT, it is generally necessary to perform amplification processing on the total number of training samples based on existing data. In the field of artificial intelligence image segmentation, conventional training data amplification techniques only perform a series of geometric transformations or numerical transformations on manually labeled images, such as translation, rotation, stretching, clipping, gaussian blurring, contrast adjustment, and the like. The problem of lack of model training samples can be solved to a certain extent through the data amplification technology.

the sample data generated by the sample data amplification method based on geometric transformation or numerical transformation is still based on the existing data, so that the training of an algorithm model is difficult to effectively improve, the accuracy of the finally obtained algorithm model is low, and a good effect is difficult to obtain when the method is applied to an actual service scene.

Based on this, the embodiment of the application provides an image sample data amplification method.

In one embodiment, as shown in fig. 1, the application provides an image sample data amplification method, and the embodiment is applied to a terminal for illustration by the method, and it can be understood that the method can also be applied to a server, and can also be applied to a system including the terminal and the server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the steps of:

step 102: an image dataset is obtained, wherein the image dataset comprises a first image sample subjected to labeling processing and a second image sample not subjected to labeling processing.

The first image sample may refer to an image sample which can be obtained in the internet or various public data websites and is manually marked, and because the application purpose of the scheme is to realize the segmentation of liver organs in medical images, the sample in the liver organ segmentation public data set based on enhanced CT or MRI (magnetic resonance imaging) and the like with larger storage in the public data set can be selected as the first image sample. In this embodiment, the enhanced CT image is taken as an example, and the description of the case and the like of applying other types of images will not be repeated. The second image sample may refer to an image sample that is not labeled, and in this embodiment, a plain CT image that is not labeled is taken as an example for illustration.

For example, the terminal may obtain data from a public database in the internet under the premise of obtaining sufficient authorization and permission, and take the marked enhanced CT image as a first image sample and the unmarked plain CT image as a second image sample. For example, a first image sample may be acquired from a liver segmentation challenge public dataset (Liver Tumor Segmentation Challenge, liTs) and a second image sample may be acquired from clinical data actually acquired by a hospital without a liver organ mask noted.

Step 104: the method comprises the steps of constructing a generated countermeasure network model, wherein the generated countermeasure network model comprises a first-stage network, the first-stage network comprises generation networks and discrimination networks, the generation networks are arranged in pairs and are used for realizing mutual generation and conversion between the first image sample and the second image sample, and the discrimination networks corresponding to the generation networks are used for carrying out true and false discrimination on results output by the generation networks.

Wherein generating an antagonism network may refer to an algorithmic model for an image generation application in an unsupervised environment. Generating the countermeasure network may include generating a network for capturing a distribution of the sample data and discriminating the network for discriminating whether the true data or the generated sample is input.

For example, after acquiring the image dataset, the terminal may construct a generated countermeasure network model, which may include a first-stage network. And the first-stage network may include a generation network and a discrimination network arranged in pairs. Wherein the generation network may be used to effect a generation conversion of the first image sample to the second image sample or to effect a generation conversion of the second image sample to the first image sample. The discrimination network may be configured to take as output an image output from the generation network, discriminate whether the image is a true image or a generated image, and output a result of the discrimination of authenticity. Because the application purpose of the scheme is to amplify the labeling sample of the flat scan CT image, the first-stage network at least comprises a generating network A for generating a second image sample based on the first image sample and a judging network A corresponding to the generating network A.

Step 106: training the generated countermeasure network model to be converged by using the image data set, and extracting a target generation network for synthesizing the first image sample into the second image sample in the generated countermeasure network model.

Wherein the composite image sample may refer to an image sample generated by the generation network. The third image sample may refer to a composite image sample carrying annotation information.

After the terminal builds the generated countermeasure network model based on the image data set, the terminal can train the generated countermeasure network by using the image data set. In training, the terminal may iterate the generating the countermeasure network with the image dataset as input until parameters of the generating countermeasure network model converge, or the number of iterations of generating the countermeasure network reaches a preset number.

In this way, the terminal can acquire the generated countermeasure network model after parameter convergence. At this time, in order to achieve the purpose of sample amplification for amplifying the flat scan CT image, the terminal may extract a generation network a for synthesizing the first image sample into the second image sample as a target generation network from the trained generation countermeasure network model.

Step 108: and synthesizing the first image sample in the image data set into a synthesized second image sample by utilizing the target generation network, and matching and splicing the synthesized second image sample and the labeling information of the first image sample to obtain a third image sample.

The labeling information may refer to information for performing auxiliary description on specific information in the sample image, and in this scheme, the labeling information may be a manually labeled liver mask corresponding to the first image sample.

The terminal may input the first image sample with the manually marked liver mask into the target generating network after extracting the target generating network, so as to obtain a composite image sample generated by the target generating network based on the first image sample, where the first image sample is an enhanced CT image with the manually marked liver mask, the composite image sample is a flat scan CT image generated by the enhanced CT image, and the flat scan CT image is generated and is different from a flat scan CT image existing in reality. After the terminal obtains the synthesized image sample, the liver mask corresponding to the corresponding first image sample may be spliced with the synthesized image sample, and finally a third image sample is obtained.

In the image sample data amplification method, reasonable deduction is carried out by combining technical characteristics, so that the following beneficial effects of solving the technical problems in the background technology are achieved:

In one embodiment, as shown in fig. 2, generating the countermeasure network model may include a first generation network, a second generation network, a first discrimination network, and a second discrimination network, step 106 including:

step 202: and generating, by the first generation network, a composite second image sample based on the first image sample and the composite first image sample.

Wherein the first generation network may refer to a generation network for implementing synthetic conversion of the first image sample to the second image sample, and may be the generation network a.

For example, as shown in fig. 3, the terminal may input a first image sample to the first generation network and generate a composite second image sample through the first generation network. That is, the terminal inputs the real enhanced CT image into the generating network A to obtain the synthesized plain CT image.

Step 204: and carrying out true and false discrimination on the synthesized second image sample through the first discrimination network to obtain a first discrimination result.

The first discrimination network may be a discrimination network for realizing discrimination of authenticity of the synthesized second image sample, and may be a discrimination network B.

For example, as shown in fig. 3, the terminal may input the synthesized second image sample output in the first generating network into the first discriminating network, and the first discriminating network may simultaneously input the true second image sample and output the first discriminating result after the authenticity discrimination is completed. That is, the terminal inputs the synthesized plain scan CT image and the true plain scan CT image into the first discrimination network, and the first discrimination network performs the true-false discrimination and outputs the result.

Step 206: the composite first image sample is generated, by the second generation network, based on the second image sample and the composite second image sample.

Wherein the second generation network may refer to a generation network for implementing a synthetic transformation of the second image sample to the first image sample, and may be the generation network B.

For example, as may be shown in fig. 4, the terminal may input the second image sample to the second generation network and generate the composite first image sample through the second generation network. That is, the terminal inputs the real plain scan CT image into the generation network B to obtain the synthesized enhanced CT image.

Step 208: and carrying out true and false discrimination on the synthesized first image sample through the second discrimination network to obtain a second discrimination result.

The second discrimination network may be a discrimination network for implementing discrimination of authenticity of the synthesized first image sample, and may be a discrimination network a.

For example, as shown in fig. 4, the terminal may input the synthesized first image sample output in the second generation network into the second discrimination network, and the second discrimination network may simultaneously input the true first image sample and output the second discrimination result after the completion of the authenticity discrimination. That is, the terminal inputs the synthesized enhanced CT image and the true enhanced CT image into the second discrimination network, and the second discrimination network performs the true-false discrimination and outputs the result.

Step 2010: the first generation network iterates according to the first discrimination result, and the second generation network iterates according to the second discrimination result.

For example, as shown in fig. 5, the terminal may iterate the first generation network according to the first discrimination result, and iterate the second generation network according to the second discrimination result.

In this embodiment, two pairs of generating networks and countermeasure networks are provided, which is conducive to realizing the architecture of circularly generating the countermeasure network model, so as to improve the training efficiency of generating the countermeasure network and the model effect obtained by final training.

In one embodiment, as shown in fig. 6, step 2010 includes:

step 602: obtaining discrimination losses, and updating parameters of the first generation network and the second generation network based on the discrimination losses in iteration, wherein the discrimination losses comprise a first network discrimination loss, a second network discrimination loss, a loop coincidence loss and a global loss.

Wherein the loss function is a function that maps random events or their values of related random variables to non-negative real numbers to represent the loss of the random event, and is a function used in an application to evaluate a model.

Illustratively, the terminal may calculate a loss function for each node in the network in an iteration, which may include a first network discrimination loss, a second network discrimination loss, a loop consistency loss, and a global loss.

In particular, the data flow in the first phase network in an iteration may be as shown in fig. 5. With the exemplary features of this embodiment, in each iteration, the detailed processing steps of the terminal may be:

acquiring a real enhanced CT image RA and a real flat scan CT image RB in an image data set;

inputting the real enhanced CT image into a generation network A to obtain a synthesized flat scanning CT image FB;

the real flat scan CT image RB and the synthesized flat scan CT image FB are respectively input into a discrimination network B, and discrimination loss is calculatedAnd according to->And updating parameters of the judging network B. Specifically, in the present embodiment +.>A cross entropy loss function may be employed, and the specific calculation mode may be as follows:

wherein the method comprises the steps ofTo distinguish network B;

inputting the real flat scan CT image RB to a generation network B to obtain a synthesized enhanced CT image FA;

the real enhanced CT image RA and the synthesized enhanced CT image FA are respectively input into a discrimination network A, and discrimination loss is calculated And according to->And updating parameters of the judging network A. In this embodiment +.>The cross entropy loss function is adopted, and the specific calculation mode is as follows:

wherein the method comprises the steps ofTo distinguish network a;

inputting the synthesized enhanced CT image FA and the synthesized flat scan CT image FB into a generating network A and a generating network B respectively to obtain a circularly synthesized flat scan CT image FB 'and an enhanced CT image FA';

calculating a cyclic coincidence loss. In this embodiment, the L2 norms between the real enhanced CT image RA and the circularly synthesized enhanced CT image FA 'and between the real flat scan CT image RB and the circularly synthesized flat scan CT image FB' may be calculated respectively, and the two may be summed up, where the specific calculation mode is as follows:

8) Calculating global lossesAnd according to->The parameters of the generation network a and the generation network B are updated by the values of (a). The concrete form is as follows:

wherein the method comprises the steps of、/>、/>Weights for three loss functions.

In this embodiment, loop iteration is performed on the first-stage network based on the loss function, the loop consistency loss and the global loss of the discrimination network in the first-stage network, which is helpful for improving the quality of the finally obtained model.

In one embodiment, as shown in FIG. 7, step 106 includes:

step 702: and preprocessing the image samples in the image data set, wherein the preprocessing comprises downsampling processing to obtain a downsampled image data set, and training the generated countermeasure network model to be converged based on the downsampled image data set.

The downsampling process may refer to a process manner of reducing the number of images, and may be implemented by selecting pixels on the images at a certain interval.

For example, as shown in fig. 8, before iterating the generation of the countermeasure network based on the image dataset, in order to improve the efficiency of the network iteration, a certain preprocessing step may be performed on the image dataset, where a downsampling process may be included. In particular, exemplary features may be extended, and it may be assumed that the resolution of the enhanced CT image and the pan CT image in the image dataset is unified at 512 x 512. After the terminal acquires the image data, the terminal can adjust the gray value of the CT image in the image data to the window width and the window level of the liver, and normalize the gray value to the interval of [ -1,1 ]. Then, the terminal can extract the slices of the normalized CT image on the cross section one by one to form a set formed by n 2-dimensional CT images, wherein n is the number of the slices of each CT image on the axis. At this time, the terminal can obtain the image data set. For the image samples in the image dataset, the terminal may perform downsampling processing on the image samples, for example, the resolution is reduced to 1/2, that is, from 512×512 to 256×256, so as to obtain a downsampled image dataset.

In this way, the terminal can utilize the obtained downsampled image dataset to perform subsequent iterative training to generate an countermeasure network.

In this embodiment, the downsampling process is performed on the image dataset, so that accuracy of the trunk algorithm is improved, the trunk algorithm can perform conversion processing on samples with fewer features, meanwhile, calculation efficiency of the model is improved, and calculation resources occupied in the training and using processes of the model are reduced. On the other hand, the method is beneficial to improving the efficiency of the subsequent model iterative training and reducing the overall calculation amount required to be completed in iteration.

In one embodiment, as shown in fig. 9 and fig. 10, the generating countermeasure network includes a second stage network, the second stage network includes a third generating network and a third discriminating network, and the method further includes:

step 902: and generating a restored second image sample conforming to the second image sample format based on the downsampled second image sample in the downsampled image dataset through the third generating network.

Illustratively, the third generating network may refer to a generating network for implementing the conversion of the downsampled image samples into the original format, and may be the generating network C.

For example, the terminal may input the downsampled second image samples to the third generating network and output restored second image samples conforming to the second image sample format. That is, the terminal inputs the downsampled flat scan CT to the generating network C, resulting in the original resolution flat scan CT output by the generating network C.

Step 904: and carrying out true and false discrimination on the restored second image sample through the third discrimination network.

The third discrimination network may be a network for performing discrimination of authenticity of the restored second image sample, and may be a discrimination network C.

For example, the terminal may use the restored second image sample and the original second image sample as the input of the third discrimination network at the same time, and obtain the discrimination result output by the third discrimination network. That is, the terminal inputs the synthesized original resolution flat scan CT and the true original resolution flat scan CT into the discrimination network C at the same time, so as to obtain whether the true flat scan CT image is output by the discrimination network C.

Step 906: and iterating the third generation network based on the output result of the third discrimination network until convergence.

Illustratively, the terminal may iterate the second-stage network according to a loss function of the discrimination network C, as shown in fig. 11. Specifically, the iterative step may include:

Acquiring a downsampled flat-scan CT image RB in a downsampled image dataset, and acquiring a corresponding original flat-scan CT image RC in an original image dataset;

inputting the downsampled flat scan CT image RB to a generation network C to obtain a synthesized flat scan CT image FC;

the original flat scan CT image RC and the synthesized flat scan CT image FC are respectively input into a discrimination network C to calculate discrimination lossAnd according to->And updating parameters of the discrimination network C. In this embodiment +.>The cross entropy loss function can be adopted, and the specific calculation mode is as follows:

wherein the method comprises the steps ofTo discriminate network C;

computing generated network lossesAnd according to->Updating parameters of the generation network C. In this embodiment +.>The cross entropy loss function can be adopted, and the specific calculation mode is as follows:

additionally, in this embodiment, the first stage network and the second stage network may perform independent training respectively, and do not share the training process and parameters.

In this embodiment, the second-stage network is trained, so that the recovery efficiency and effect of the CT image can be improved by the second-stage network obtained through training.

In one embodiment, as shown in FIG. 12, step 108 may include:

step 1202: and splicing the target generation network with the converged third generation network to obtain an image synthesis model.

For example, the terminal may splice the target generating network and the third generating network, that is, take the output of the target generating network as the input of the third generating network, to obtain the image synthesis model.

Step 1204: and synthesizing the downsampled first image samples in the downsampled image dataset into downsampled synthesized image samples using the target generating network.

For example, the terminal may synthesize the downsampled enhanced CT image into downsampled synthesized image samples.

Step 1206: synthesizing the downsampled synthesized image samples into the synthesized second image samples conforming to the second image sample format using the third generating network.

Specifically, referring to the above example, after the terminal finishes the two-stage generation countermeasure network training, the terminal may extract the generation network a and the generation network C, and splice the generation network a and the generation network C into the image synthesis model. Then, the terminal can preprocess the enhanced CT image with the artificially marked liver, and input the preprocessed enhanced CT image into a generating network A to obtain the synthesized downsampled flat scanning CT image. And then, the terminal can input the synthesized downsampled flat scan CT image into a generating network C to obtain the synthesized original resolution flat scan CT image. And after the synthesized original resolution flat scanning CT image is recombined on the axial position, the synthesized original resolution flat scanning CT image can be paired with a manually marked liver mask, and amplified image sample data can be obtained.

For example, the terminal may utilize the third generation network to convert the downsampled composite image samples output by the target generation network back into composite second image samples, i.e., composite flat scan CT images that meet the resolution requirements.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides an image labeling method, which comprises the following steps:

Step S1: acquiring an image to be marked;

step S2, inputting the image to be marked into an image segmentation model, obtaining marking information corresponding to the image to be marked through the image segmentation model, wherein the image segmentation model is obtained by training based on an image sample set, the image sample set comprises a third image sample, and the third image sample is obtained by the image sample data amplification method according to any one of the method embodiments.

Based on the same inventive concept, the embodiment of the application also provides an image sample data amplifying device for implementing the image sample data amplifying method. The implementation of the solution provided by the apparatus is similar to that described in the above method, so the specific limitations of one or more embodiments of the image sample data amplifying apparatus provided below can be referred to above for the limitations of the image sample data amplifying method, and will not be repeated here.

In one embodiment, as shown in fig. 13, there is provided an image sample data amplifying apparatus, comprising: the system comprises a data acquisition module, a network construction module, a network training module and a sample amplification module, wherein:

In one embodiment, the network training module comprises:

In one embodiment, the iterative training module comprises:

In one embodiment, the network training module comprises:

In one embodiment, the apparatus further comprises:

In one embodiment, the sample amplification module comprises:

Based on the same inventive concept, the embodiment of the present application further provides an image labeling device, including:

the acquisition module is used for acquiring the image to be marked;

the application module is used for inputting the image to be marked into an image segmentation model, obtaining marking information corresponding to the image to be marked through the image segmentation model, wherein the image segmentation model is obtained by training based on an image sample set, the image sample set comprises a third image sample, and the third image sample is obtained by the image sample data amplification method according to any one of the method embodiments.

The above-mentioned various modules in an image sample data amplifying device or an image labeling device may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a terminal, and the internal structure thereof may be as shown in fig. 14. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input means. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a method for amplifying image sample data. The display unit of the computer device is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be a key, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the structure shown in fig. 14 is merely a block diagram of a portion of the structure associated with the present application and is not limiting of the computer device to which the present application applies, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.

In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.

In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

It should be noted that, the user information (including, but not limited to, user equipment information, user personal information, etc.) and the data (including, but not limited to, data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to comply with the related laws and regulations and standards of the related countries and regions.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.

Claims

1. A method for amplifying image sample data, the method comprising:

2. The method of claim 1, wherein the generating the countermeasure network model includes a first generating network, a second generating network, a first discriminating network, and a second discriminating network, and wherein the training the generating the countermeasure network model to converge using the image dataset includes:

3. The method of claim 2, wherein the first generation network iterates according to the first discrimination result, and the second generation network iterates according to the second discrimination result comprises:

4. The method of claim 1, wherein training the generated countermeasure network model to converge using the image dataset comprises:

5. The method of claim 4, wherein the generation of the countermeasure network includes a second stage network including a third generation network and a third discrimination network, the method comprising:

6. The method of claim 5, wherein synthesizing the first image sample in the imagery data set using the target generation network into a synthesized second image sample comprises:

7. An image labeling method, characterized in that the method comprises the following steps:

acquiring an image to be marked;

inputting the image to be marked into an image segmentation model, obtaining marking information corresponding to the image to be marked through the image segmentation model, wherein the image segmentation model is obtained by training based on an image sample set, the image sample set comprises a third image sample, and the third image sample is obtained by the image sample data amplification method according to any one of claims 1 to 6.

8. An image sample data amplification apparatus, the apparatus comprising:

9. An image annotation device, the device comprising:

the acquisition module is used for acquiring the image to be marked;

the application module is configured to input the image to be marked into an image segmentation model, obtain marking information corresponding to the image to be marked through the image segmentation model, obtain the image segmentation model based on training of an image sample set, and obtain a third image sample through an image sample data amplification method according to any one of claims 1 to 6.

10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 6 or the steps of the method of claim 7 when the computer program is executed.

11. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any one of claims 1 to 6, or the steps of the method of claim 7.