CN115131452B

CN115131452B - Image processing method and device for artifact removal

Info

Publication number: CN115131452B
Application number: CN202210408968.2A
Authority: CN
Inventors: 王红; 李悦翔; 郑冶枫; 孟德宇
Original assignee: Tencent Healthcare Shenzhen Co Ltd
Current assignee: Tencent Healthcare Shenzhen Co Ltd
Priority date: 2022-04-19
Filing date: 2022-04-19
Publication date: 2024-11-08
Anticipated expiration: 2042-04-19
Also published as: CN115131452A; WO2023202265A1

Abstract

Embodiments of the present disclosure provide an image processing method, apparatus, computer program product and storage medium for artifact removal. The image processing method of the present disclosure includes: and establishing a training data set for training a neural network, performing artifact removal processing on the artifact-bearing image by using an adaptive convolutional dictionary network to obtain a processed image, and performing iterative training on the adaptive convolutional dictionary network based on the artifact-free image and the processed image and an objective function processed by the image mask to optimize network parameters of the adaptive convolutional dictionary network. The image processing method can simply and effectively remove the artifacts in the image so as to obtain a clearer image which is not interfered by the artifacts.

Description

Image processing method and device for artifact removal

Technical Field

The present disclosure relates to the field of artificial intelligence, and more particularly, to an image processing method, apparatus, computer program product, and storage medium for artifact removal.

Background

Images are a great weight in various information bases as one of the most important sources for human acquisition of information. With the rapid development of computer technology, image processing has been widely applied to various aspects of human social life, such as: industrial inspection, medicine, intelligent robots, etc. Images are often applied to various fields to describe and express characteristics and logical relationships of things in terms of their liveliness and intuitiveness, and thus, the application range is wide, so that development of image processing technology and information processing for various fields are extremely important.

The image processing technique is a technique of processing image information with a computer. Mainly comprises image digitizing, image enhancing and restoring, image data encoding, image dividing, image identifying and the like. Among them, the purpose of the image restoration technique is to restore the degraded image to the original real appearance as much as possible. For example, the method is used in the field of removing artifacts in images, such as removing noise in images, removing raindrops in images with rain photographed in rainy days, removing metal artifacts in CT images, and the like. However, the current image processing method faces technical bottlenecks such as complex mathematical model, limited application range, difficult acquisition of partial data and the like when removing the artifacts.

Therefore, there is a need for an image processing method that can be widely used and effectively identify artifacts in images and remove them from artifact-bearing images to help people obtain clearer images that are not disturbed by artifacts.

Disclosure of Invention

In order to solve the above-described problems, the present disclosure provides an image processing method, apparatus, computer program product, and storage medium for artifact removal. The image processing method for artifact removal provided by the present disclosure includes: establishing a training data set for training a neural network, wherein the training data set comprises a plurality of groups of image samples, and each group of image samples comprises an image (Y) with artifacts and an image mask (I) without artifacts corresponding to the image (Y); performing an artifact removal process on the artifact-bearing image (Y) using an adaptive convolutional dictionary network for at least one of the plurality of sets of image samples to obtain a processed image, iteratively training the adaptive convolutional dictionary network based on the artifact-free image (X) and the processed image, and an objective function processed by the image mask (I) to optimize network parameters of the adaptive convolutional dictionary network, wherein the adaptive convolutional dictionary network comprises a basic artifact dictionary that is a sample-invariant convolutional dictionary and comprises a first number of artifact convolution kernels, and determining a second number of adaptive convolution kernels for the image samples from a plurality of artifact convolution kernels and sample-variant weighting coefficients in the basic artifact dictionary, wherein an artifact image in the artifact-bearing image is determined from a convolution of the second number of adaptive convolution kernels with image features of the artifact-bearing image, and removing the processed image from the artifact-bearing image.

The image processing method can simply and effectively remove the artifacts in the image so as to obtain a clearer image which is not interfered by the artifacts.

According to an embodiment of the disclosure, the first number of artifact convolution kernels indicates an artifact pattern, and the image feature indicates a location of the artifact pattern, wherein the adaptive convolution dictionary network includes a T-level network, and in the T-level network, a weighting coefficient and an image feature output by the T-1-level network are updated by using an iterative update rule based on a near-end gradient descent, so as to obtain the weighting coefficient and the image feature of the T-level network, where T is an integer greater than 1 and less than or equal to T.

According to an embodiment of the disclosure, each level of network comprises a weighting coefficient update network, an image feature update network, and an artifact-removed image update network, wherein the weighting coefficient update network, the image feature update network, and the artifact-removed image update network comprise a residual network structure and a normalization processing layer.

According to an embodiment of the present disclosure, the establishing a training data set for training the neural network further comprises at least one of: normalizing the pixel values of the artifact-bearing image (Y); and performing random cropping on the image with the artifact (Y) to obtain an image block, and performing random flipping processing on the image block according to a preset probability.

According to an embodiment of the disclosure, the objective function is a loss objective function constructed based on the artifact-free image (X) and the processed image, wherein the iteratively training the adaptive convolutional dictionary network based on the artifact-free image (X) and the processed image, and the objective function to optimize network parameters of the adaptive convolutional dictionary network further comprises: a loss objective function is calculated and its results are back-propagated to the adaptive convolutional dictionary network and network parameters of the adaptive convolutional dictionary network are optimized based on an adaptive moment estimation (Adaptive moment estimation, adam) algorithm.

According to an embodiment of the present disclosure, the image processing method further includes: after training is completed, testing the adaptive convolutional dictionary network, wherein the testing the adaptive convolutional dictionary network comprises: preprocessing an image with an artifact to be tested, and inputting the image into the self-adaptive convolution dictionary network; and processing the image with the artifact to be tested by using the self-adaptive convolution dictionary network to obtain a processed image with the artifact removed.

According to an embodiment of the disclosure, the artifact is a metal artifact, the artifact-bearing image is a CT image with a metal artifact, the training dataset includes a plurality of sets of CT image samples, each set of image samples including the CT image with a metal artifact and a corresponding CT image without a metal artifact and a non-metal region mask.

According to an embodiment of the disclosure, each level of network comprises a weighting coefficient update network, a metal artifact image feature update network, and a metal artifact removal image update network, wherein the weighting coefficient update network, the metal artifact image feature update network, and the metal artifact removal image update network comprise a residual network structure; and the weighting coefficient updating network includes: a linear layer, a modified linear Unit (RECTIFIED LINEAR Unit, reLU) layer, a cross-link layer, and a batch normalization (Batch Normalization, BN) layer; the metal artifact image feature update network comprises: a convolutional layer, a BN layer, a ReLU layer, and a cross-link layer; the metal artifact removal image update network comprises: convolutional layer, BN layer, reLU layer, and cross-link layer.

The embodiment of the disclosure also provides an image processing method for artifact removal, comprising: acquiring an input image to be processed; processing the input image with an adaptive convolutional dictionary network to obtain an artifact-removed processed image, wherein the adaptive convolutional dictionary network is trained based on an artifact database (D) and comprises a T-level network, wherein a first-level network obtains a first-level image feature and a first-level artifact-removed image output by the first-level network based on the input image; the T-level network obtains and outputs T-level image features and T-level artifact removed images output by the T-level network based at least in part on the T-1-level image features and the T-1-level artifact removed images output by the T-1-level network, wherein T is greater than 1 and less than or equal to T; and the T-stage network obtains and outputs the T-stage artifact removal image output by the T-stage network as the artifact removal processed image based on the T-1 stage image characteristics and the T-1 stage artifact removal image output by the T-1 stage network.

According to an embodiment of the present disclosure, the adaptive convolutional dictionary network includes a basic artifact dictionary that is a convolutional dictionary that does not vary with an input image and includes a first number of artifact convolutional kernels, a t-th level weighting coefficient of a t-th level network is determined, a second number of adaptive convolutional kernels for the t-th level network is determined by a plurality of artifact convolutional kernels in the basic artifact dictionary and the t-th level weighting coefficient; and determining a t-th stage artifact-removed image based on the second number of adaptive convolution kernels and image features of the t-th stage network.

Embodiments of the present disclosure provide an image processing apparatus for artifact removal, including: a training data set creation module configured to: establishing a training data set for training a neural network, wherein the training data set comprises a plurality of groups of image samples, and each group of image samples comprises an image (Y) with artifacts and an image mask (I) without artifacts corresponding to the image (Y); an adaptive convolutional dictionary network configured to: performing artifact removal processing on the artifact-bearing image (Y) for at least one of the plurality of sets of image samples to obtain a processed image; a training module configured to: iteratively training the adaptive convolutional dictionary network based on the artifact-free image (X) and the processed image, and an objective function processed by the image mask (I), to optimize network parameters of the adaptive convolutional dictionary network; wherein the adaptive convolution dictionary network comprises a basic artifact dictionary that is a sample-invariant convolution dictionary and that comprises a first number of artifact convolution kernels and a second number of adaptive convolution kernels for the image samples are determined by a plurality of artifact convolution kernels in the basic artifact dictionary and sample-variant weighting coefficients, wherein an artifact image in the artifact image is determined by a convolution of the second number of adaptive convolution kernels with image features of the artifact image and the artifact image is removed from the artifact image to obtain the processed image.

Embodiments of the present disclosure provide an image processing apparatus for artifact removal, including: an image acquisition module configured to: acquiring an input image to be processed; an image processing module configured to: processing the input image by using an adaptive convolution dictionary network to obtain a processed image from which artifacts are removed; wherein the adaptive convolutional dictionary network is trained based on an artifact database (D) and comprises a T-level network, wherein a first-level network obtains a first-level image feature and a first-level artifact removed image output by the first-level network based on the input image; the T-level network obtains and outputs T-level image features and T-level artifact removed images output by the T-level network based at least in part on the T-1-level image features and the T-1-level artifact removed images output by the T-1-level network, wherein T is greater than 1 and less than or equal to T; and the T-stage network obtains and outputs the T-stage artifact removal image output by the T-stage network as the artifact removal processed image based on the T-1 stage image characteristics and the T-1 stage artifact removal image output by the T-1 stage network.

Embodiments of the present disclosure provide a computer program product comprising computer software code which, when run by a processor, provides the above method.

Embodiments of the present disclosure provide a computer readable storage medium having stored thereon computer executable instructions that, when executed by a processor, provide the above-described method.

The image processing method disclosed by the invention can be used for performing artifact removal processing based on the image to be processed only, and the chord graph of the image is not required to be additionally acquired. The adaptive convolution dictionary network adopted by the method fully utilizes the prior structure of the artifact image, the artifact removal effect is better, and the model generalization is stronger. In addition, the mathematical model adopted by the image processing method is clear in physical meaning and strong in interpretation, and the physical meaning of each network module is more clear, so that the method is convenient for people in the field to understand and apply. The image processing method can simply and effectively remove the artifacts in the image so as to obtain a clearer image which is not interfered by the artifacts.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings that are used in the description of the embodiments will be briefly described below. It should be apparent that the drawings in the following description are only some exemplary embodiments of the present disclosure, and that other drawings may be obtained from these drawings by those of ordinary skill in the art without undue effort.

Here, in the drawings:

1A-1B are schematic diagrams illustrating an artifact-free image and an artifact-free image, respectively, according to an embodiment of the present disclosure;

2A-2B are schematic diagrams illustrating chord graph-based image processing methods according to embodiments of the present disclosure;

3A-3B are schematic flow diagrams illustrating an image processing method for artifact removal according to an embodiment of the present disclosure;

FIG. 4 is an exemplary diagram illustrating an image processing model based on a weighted adaptive convolution dictionary in accordance with an embodiment of the present disclosure;

5A-5B are schematic diagrams illustrating a process of iterative updating of various levels of the network in an adaptive convolutional dictionary network, in accordance with an embodiment of the present disclosure;

FIG. 6 is a schematic flow chart diagram illustrating an image processing procedure for artifact removal according to an embodiment of the present disclosure;

Fig. 7A to 7B are constituent diagrams showing an image processing apparatus for artifact removal according to an embodiment of the present disclosure;

FIG. 8 is an architecture illustrating a computing device according to an embodiment of the present disclosure; and

Fig. 9 is a schematic diagram illustrating a storage medium according to an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the present disclosure more apparent, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present disclosure and not all of the embodiments of the present disclosure, and that the present disclosure is not limited by the example embodiments described herein.

In addition, in the present specification and the drawings, steps and elements having substantially the same or similar are denoted by the same or similar reference numerals, and repeated descriptions of the steps and elements will be omitted.

Furthermore, in the present specification and drawings, elements are described in the singular or plural form according to the embodiments. However, the singular and plural forms are properly selected for the proposed case only for convenience of explanation and are not intended to limit the present disclosure thereto. Accordingly, the singular may include the plural and the plural may include the singular unless the context clearly indicates otherwise.

In the present specification and drawings, steps and elements having substantially the same or similar are denoted by the same or similar reference numerals, and repeated descriptions of the steps and elements will be omitted. Meanwhile, in the description of the present disclosure, the terms "first," "second," and the like are used merely to distinguish the descriptions, and are not to be construed as indicating or implying relative importance or order.

For purposes of describing the present disclosure, the following presents concepts related to the present disclosure.

The methods of the present disclosure may be artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) based. Artificial intelligence is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and expand human intelligence, sense the environment, acquire knowledge and use the knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. For example, for artificial intelligence based methods, it is possible to perform machine learning in a manner similar to human perception, such as extracting image information by training a neural network, performing image analysis and processing.

The image processing technique is a technique of processing image information with a computer. Mainly comprises the following steps: image digitization, image enhancement and restoration, image data encoding, image segmentation, image recognition, and the like. Currently image processing techniques are mainly implemented on a computer basis. The artificial neural network (ARTIFICIAL NEURAL NETWORKS, ANN) is an algorithmic mathematical model that simulates the structure and behavior of a biological nervous system, performing distributed parallel information processing. The ANN achieves the purpose of processing information by adjusting the weight relation between internal neurons. When processing the image, the computer takes the original image or the image which is suitably preprocessed as the input signal of the neural network, and obtains the processed image signal or the classification result at the output end of the neural network.

In summary, embodiments of the present disclosure relate to computer technologies such as artificial intelligence, image processing, and neural networks, and are further described below with reference to the accompanying drawings.

Taking a CT image as an example, fig. 1A is a schematic diagram illustrating an artifact-bearing image according to an embodiment of the present disclosure.

X-ray Computed Tomography (CT) has been widely used for clinical diagnosis. However, during imaging, the presence of metallic implants in the patient, such as dental fillings and hip prostheses, often results in a loss of projection data, causing severe metallic artifacts to appear in the reconstructed CT image. As shown in fig. 1A, the structure of the metal artifact is in the form of a set of morphologically similar striped shadows. The thickness and shade of the striped shadows may vary for different tissue structures and/or metallic implants. As can be seen from fig. 1A, the existence of the metal artifact obscures the CT image, seriously interferes with the clear presentation of the detailed image of the patient body in the CT image, and makes it difficult for a doctor to make a correct judgment according to the CT image.

There is therefore a need for an image processing method that can effectively identify artifacts in images and remove them from artifact-bearing images so that the processed artifact-free images are as shown in fig. 1B. By providing a clear, artifact-free CT image, such as that shown in FIG. 1B, a physician can be aided in making a diagnosis more accurately.

Fig. 2A shows a schematic diagram of a chord graph-based image processing method according to an embodiment of the present disclosure.

As shown in fig. 2A, duDoNet ++ is a joint learning scheme based on CT images and chord diagrams, which uses two network modules, SE-Net and IE-Net, to process CT images with metal artifacts together, where SE-Net is a network module for chord diagrams and IE-Net is a network module for CT images. In fig. 2A, S _ma represents a chord chart contaminated with metal, S _se represents a chord chart after repair enhancement, X _se represents a CT image obtained by back projection transformation (RIL) from S _se, M represents a shape feature of metal (i.e., mask) of a CT image domain, M _p represents a shape feature of metal of a chord chart domain obtained by radon transformation (FP) from M, X _ma represents a CT image with metal artifact, and X _out represents an output reconstructed image. As can be seen from fig. 2A, the CT image with metal artifacts is subjected to the joint processing of the chord domain (using SE-Net) and the CT image domain (using IE-Net) to achieve the purpose of image artifact removal, wherein the transformation between the chord domain and the CT image domain is converted by the differentiable radon transform layer.

Similarly, fig. 2B shows a schematic diagram of another chord graph-based image processing method according to an embodiment of the disclosure.

As shown in fig. 2B, DSCMAR is also a combined processing scheme based on CT images and chord diagrams, unlike DuDoNet ++, the scheme uses PriorNet to obtain a cleaner repair image, then uses SinoNet to further modify and enhance the chord diagrams, and finally uses filtering and then converting the projection layers (Filtered back projection, FBP) to obtain a reconstructed CT image. In fig. 2B, S _ma represents a chord chart contaminated with metal, T _r represents a chord chart of a metal feature, S _LI represents a chord chart after linear interpolation processing, X _LI represents a CT image obtained by subjecting S _LI to back projection conversion, X _ma represents a CT image with metal artifacts, X _prior represents an initial repair CT image obtained by subjecting PriorNet to forward projection conversion, S _prior represents an initial repair chord chart obtained by subjecting X _prior to front projection conversion, S _res represents a chord chart obtained by making a difference between S _LI and S _prior, S _corr represents a chord chart after further correction enhancement after processing by SinoNet, and an image free of metal artifacts can be obtained by subjecting S _corr to filter back projection conversion.

The chord graph-based image processing method shown in fig. 2A and 2B enables image processing for artifact removal as illustrated in fig. 1A and 1B. However, duDoNet ++ and DSCMAR have common limitations: in both of these schemes, the chord graph information used is difficult to obtain in practice and typically requires the equipment manufacturer to provide it. Moreover, the designed network does not embed a priori information specific to the Metal Artifact Removal (MAR) task well, and the generalization capability of the network model is limited. Furthermore, each of the network modules included in these two schemes is physically poorly interpretable and not readily understood and used by those skilled in the art.

In view of these problems, the present disclosure proposes an image processing method for artifact removal, which can perform reconstruction processing based on only an image with artifacts, and overcomes the problem of difficulty in chord graph data acquisition. In addition, the image processing method disclosed by the invention utilizes a specific weighted convolution dictionary model to encode the prior structure of the artifact, so that the network is more reliable and has stronger generalization capability. In addition, the adaptive convolutional dictionary network model of the present disclosure has clear physical interpretability, which is easy for those skilled in the art to understand and use.

FIG. 4 is an exemplary schematic diagram of an image processing model based on a weighted adaptive convolution dictionary in accordance with an embodiment of the present disclosure.

For CT images with metal artifacts, its non-metallic regions can be decomposed into the following models:

I⊙Y＝I⊙X+I⊙A， (1)

wherein, Is a CT image with metal artifacts; h and W are the height and width of the image, respectively; x is a clean CT image to be restored; i is a non-metallic region mask, the elements of which are {0,1}, wherein 1 represents a non-metallic region; a is a metal artifact. According to embodiments of the present disclosure, non-metallic regions maskI may be known that are used to focus the solved mathematical model on non-metallic regions, without focusing on the solution of metallic regions.

It will be appreciated that for different CT images with metal artifacts, the metal artifacts a contained therein exhibit substantially a common or similar pattern, i.e. a morphologically similar stripe-like structure. Meanwhile, due to the interaction of the normal tissue of the human body and the metal artifact, the artifact patterns contained in different CT images with the metal artifact have specific properties, such as pixel intensity.

Considering the above-described features of a CT image with metal artifacts, according to an embodiment of the present disclosure, for each artifact-bearing image (Y) to be processed, a basic artifact dictionary learned in advance from existing samples is utilizedTo construct an adaptive convolution kernel for the imageAnd extracting artifact features of the imageAn adaptive convolution kernel based on the image may then be usedAnd artifact featuresAnd obtaining the metal artifact A of the image, thereby realizing the extraction of the metal artifact. It should be appreciated that a basic artifact dictionaryThe method can be a basic artifact dictionary which is known in the prior art, or can be obtained by training and synthesizing the prior samples.

According to embodiments of the present disclosure, a weighted convolution dictionary model may be employed to model the coding of metal artifact a:

wherein, Is a sample-invariant dictionary comprising d convolution kernels representing common patterns of different metal artifacts, in short,A common database representing different metal artifact types in all CT images with metal artifacts; is a weighting coefficient that varies with the sample; representing a particular convolution kernel, which represents a pattern in which a certain metal artifact repeatedly appears, p x p being the convolution kernel size; Is a feature layer representing the location where the local pattern repeatedly appears; n is the number of true specific convolution kernels used to encode a; is a two-dimensional plane convolution operation, and p and d are positive integers.

In addition, in the case of the optical fiber,And

That is, the basic artifact dictionary is a sample-invariant convolution dictionaryAnd includes a first number (d) of artifact convolution kernels. A second number of adaptive convolution kernels for the image samples may be determined by a plurality of artifact convolution kernels in the base artifact dictionary and a weighting coefficient (K) that varies with the sample

And (3) bringing the equation (2) into the equation (1) to obtain a model of a non-metal region corresponding to the final CT image with metal artifact, wherein the model is as follows:

Fig. 5A shows a schematic structure of an adaptive convolutional dictionary network, including a T-level network, according to an embodiment of the present disclosure. In each level of network, weighting coefficient K and characteristic layer are respectively carried out And updating the clean CT image X. Fig. 5B shows a schematic structure of each level of network according to an embodiment of the present disclosure.

The image processing method according to the embodiment of the present disclosure will be described first with reference to fig. 4, 5A and 5B, and then the mathematical model of the adaptive convolution dictionary network of fig. 5A and 5B.

Fig. 3B is a schematic flowchart 320 illustrating a model use procedure of an image processing method for artifact removal according to an embodiment of the present disclosure.

In step S321, an input image to be processed is acquired.

According to an embodiment of the present disclosure, the input image to be processed may be an image obtained after applying its corresponding image mask (I) to the original CT image, i.e., i.y. Alternatively, the input image to be processed may be an image with artifacts, or may be an image with artifacts subjected to preprocessing (e.g., denoising processing, normalization processing). The image mask (I) in the present disclosure is a mask for a region to be studied or a region of interest.

In step S322, the input image is processed using an adaptive convolutional dictionary network to obtain a artifact-removed processed image.

Wherein the adaptive convolutional dictionary network is trained based on an artifact database (D) and comprises a T-level network (as shown in fig. 5A), wherein a first-level network obtains a first-level image feature and a first-level artifact-removed image output by the first-level network based on the input image; the T-level network obtains and outputs T-level image features and T-level artifact removed images output by the T-level network based at least in part on the T-1-level image features and the T-1-level artifact removed images output by the T-1-level network, wherein T is greater than 1 and less than or equal to T; and the T-stage network obtains and outputs the T-stage artifact removal image output by the T-stage network as the artifact removal processed image based on the T-1 stage image characteristics and the T-1 stage artifact removal image output by the T-1 stage network. The processed image after removing the artifacts can be output by a display mode or can be output to a user by a film mode, a drawing mode and the like.

According to an embodiment of the present disclosure, the adaptive convolutional dictionary network includes a basic artifact dictionary that is a convolutional dictionary that does not vary with an input image and includes a first number of artifact convolutional kernels, in a t-th level network, first determining a t-th level weighting coefficient of the t-th level network, then determining a second number of adaptive convolutional kernels for the t-th level network by the first number of artifact convolutional kernels and the t-th level weighting coefficient in the basic artifact dictionary; a t-th stage artifact-removed image is then determined based on the second number of adaptive convolution kernels and image features of the t-th stage network.

According to an embodiment of the present disclosure, each level of network may include a weighting coefficient update network, an image feature update network, and an artifact-removed image update network, wherein the weighting coefficient update network, the image feature update network, and the artifact-removed image update network include a residual network structure and a normalization processing layer. It should be appreciated that the network structure of the weighting factor update network, the image feature update network, and the artifact-removal image update network may be varied, with the network structure generally being related to the dimensional characteristics of the solution variables to which the network corresponds. For example, the weighting coefficient update network solution variables are coefficients, so typically include linear layers, while the image feature update network and artifact removal image update network solution variables are two-dimensional images, so typically include convolution layers.

According to an embodiment of the disclosure, the artifact may be a metal artifact, the input image to be processed may be a CT image with a metal artifact, and the image mask (I) may be a non-metal area mask corresponding to the CT image with a metal artifact. In this embodiment, each level of network may include a weighting coefficient update network, a metal artifact image feature update network, and a metal artifact removal image update network, wherein the weighting coefficient update network, the metal artifact image feature update network, and the metal artifact removal image update network include a residual network structure; and the weighting coefficient updating network includes: a linear layer, a modified linear Unit (RECTIFIED LINEAR Unit, reLU) layer, a cross-link layer, and a batch normalization (Batch Normalization, BN) layer; the metal artifact image feature update network comprises: a convolutional layer, a BN layer, a ReLU layer, and a cross-link layer; the metal artifact removal image update network comprises: convolutional layer, BN layer, reLU layer, and cross-link layer.

Fig. 3A is a schematic flow chart 310 illustrating a neural network training process for an image processing method for artifact removal according to an embodiment of the present disclosure.

In step S311, a training data set for training a neural network is created, wherein the training data set includes a plurality of sets of image samples, each set of image samples including an artifact-free image (Y) and an artifact-free image (X) and an image mask (I) corresponding thereto.

According to the embodiment of the disclosure, the disclosed image library and different types of metal masks (masks) can be used for synthesizing the artifact according to a data simulation flow, and the artifact-carrying image and the metal mask (M _e) corresponding to the artifact-carrying image are used as training data. For example, in an application scenario for removing artifacts from a CT image with metal artifacts, the disclosed DeepLesion image library and different types of metal masks may be used to synthesize metal artifacts according to a data simulation flow, and the CT image with metal artifacts and the metal masks are used as training data. It should be appreciated that the different types of masks (masks) or different types of metal masks used to set the size and shape of the implant in the CT image may be simulated to obtain their corresponding artifact shape based on the size and shape of the implant. The correspondence between the metal mask (M _e) and the non-metal region mask (I) can be deduced from each other, i.e. M _e +i=1.

In accordance with an embodiment of the present disclosure, in order to process sample data of a larger data range, the establishing a training data set for training the neural network may further include: and carrying out normalization processing on the pixel value of the artifact-carrying image (Y).

Furthermore, in order to obtain the diversity of the sample data, the establishing a training data set for training the neural network may further include: and carrying out random clipping on the image with the artifact (Y) to obtain an image block, and carrying out random overturning processing on the image block according to a preset probability.

For example, the range of values of the artifact-bearing image in the training data may be cropped (e.g., to remove values that are too large or negative such that the unwanted range of values is no longer preserved, but the desired tissue information is not lost), and then normalized such that the pixel values of the image are stuck within the threshold [0, 1]. Optionally, the normalized data may also be reconverted to the [0,255] range to facilitate computer processing.

According to embodiments of the present disclosure, each training image and the corresponding mask may also be randomly cropped to form smaller image blocks (e.g., image blocks that may be 64x64 pixels in size), and then randomly horizontal mirror-inverted and randomly vertical mirror-inverted, respectively, with a predetermined probability (e.g., may be 0.5) to obtain more diversified training sample data.

In step S312, for at least one of the plurality of sets of image samples, the artifact-removed image (Y) is processed using an adaptive convolutional dictionary network to obtain a processed image.

According to an embodiment of the present disclosure, as shown in fig. 4, the adaptive convolution dictionary network may include a basic artifact dictionary that is a sample-invariant convolution dictionary and includes a first number of artifact convolution kernels, and a second number of adaptive convolution kernels for the image samples may be determined from a plurality of artifact convolution kernels in the basic artifact dictionary and sample-variant weighting coefficients. Moreover, an artifact image in the artifact image may be determined by a convolution of the second number of adaptive convolution kernels with image features of the artifact image and the artifact image is removed from the artifact image to obtain the processed image.

In step S313, the adaptive convolutional dictionary network is iteratively trained based on the artifact-free image (X) and the processed image, and an objective function processed by the image mask (I), to optimize network parameters of the adaptive convolutional dictionary network.

According to an embodiment of the disclosure, the objective function may be a loss objective function constructed based on the artifact-free image (X) and the processed image, wherein the iteratively training the adaptive convolutional dictionary network based on the artifact-free image (X) and the processed image, and the objective function to optimize network parameters of the adaptive convolutional dictionary network further comprises: a loss objective function is calculated and its results are back-propagated to the adaptive convolutional dictionary network and network parameters of the adaptive convolutional dictionary network are optimized based on an adaptive moment estimation (Adaptive moment estimation, adam) algorithm.

According to an embodiment of the disclosure, the objective function may also be a loss objective function constructed based on the artifact-free image (X) and the processed image and processed by an image mask (I).

According to an embodiment of the disclosure, the first number of artifact convolution kernels indicates an artifact pattern, and the image feature indicates a location of the artifact pattern, wherein the adaptive convolution dictionary network includes a T-level network, wherein in the T-level network, a weighting coefficient and an image feature output by the T-1-level network are updated by using an iterative update rule based on a near-end gradient descent, so as to obtain the weighting coefficient and the image feature of the T-level network, where T is an integer greater than 1 and less than or equal to T

According to an embodiment of the present disclosure, the image processing method for artifact removal may further include: and after training is completed, testing the self-adaptive convolution dictionary network to evaluate the effect of image processing. Wherein said testing said adaptive convolutional dictionary network comprises: preprocessing an image with an artifact to be tested, and inputting the image into the self-adaptive convolution dictionary network; and processing the image with the artifact to be tested by using the self-adaptive convolution dictionary network to obtain a processed image with the artifact removed.

According to an embodiment of the present disclosure, the artifact may be a metal artifact and the artifact-bearing image may be a CT image with a metal artifact. Thus, the training data set comprises a plurality of groups of CT image samples, each group of image samples comprising a CT image with metal artifacts and a CT image without metal artifacts corresponding to the CT image with metal artifacts; the adaptive convolution dictionary network includes a base metal artifact dictionary that is a metal artifact convolution dictionary that does not vary with CT image samples and includes a first number of metal artifact convolution kernels and a second number of adaptive convolution kernels for the CT image samples are determined from a plurality of metal artifact convolution kernels in the base metal artifact dictionary and weighting coefficients that vary with the CT image samples, wherein the first number of metal artifact convolution kernels indicates a metal artifact pattern and the metal artifact image features indicate a location of the metal artifact pattern.

According to an embodiment of the disclosure, in an application scenario for CT image removal artifacts, each level of network may include a weighting coefficient update network, a metal artifact image feature update network, and a metal artifact removal image update network, wherein the weighting coefficient update network, the metal artifact image feature update network, and the metal artifact removal image update network include a residual network structure; and the weighting coefficient updating network includes: a linear layer, a modified linear Unit (RECTIFIED LINEAR Unit, reLU) layer, a cross-link layer, and a batch normalization (Batch Normalization, BN) layer; the metal artifact image feature update network comprises: a convolutional layer, a BN layer, a ReLU layer, and a cross-link layer; the metal artifact removal image update network comprises: convolutional layer, BN layer, reLU layer, and cross-link layer.

Continuing with the description below of the mathematical model of fig. 4 for removing metal artifacts for a metal-artifact-bearing CT image as an example, the structure of a mathematical model and an adaptive convolutional dictionary network specifically describing the image processing method for artifact removal described in the present disclosure will be described with reference to fig. 4, 5A and 5B.

According to the embodiment of the disclosure, the common database representing different metal artifact types in all CT images with metal artifacts can be used for processing the CT imagesAnd constructing a convolutional layer, constructing an adaptive convolutional dictionary network based on the convolutional layer, and obtaining optimization parameters of the adaptive convolutional dictionary network through training data set end-to-end training.

For the mathematical model shown in equation (4) above:

The solution target is to estimate K from Y, And X, the corresponding optimization problem is:

subject to‖K_n‖₂＝1,n＝1,2,…,N (5)

Wherein, alpha, beta and gamma are compromise parameters, f ₁(·)、f₂ (& gt) and f ₃ (& gt) are regular terms, and respectively represent a weighting coefficient K and a characteristic layer And the a priori structure of the clean CT image X, which can be designed as a neural network module for solution.

To solve the optimization problem in (5), the weighting coefficient K and the feature layer can be updated alternately by adopting a near-end gradient technologyAnd a clean CT image X. The method comprises the following steps:

Updating K:

in the (t+1) th iteration, K may be updated as:

subject to‖K_n‖₂＝1,n＝1,2,…,N (6)

the corresponding quadratic approximation form is:

Wherein, Ω= { k||||k _n‖₂ =1, n=1, 2, …, N };

η ₁ is the update step size, which can be derived:

wherein, Is a deep convolution operation; Representing the expansion of the tensor in the 3 rd dimension; vec (·) represents the vectorization operation.

Equation (7) can be equivalently written as:

for a general prior term f ₁ (·), equation (9) can be written as:

wherein,

Is a near-end operator, related to the canonical term f ₁ (); omega can be controlled by a pair ofA normalization operation is introduced.

Updating

Similar to the update of K, in the (t + 1) th iteration,Can be updated as:

wherein, eta ₂ is the update step length, For a general prior term f ₂ (·), equation (11) can be written as:

wherein, Is a near-end operator, related to the canonical term f ₂ ();

is a transpose convolution operation.

Updating X:

given K ^(t+1) and Can be updated as:

wherein, Further, the update rule of X may be obtained as:

wherein, Is a near-end operator, related to the canonical term f ₃ (.).

By expanding the updated formulas (10) (12) (14), a complete adaptive convolutional dictionary network (Adaptive Convolutional Dictionary Network, ACDNet) can be ultimately constructed, wherein each network has good physical interpretation.

A schematic structure of an adaptive convolutional dictionary network, including a T-level network, according to an embodiment of the present disclosure is shown in fig. 5A. In each level of network, weighting coefficient K and characteristic layer are respectively carried outAnd the processed CT image X. A schematic structure of each level of network according to an embodiment of the present disclosure is shown in fig. 5B.

According to the embodiment of the disclosure, ACDNet as shown in fig. 5A is composed of T stages (i.e., a T-stage network), in each stage, the corresponding network structure is composed of K-net in turn,And X-net, which are used to realize K,And iterative updating of X.

Next, the correspondence between the network structure and the above mathematical model in fig. 5A and 5B is explained.

In fig. 5A and 5B, in particular,

Wherein the method comprises the steps ofIs a residual structure, specifically: linear layer, reLU layer, linear layer, cross-link layer, normalized operation layer at dimension d;

Wherein the method comprises the steps of Is composed of 3 residual blocks, each of which in turn comprises: convolutional layer, BN layer, reLU layer, convolutional layer, BN layer, and cross-link layer;

Wherein the method comprises the steps of Is composed of 3 residual blocks, each of which in turn comprises: convolutional layer, BN layer, reLU layer, convolutional layer, BN layer, and cross-link layer.

As can be seen from fig. 5A, the first-stage network obtains a first-stage image feature M ⁽¹⁾ and a first-stage artifact-removed image X ⁽¹⁾ output by the first-stage network based on the input image; the T-level network obtains and outputs T-level image features M ^(t) and T-level artifact removed images X ^(t) output by the T-level network based at least in part on T-1 level image features M ^(t-1) and T-1 level artifact removed images X ^(t-1) output by the T-1 level network, wherein T is greater than 1 and less than or equal to T; the T-stage network obtains and outputs a T-stage artifact removed image X ^(T) output by the T-stage network as the artifact removed processed image based on the T-1 stage image feature M ^(T-1) and the T-1 stage artifact removed image X ^(T-1) output by the T-1 stage network.

For each level of the network in the adaptive convolutional dictionary network, its iterative solution process is shown in FIG. 5B. And in the T-stage network, updating the weighting coefficient and the image characteristic output by the T-1 stage network by using an iteration update rule based on the near-end gradient descent to obtain the weighting coefficient and the image characteristic of the T-stage network, wherein T is an integer which is more than 1 and less than or equal to T. K-net, M-net and X-net are sequentially and iteratively solved in a serial mode.

Iterative training may be performed for an adaptive convolutional dictionary network based on artifact-free images (X) and processed images, and an objective function processed by an image mask (I) to optimize network parameters of the adaptive convolutional dictionary network. Wherein the objective function may be a loss objective function constructed based on the artifact-free image (X) and the processed image, i.e

Where μ _t is a compromise parameter, ω ₁ and ω ₂ are weights used to balance the losses. For example, in a simulation experiment, μ _t＝0.1(t＝0,1,…,T-1),μ_T＝1,ω₁＝ω₂＝5×10^-4, t=10 can be set.

In accordance with embodiments of the present disclosure, adam-based algorithms (Adaptive moment estimation) may be employed to update solution optimization parameters, including,Convolution kernelStep sizes η ₁、η₂ and η ₃. In each iteration process, calculating a prediction result error and reversely propagating the prediction result error to the convolutional neural network model, calculating a gradient and updating parameters of the convolutional neural network model.

Fig. 6 is a schematic flow chart illustrating an image processing procedure for artifact removal according to an embodiment of the present disclosure.

As shown in fig. 6, an image processing procedure for artifact removal according to an embodiment of the present disclosure includes a neural network training phase and a testing phase.

The image with the artifacts can be preprocessed in the training stage of the neural network to establish a training data set for training the neural network, wherein the training data set comprises a plurality of groups of image samples, and each group of image samples comprises an image (Y) with the artifacts and an image mask (I) without the artifacts corresponding to the image (Y) with the artifacts. The computer then iterates the training ACDNet according to the neural network training settings based on the preprocessed image samples, wherein during the training process, the parameters of ACDNet are updated based on the predetermined objective function and Adam optimization algorithm. The trained model is saved if a predetermined number of iterations is reached during the training process, and training is continued ACDNet if the predetermined number of iterations is not reached.

It should be appreciated that the use of up to a predetermined number of iterations as a criterion for the completion of training of decision ACDNet is herein intended to prevent network overfitting. Optionally, the optimized image may also be visually output, and the continued training ACDNet may be stopped after the optimized image is confirmed to meet the requirements.

In the test stage, the input image to be processed and the image mask (I) corresponding to the input image are input into a computer, the computer loads the trained model, an image with the artifacts removed is obtained through ACDNet forward calculation, and the computer can output the image with the artifacts removed for reference by a user.

Similarly, in the actual use process, the image processing process is similar to the test stage, and thus, a description thereof will not be repeated.

Fig. 7A is a composition diagram 710 illustrating an image processing apparatus for artifact removal according to an embodiment of the present disclosure, the apparatus 710 being used for a neural network training process at the time of image processing.

According to an embodiment of the present disclosure, the image processing apparatus 710 for artifact removal may include: a training data set creation module 711, an adaptive convolutional dictionary network 712, and a training module 713.

Wherein the training data set creation module 711 may be configured to: a training data set for training a neural network is established, wherein the training data set comprises a plurality of groups of image samples, and each group of image samples comprises an image (Y) with artifacts and an image mask (I) without artifacts corresponding to the image (Y).

Optionally, the establishing a training data set for training the neural network may further include at least one of: normalizing the pixel values of the artifact-bearing image (Y); and performing random cropping on the image with the artifact (Y) to obtain an image block, and performing random flipping processing on the image block according to a preset probability.

The adaptive convolutional dictionary network 712 may be configured to: and performing artifact removal processing on the artifact-bearing image (Y) for at least one of the plurality of sets of image samples to obtain a processed image.

The adaptive convolution dictionary network 712 may include a basic artifact dictionary that is a sample-invariant convolution dictionary and that includes a first number of artifact convolution kernels, a second number of adaptive convolution kernels for the image samples may be determined from a plurality of artifact convolution kernels in the basic artifact dictionary and sample-variant weighting coefficients. Moreover, an artifact image in the artifact image may be determined by a convolution of the second number of adaptive convolution kernels with image features of the artifact image and the artifact image is removed from the artifact image to obtain the processed image.

Training module 713 may be configured to: the adaptive convolutional dictionary network is iteratively trained based on the artifact-free image (X) and the processed image, and an objective function processed by the image mask (I), to optimize network parameters of the adaptive convolutional dictionary network.

According to an embodiment of the disclosure, each level of network includes a weighting coefficient update network, an image feature update network, and an artifact-removed image update network, wherein the weighting coefficient update network, the image feature update network, and the artifact-removed image update network include a residual network structure and a normalization processing layer.

According to an embodiment of the present disclosure, the image processing apparatus 710 for artifact removal may further include: a test module 714 configured to: after training is completed, testing the adaptive convolutional dictionary network, wherein the testing the adaptive convolutional dictionary network comprises: preprocessing an image with an artifact to be tested, and inputting the image into the self-adaptive convolution dictionary network; and processing the image with the artifact to be tested by using the self-adaptive convolution dictionary network to obtain a processed image with the artifact removed.

Fig. 7B is a composition diagram 720 showing an image processing apparatus for artifact removal according to an embodiment of the present disclosure, the apparatus 720 being used for a model use procedure at the time of image processing.

According to an embodiment of the present disclosure, the image processing apparatus 720 for artifact removal may include: an image acquisition module 721, an image processing module 722.

Wherein the image acquisition module 721 may be configured to: an input image to be processed is acquired.

According to an embodiment of the present disclosure, the input image to be processed may be an image obtained after applying the original CT image to its corresponding image mask. For example, the image acquisition module 721 may receive an original CT image and an image mask corresponding thereto, and apply the original CT image to the image mask corresponding thereto, thereby acquiring an input image to be processed.

According to the embodiments of the present disclosure, the input image to be processed may be an image with artifacts, or may be an image with artifacts that is subjected to preprocessing (e.g., denoising processing, normalization processing).

The image processing module 722 may be configured to: and processing the input image by using an adaptive convolution dictionary network to obtain a processed image with the artifacts removed.

Wherein the adaptive convolutional dictionary network is trained based on an artifact database (D) and comprises a T-level network, wherein a first-level network obtains a first-level image feature and a first-level artifact removed image output by the first-level network based on the input image; the T-level network obtains and outputs T-level image features and T-level artifact removed images output by the T-level network based at least in part on the T-1-level image features and the T-1-level artifact removed images output by the T-1-level network, wherein T is greater than 1 and less than or equal to T; and the T-stage network obtains and outputs the T-stage artifact removal image output by the T-stage network as the artifact removal processed image based on the T-1 stage image characteristics and the T-1 stage artifact removal image output by the T-1 stage network.

According to an embodiment of the present disclosure, the adaptive convolutional dictionary network includes a basic artifact dictionary that is a convolutional dictionary that does not vary with an input image and includes a first number of artifact convolutional kernels, a t-level weighting coefficient of a t-level network is determined, a second number of adaptive convolutional kernels for the t-level network is determined by a plurality of artifact convolutional kernels in the basic artifact dictionary and the t-level weighting coefficient; and determining a t-th stage artifact-removed image based on the second number of adaptive convolution kernels and image features of the t-th stage network.

In general, the various example embodiments of the disclosure may be implemented in hardware or special purpose circuits, software, firmware, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device. While aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that the blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

For example, a method or apparatus according to embodiments of the present disclosure may also be implemented by means of the architecture of computing device 3000 shown in fig. 8. As shown in fig. 8, computing device 3000 may include a bus 3010, one or more CPUs 3020, a Read Only Memory (ROM) 3030, a Random Access Memory (RAM) 3040, a communication port 3050 connected to a network, an input/output component 3060, a hard disk 3070, and the like. A storage device in the computing device 3000, such as a ROM 3030 or hard disk 3070, may store various data or files for processing and/or communication of the methods provided by the present disclosure and program instructions for execution by the CPU. The computing device 3000 may also include a user interface 3080. Of course, the architecture shown in FIG. 8 is merely exemplary, and one or more components of the computing device shown in FIG. 8 may be omitted as may be practical in implementing different devices.

According to yet another aspect of the present disclosure, a computer-readable storage medium is also provided. Fig. 9 shows a schematic diagram 4000 of a storage medium according to the present disclosure.

As shown in fig. 9, computer-readable instructions 4010 are stored on a computer storage medium 4020. When the computer readable instructions 4010 are executed by the processor, a method according to an embodiment of the disclosure described with reference to the above figures may be performed. The computer readable storage medium in embodiments of the present disclosure may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be Random Access Memory (RAM), which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate synchronous dynamic random access memory (ddr SDRAM), enhanced Synchronous Dynamic Random Access Memory (ESDRAM), synchronous Link Dynamic Random Access Memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that the memory of the methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory. It should be noted that the memory of the methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

Embodiments of the present disclosure also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from a computer-readable storage medium, the processor executing the computer instructions, causing the computer device to perform a method according to an embodiment of the present disclosure.

In summary, embodiments of the present disclosure provide an image processing method, apparatus, computer program product, and storage medium for artifact removal. The image processing method for artifact removal provided by the present disclosure includes: establishing a training data set for training a neural network, wherein the training data set comprises a plurality of groups of image samples, and each group of image samples comprises an image (Y) with artifacts and an image mask (I) without artifacts corresponding to the image (Y); performing an artifact removal process on the artifact-bearing image (Y) using an adaptive convolutional dictionary network for at least one of the plurality of sets of image samples to obtain a processed image, iteratively training the adaptive convolutional dictionary network based on the artifact-free image (X) and the processed image, and an objective function processed by the image mask (I) to optimize network parameters of the adaptive convolutional dictionary network, wherein the adaptive convolutional dictionary network comprises a basic artifact dictionary that is a sample-invariant convolutional dictionary and comprises a first number of artifact convolution kernels, and determining a second number of adaptive convolution kernels for the image samples from a plurality of artifact convolution kernels and sample-variant weighting coefficients in the basic artifact dictionary, wherein an artifact image in the artifact-bearing image is determined from a convolution of the second number of adaptive convolution kernels with image features of the artifact-bearing image, and removing the processed image from the artifact-bearing image.

It is noted that the flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises at least one executable instruction for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The present disclosure uses specific words to describe embodiments of the disclosure. Such as "first/second embodiment," "an embodiment," and/or "some embodiments," means a particular feature, structure, or characteristic associated with at least one embodiment of the present disclosure. Thus, it should be emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various positions in this specification are not necessarily referring to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of the present disclosure may be combined as suitable.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

The foregoing is illustrative of the present invention and is not to be construed as limiting thereof. Although a few exemplary embodiments of this invention have been described, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this invention. Accordingly, all such modifications are intended to be included within the scope of this invention as defined in the following claims. It is to be understood that the foregoing is illustrative of the present invention and is not to be construed as limited to the specific embodiments disclosed, and that modifications to the disclosed embodiments, as well as other embodiments, are intended to be included within the scope of the appended claims. The invention is defined by the claims and their equivalents.

Claims

1. An image processing method for artifact removal, comprising:

Establishing a training data set for training a neural network, wherein the training data set comprises a plurality of groups of image samples, and each group of image samples comprises an image (Y) with artifacts and an image mask (I) without artifacts corresponding to the image (Y);

For at least one of the plurality of sets of image samples,

Performing an artifact removal process on the artifact-bearing image (Y) using an adaptive convolutional dictionary network to obtain a processed image,

Iteratively training the adaptive convolutional dictionary network based on the artifact-free image (X) and the processed image, and an objective function processed by the image mask (I) to optimize network parameters of the adaptive convolutional dictionary network,

Wherein the adaptive convolutional dictionary network comprises a basic artifact dictionary that is a sample-invariant convolutional dictionary and that comprises a first number of artifact convolutional kernels, and a second number of adaptive convolutional kernels for the image samples is determined by a plurality of artifact convolutional kernels in the basic artifact dictionary and sample-variant weighting coefficients,

Wherein an artifact image in the artifact image is determined by a convolution of the second number of adaptive convolution kernels with image features of the artifact image and the artifact image is removed from the artifact image to obtain the processed image.

2. The image processing method of claim 1 wherein said first number of artifact convolution kernels indicates an artifact pattern, said image features indicate a location of said artifact pattern,

The self-adaptive convolution dictionary network comprises a T-level network, wherein in the T-level network, the weighting coefficient and the image characteristic output by the T-1-level network are updated by using an iteration update rule based on the gradient descent of a near end, so as to obtain the weighting coefficient and the image characteristic of the T-level network, wherein T is an integer which is more than 1 and less than or equal to T.

3. The image processing method of claim 2, wherein each level of network comprises a weighting factor update network, an image feature update network, and an artifact-removing image update network, wherein,

The weighting coefficient updating network, the image characteristic updating network and the artifact removal image updating network comprise a residual network structure and a normalization processing layer.

4. The image processing method of claim 1, wherein the establishing a training data set for training a neural network further comprises at least one of:

Normalizing the pixel values of the artifact-bearing image (Y); and

And carrying out random clipping on the image with the artifact (Y) to obtain an image block, and carrying out random overturning processing on the image block according to a preset probability.

5. The image processing method according to claim 1, wherein the objective function is a loss objective function constructed based on the artifact-free image (X) and the processed image,

Wherein said iteratively training said adaptive convolutional dictionary network based on said artifact-free image (X) and said processed image, and an objective function to optimize network parameters of said adaptive convolutional dictionary network further comprises:

A loss objective function is calculated and its results are back-propagated to the adaptive convolutional dictionary network and network parameters of the adaptive convolutional dictionary network are optimized based on an adaptive moment estimation (Adaptive moment estimation, adam) algorithm.

6. The image processing method according to claim 1, further comprising: after training is completed, the adaptive convolutional dictionary network is tested,

Wherein said testing said adaptive convolutional dictionary network comprises:

Preprocessing an image with an artifact to be tested, and inputting the image into the self-adaptive convolution dictionary network;

and processing the image with the artifact to be tested by using the self-adaptive convolution dictionary network to obtain a processed image with the artifact removed.

7. The image processing method of claim 1, wherein the artifact is a metal artifact, the artifact-bearing image is a CT image with a metal artifact,

The training data set comprises a plurality of groups of CT image samples, and each group of image samples comprises a CT image with metal artifacts, a CT image without metal artifacts corresponding to the CT image with metal artifacts and a non-metal area mask;

The adaptive convolution dictionary network includes a base metal artifact dictionary that is a metal artifact convolution dictionary that does not vary with CT image samples and includes a first number of metal artifact convolution kernels and a second number of adaptive convolution kernels for the CT image samples are determined from a plurality of metal artifact convolution kernels in the base metal artifact dictionary and weighting coefficients that vary with the CT image samples, wherein the first number of metal artifact convolution kernels indicates a metal artifact pattern and the metal artifact image features indicate a location of the metal artifact pattern.

8. The image processing method of claim 7, wherein each level of network comprises a weighting coefficient update network, a metal artifact image feature update network, and a metal artifact removal image update network, wherein,

The weighting coefficient updating network, the metal artifact image feature updating network and the metal artifact removal image updating network comprise residual error network structures; and

The weighting coefficient updating network includes: a linear layer, a modified linear Unit (RECTIFIED LINEAR Unit, reLU) layer, a cross-link layer, and a batch normalization (Batch Normalization, BN) layer;

the metal artifact image feature update network comprises: a convolutional layer, a BN layer, a ReLU layer, and a cross-link layer;

The metal artifact removal image update network comprises: convolutional layer, BN layer, reLU layer, and cross-link layer.

9. An image processing method for artifact removal, comprising:

acquiring an input image to be processed;

processing the input image using an adaptive convolutional dictionary network to obtain a artifact-removed processed image,

10. The image processing method of claim 9, wherein the adaptive convolutional dictionary network comprises a basic artifact dictionary that is a convolutional dictionary that does not vary with the input image and comprises a first number of artifact convolutional kernels,

Determining a t-th level weighting coefficient for a t-th level network, determining a second number of adaptive convolution kernels for the t-th level network by a plurality of artifact convolution kernels in the basic artifact dictionary and the t-th level weighting coefficient; and determining a t-th stage artifact-removed image based on the second number of adaptive convolution kernels and image features of the t-th stage network.

11. The image processing method of claim 9, wherein each level of network comprises a weighting factor update network, an image feature update network, and an artifact-removing image update network, wherein,

12. The image processing method according to claim 9, wherein the input image to be processed is an image to which an image mask (I) is applied.

13. The image processing method as claimed in claim 12, wherein the artifact is a metal artifact, the input image to be processed is a CT image with a metal artifact, the image mask (I) is a non-metal area mask corresponding to the CT image with a metal artifact,

Wherein each level of network comprises a weighting coefficient updating network, a metal artifact image feature updating network, and a metal artifact removal image updating network, wherein,

14. An image processing apparatus for artifact removal, comprising:

a training data set creation module configured to: establishing a training data set for training a neural network, wherein the training data set comprises a plurality of groups of image samples, and each group of image samples comprises an image (Y) with artifacts and an image mask (I) without artifacts corresponding to the image (Y);

an adaptive convolutional dictionary network configured to: performing artifact removal processing on the artifact-bearing image (Y) for at least one of the plurality of sets of image samples to obtain a processed image;

a training module configured to: iteratively training the adaptive convolutional dictionary network based on the artifact-free image (X) and the processed image, and an objective function processed by the image mask (I), to optimize network parameters of the adaptive convolutional dictionary network;

15. An image processing apparatus for artifact removal, comprising:

an image acquisition module configured to: acquiring an input image to be processed;

an image processing module configured to: processing the input image by using an adaptive convolution dictionary network to obtain a processed image from which artifacts are removed;

16. A computer program product comprising computer software code for implementing the method of any of claims 1-13 when executed by a processor.

17. A computer readable storage medium having stored thereon computer executable instructions for implementing the method of any of claims 1-13 when executed by a processor.