CN117115031A

CN117115031A - CBCT metal artifact removal method and system based on unpaired learning

Info

Publication number: CN117115031A
Application number: CN202311113460.0A
Authority: CN
Inventors: 姜斯浩; 杨振华
Original assignee: Changzhou Boen Zhongding Medical Technology Co ltd
Current assignee: Changzhou Boen Zhongding Medical Technology Co ltd
Priority date: 2023-08-31
Filing date: 2023-08-31
Publication date: 2023-11-24

Abstract

The embodiment of the application provides a CBCT metal artifact removal method and system based on unpaired learning, and belongs to the field of computer oral restoration. The method comprises the steps of obtaining an artifact data set and a clean data set, wherein the artifact data set and the clean data set are unpaired data sets; performing data processing on the artifact data set and the clean data set to obtain a processed training data set; training the demetallization artifact model by using a training data set to obtain an initial demetallization artifact model; restoring the predicted artifact removal data output by the initial artifact removal model to obtain original data corresponding to the predicted artifact removal data; optimizing an initial metal artifact removal model by using the predicted artifact removal data and the original data to obtain a target metal artifact removal model; the CT image with metal artifact to be removed is input into a target metal artifact removal model to obtain a target CT image with metal artifact removed.

Description

CBCT metal artifact removal method and system based on unpaired learning

Technical Field

The application relates to the field of computer oral repair, in particular to a CBCT metal artifact removal method and system based on unpaired learning.

Background

Oromaxillofacial Cone Beam CT (CBCT for short) scanning is a medical imaging technique used in radiology to obtain detailed images of the oral cavity for diagnostic purposes. Unfortunately, the artifacts caused by metal implants appear as dark and bright fringes in CBCT, greatly reducing the accuracy of image quality and CT values, severely affecting clinical diagnosis, and thus reducing metal artifacts (Meta lArt ifact Reduct ion, MAR) is an urgent problem to be solved in CBCT imaging.

Traditional methods such as projection data correction and iterative reconstruction have limitations, and limit the large-scale application of the methods in clinical medical treatment. Thus, reducing metal artifacts in CBCT images has heretofore remained a research hotspot and difficulty in the field of medical imaging.

In recent years, deep Learning (DL) has been greatly advanced in the fields of image processing and pattern recognition. For example, convolutional neural networks (Convol utional Neural Networks, CNN) have been applied to low dose CT reconstruction and artifact reduction medical imaging. U-Net is a full convolution network with many applications in biomedical image segmentation, such as brain image segmentation and liver image segmentation. MAR methods based on DL techniques have also been widely studied, which generally employ deep convolutional neural networks to map CBCT images with metal artifacts end-to-end to artifact-free images to remove the metal artifacts and restore the original anatomy to improve the accuracy of clinical diagnosis.

Many teams currently utilize image transformations that generate a framework of a countermeasure network (Generative Adversar ial Network, GAN), by constructing a generator with feature delivery capabilities, to build a mapping from an input image to a target image, such that the generated image has features of the target image; and then distinguishing the generated image and the target image by using a discriminator, performing resistance training, and completing complex image conversion.

However, it is difficult to collect paired noisy and clean images in most real-world applications, so most teams use simulation data to learn the artifact-free image-to-artifact-free task. However, the simulation data does not represent the real artifact data, such that the learning model performs poorly on the real artifact data.

Therefore, how to solve the above-mentioned problems is a problem that needs to be solved at present.

Disclosure of Invention

The application provides a CBCT metal artifact removal method and system based on unpaired learning, aiming at improving the problems.

In a first aspect, the present application provides a CBCT metal artifact removal method based on unpaired learning, the method comprising:

obtaining an artifact data set and a clean data set, wherein the artifact data set and the clean data set are unpaired data sets;

performing data processing on the artifact data set and the clean data set to obtain a processed training data set;

training the demetallization artifact model by using the training data set to obtain an initial demetallization artifact model;

restoring the predicted artifact removal data output by the initial artifact removal model to obtain original data corresponding to the predicted artifact removal data;

optimizing the initial demetallization artifact model by using the predicted demetallization artifact data and the original data to obtain a target demetallization artifact model;

inputting the CT image with the metal artifact to be removed into the target metal artifact removal model to obtain a target CT image with the metal artifact removed.

In a possible embodiment, the performing data processing on the artifact data set and the clean data set to obtain a processed training data set includes:

converting each three-dimensional CBCT image in the artifact dataset and the clean dataset into a plurality of two-dimensional image data;

fixing each two-dimensional image data to a preset size to obtain a plurality of new two-dimensional image data, wherein each new two-dimensional image data is single-channel data;

separating the width and height of each new two-dimensional image data to obtain multi-channel two-dimensional image data;

expanding the multi-channel two-dimensional image data by using filling operation to obtain expanded two-dimensional image data;

and carrying out standardization processing on the expanded two-dimensional image data by using a k-s (i) gma conversion formula to obtain a processed training data set.

In a possible embodiment, the converting each three-dimensional CBCT image in the artifact dataset and the clean dataset into a plurality of two-dimensional image data includes:

and intercepting the two-dimensional data of each layer of each three-dimensional CBCT image in the artifact dataset and the clean dataset according to the CT value range to obtain a plurality of two-dimensional image data.

In a possible embodiment, said separating the width and height of each of said new two-dimensional image data to obtain multi-channel two-dimensional image data comprises:

the width and the height of the multi-channel two-dimensional image data are separated into 1/2 of the original width and the height, and four-dimensional data are generated;

the four-dimensional data are combined and converted into 4-channel two-dimensional image data.

In a possible embodiment, the expanding the multi-channel two-dimensional image data by using a filling operation to obtain expanded two-dimensional image data includes:

and carrying out data edge filling on the 4-channel two-dimensional image data according to a size filling algorithm to obtain the expanded two-dimensional image data.

In one possible embodiment, the k-s i gma satisfies:

wherein x and f represent the input artifact image and the output artifact image, k and σ, respectively ² And fitting the obtained parameters to the artifact images in the training data set by using a linear equation.

In one possible embodiment, the initial demetallization artifact model comprises: the device comprises a generator module, an image blurring module and a discriminator module; the generator module, the image blurring module and the discriminator module are sequentially connected;

the generator module receives the training data set and correspondingly generates an image for predicting the artifact;

the image blurring module sharpens the image of the predicted artifact removal and the clean image in the training data set to generate a sharpened image;

the discriminator module outputs a predicted probability value according to the sharpened image.

In a possible embodiment, the initial demetallization artifact model includes a loss function calculation module, the loss function calculation module includes a generator loss function sub-module and a discriminator loss function sub-module, and the loss function calculation module adopts a loss function as follows:

wherein x and z represent a clean data set P of the training data set, respectively _data(x) And artifact dataset P _noise(z) D (x) represents the return value of data x input to the arbiter loss function sub-module D, and G (z) represents the return value of data z input to the generator loss function sub-module G.

In a possible embodiment, when the generator loss function submodule is fixed, the model training is performed on the discriminator loss function submodule, and the loss function of the discriminator loss function submodule is as follows:

when the discriminator loss function submodule is fixed, model training is carried out on the generator loss function submodule, and the loss function of the generator loss function submodule is as follows:

in a second aspect, the present application provides a CBCT metal artifact removal system based on unpaired learning, the system comprising:

a data acquisition unit for acquiring an artifact data set and a clean data set, wherein the artifact data set and the clean data set are unpaired data sets;

the data processing unit is used for carrying out data processing on the artifact data set and the clean data set to obtain a processed training data set;

the model training unit is used for training the demetallization artifact model by utilizing the training data set to obtain an initial demetallization artifact model;

the data reduction unit is used for reducing the predicted artifact removal data output by the initial metal artifact removal model to obtain original data corresponding to the predicted artifact removal data;

the model optimization unit is used for optimizing the initial demetallization artifact model by utilizing the predicted demetallization artifact data and the original data to obtain a target demetallization artifact model;

and the metal artifact removing unit is used for inputting the CT image from which the metal artifact is to be removed into the target metal artifact removing model to obtain a target CT image from which the metal artifact is removed.

According to the CBCT metal artifact removal method and system based on unpaired learning, the artifact data set and the clean data set are unpaired data sets; performing data processing on the artifact data set and the clean data set to obtain a processed training data set; training the demetallization artifact model by using a training data set to obtain an initial demetallization artifact model; restoring the predicted artifact removal data output by the initial artifact removal model to obtain original data corresponding to the predicted artifact removal data; optimizing an initial metal artifact removal model by using the predicted artifact removal data and the original data to obtain a target metal artifact removal model; inputting a CT image from which metal artifacts are to be removed into a target metal artifact removal model to obtain a target CT image from which metal artifacts are removed, so that the end-to-end metal artifact removal model is trained in an unpaired artifact image and a clean image, and the practicability and the value of the method are improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic structural diagram of an electronic device according to a first embodiment of the present application;

FIG. 2 is a flowchart of a method for removing CBCT metal artifacts based on unpaired learning according to a second embodiment of the present application;

FIG. 3 is a schematic diagram of a demetallization artifact model in the CBCT metal artifact removal method based on unpaired learning shown in FIG. 2;

fig. 4 is a schematic functional block diagram of a CBCT metal artifact removal system based on unpaired learning according to a third embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

First embodiment:

fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present application, and in the present application, an electronic device 100 for implementing an example of a CBCT metal artifact removal method based on unpaired learning according to an embodiment of the present application may be described by using the schematic diagram shown in fig. 1.

As shown in fig. 1, an electronic device 100 includes one or more processors 102, one or more storage devices 104, an input device 106, and an output device 108, which are interconnected by a bus system and/or other forms of connection mechanisms (not shown). It should be noted that the components and structures of the electronic device 100 shown in fig. 1 are exemplary only and not limiting, and that the electronic device may have some of the components shown in fig. 1 or may have other components and structures not shown in fig. 1, as desired.

The processor 102 may be a Central Processing Unit (CPU) or other form of processing unit having data processing and/or instruction execution capabilities, and may control other components in the electronic device 100 to perform desired functions.

It should be appreciated that the processor 102 in embodiments of the present application may be a central processing unit (centra lprocess i ng un it, CPU), which may also be other general purpose processors, digital signal processors (d i gita l s igna l processor, DSP), application specific integrated circuits (app l icat ion spec ifi c i ntegrated ci rcu it, AS ICs), off-the-shelf programmable gate arrays (fie l dprogrammab l e gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The storage 104 may include one or more computer program products, which may include various forms of computer-readable storage media.

It should be appreciated that the storage device 104 in embodiments of the present application may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile memory may be a read-on memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (e l ectr i ca l ly EPROM, EEPROM), or a flash memory, among others. The volatile memory may be random access memory (random access memory, RAM) which acts as an external cache. By way of example but not limitation, many forms of random access memory (random access memory, RAM) are available, such as static i c RAM, SRAM, dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced Synchronous Dynamic Random Access Memory (ESDRAM), synchronous i nk DRAM (SLDRAM), and direct memory bus RAM (DR RAM).

Wherein one or more computer program instructions may be stored on the computer readable storage medium, the processor 102 may execute the program instructions to implement client functions and/or other desired functions in embodiments of the present application as described below (implemented by the processor). Various applications and various data, such as various data used and/or generated by the applications, may also be stored in the computer readable storage medium.

The input device 106 may be a device used by a user to input instructions and may include one or more of a keyboard, mouse, microphone, touch screen, and the like.

Second embodiment:

referring to a flowchart of a CBCT metal artifact removal method based on unpaired learning shown in fig. 2, the method specifically includes the following steps:

in step S201, an artifact dataset and a clean dataset are acquired.

Wherein the artifact dataset and the clean dataset are unpaired datasets.

Optionally, the unpaired artifact dataset and the clean dataset each contain CBCT images of multiple sizes, such as 650x650x435, 800x800x500, and so on. The artifact data set generally comprises images such as CBCT imaging noise, bone artifacts, metal artifacts and the like, and the clean data set needs to meet the conditions of low noise, no artifacts or few artifacts, clear details and the like. In the embodiment, various data are used as a training set of the model, so that the practicability and generalization of the model are improved.

That is, the images in the clean dataset are clean images, and the images in the artifact dataset are artifact images.

Step S202, performing data processing on the artifact data set and the clean data set, to obtain a processed training data set.

As one embodiment, step S202 includes: converting each three-dimensional CBCT image in the artifact dataset and the clean dataset into a plurality of two-dimensional image data; fixing each two-dimensional image data to a preset size to obtain a plurality of new two-dimensional image data, wherein each new two-dimensional image data is single-channel data; separating the width and height of each new two-dimensional image data to obtain multi-channel two-dimensional image data; expanding the multi-channel two-dimensional image data by using filling operation to obtain expanded two-dimensional image data; and carrying out standardization processing on the expanded two-dimensional image data by using a k-s sigma transformation formula to obtain a processed training data set.

Optionally, said converting each three-dimensional CBCT image in said artifact dataset and said clean dataset into a plurality of two-dimensional image data comprises: and intercepting the two-dimensional data of each layer of each three-dimensional CBCT image in the artifact dataset and the clean dataset according to the CT value range to obtain a plurality of two-dimensional image data.

Specifically, the fixed non-paired artifact and clean data may be sized to fix the multi-sized data and perform normalization operations for parallelization training of the data. In this embodiment, firstly, unpaired artifact and clean data are obtained respectively, and the two-dimensional data of each layer (i.e. the dimension of the third dimension) can be intercepted according to the CT value range (the CT value range is adjusted to-1000-5500 in consideration of the existence of larger CT value of metal), so as to reduce the influence of the minimum or maximum CT value on the image quality. Further, the size of the input two-dimensional data may be fixed to a preset size according to a preset size (default to 768, 768). Further, the data range can be adjusted to 0-1 using normalization operations for subsequent data normalization operations.

Optionally, said separating the width and height of each of said new two-dimensional image data to obtain multi-channel two-dimensional image data includes: the width and the height of the multi-channel two-dimensional image data are separated into 1/2 of the original width and the height, and four-dimensional data are generated; the four-dimensional data are combined and converted into 4-channel two-dimensional image data.

In this embodiment, first, the width and height of data are separated into 1/2 of the original, and two-dimensional data are converted into four-dimensional data, i.e., (H, W) → (H/2, W/2, 2), where H and W represent the width and height of data, respectively; further, the data may be combined and converted into 4-channel data, i.e., (H/2, W/2, 2) → (H/2, W/2, 4); further, the data dimensions, i.e., (H/2, W/2, 4) → (4, H/2, W/2), may be swapped for subsequent training of the model.

Optionally, the expanding the multi-channel two-dimensional image data by using a filling operation to obtain expanded two-dimensional image data includes: and carrying out data edge filling on the 4-channel two-dimensional image data according to a size filling algorithm to obtain the expanded two-dimensional image data.

In this embodiment, the data is augmented with a 0-padding operation to protect the edge information of the image. In this embodiment, data edge filling is performed according to a size filling formula, and the calculation formula is as follows:

p _h ＝(s-h％s)/2

wherein p is _h Representing the size to be filled in the data width direction, s representing the designated multiple size (default 32), and similarly, the size p to be filled in the data height direction can be calculated _w 。

Optionally, the k-s igma satisfies:

wherein x and f represent the input artifact image and the output artifact image, k and σ, respectively ² And fitting the obtained parameters to the artifact images in the training data set by using a linear equation. Specifically, the present embodiment first calculates the mean value E (x) of each pixel position on a certain artifact image; further, all pixel values with the same mean value are collected to obtain a variance Var (x) of the pixel values; further, in the graph with E (x) as the horizontal axis and Var (x) as the vertical axis, the k and sigma of the artifact image are obtained using linear equation fitting ² 。

Preferably, after normalizing the data, the data range is adjusted to 0-255 by a de-normalization operation for subsequent model training.

And step S203, training the demetallization artifact model by using the training data set to obtain an initial demetallization artifact model.

Wherein, as shown in fig. 3, the initial demetallization artifact model includes: the device comprises a generator module, an image blurring module and a discriminator module; the generator module, the image blurring module and the discriminator module are sequentially connected; the generator module receives the training data set and correspondingly generates an image for predicting the artifact; the image blurring module sharpens the image of the predicted artifact removal and the clean image in the training data set to generate a sharpened image; the discriminator module outputs a predicted probability value according to the sharpened image.

Specifically, a generator module that generates a predicted de-artifacted image from the preprocessed data may include a nonlinear inactivity module, an encoder module, and a decoder module.

Optionally, the encoder module and the decoder module are both composed of a plurality of nonlinear inactive modules, the encoder module and the decoder module are connected, and the plurality of nonlinear inactive modules are connected in sequence, so that the output of the previous module is the input of the next module.

Further, in this embodiment, the nonlinear inactive module normalizes the artifact data and the clean data after the inverse normalization through the layer normalization operation, so as to accelerate convergence of the model and improve performance of the model; then, the number of channels is adjusted to be 2 times of the original number by using a standard convolution of 1x1 and a channel-by-channel convolution of 3x3 so as to reduce the calculated amount of the model; the channel control unit can separate and multiply the channel of the input feature map, change the channel into 1/2 of the original channel, and replace the nonlinear activation function by a simplified pixel-by-pixel multiplication method; the channel attention module can generate weight parameters corresponding to the channel number through self-adaptive average pooling and standard convolution of 1x1, and the weight parameters are multiplied by an input feature graph, so that the learning performance of the feature is improved in a simplified attention calculation mode; then adding the input feature images of the non-linear inactive modules through standard convolution of 1x 1; finally, the depth of the model is deepened by layer normalization, standard convolution of 1x1, channel control unit and standard convolution of 1x 1.

Further, the encoder module may convert the number of input channels (default to 4) to 16 via a standard convolution of 1x 1; further, a four-layer coding structure can be formed according to the nonlinear inactive modules, the channel numbers are 16, 32, 64 and 128 respectively, wherein the first layer and the second layer comprise 1 nonlinear inactive module, the third layer and the fourth layer comprise 2 nonlinear inactive modules, and downsampling is carried out through 3x3 convolution with the step length of 2 after each layer of coding structure. Further, the middle layer consists of 1 nonlinear inactive module, and the number of channels is 256, so that the feature extraction capability is improved.

Further, the decoder module can form a four-layer decoding structure according to the nonlinear inactive modules, the channel numbers are 128, 64, 32 and 16 respectively, each layer only comprises 1 nonlinear inactive module, wherein the up-sampling is carried out on each layer of decoding structure through 3x3 deconvolution with the step length of 2, and the feature map of each layer of decoding structure is added with the coding structure feature map with the corresponding feature size; further, the number of channels is converted to the number of channels input by the encoder module through the standard convolution of 1x1, i.e., the number of output channels of the standard convolution of 1x1 is 4.

In this embodiment, the input of the image blurring module is an image for predicting the artifact removal and a clean image, and the sharpening technique uses 3×3 as a kernel size for calculating the local mean, and further uses the sharpened image and the original image as two sets of input channels of the discriminator module.

In this embodiment, the image blurring module is called twice, and the input is the image for predicting the artifact removal and the clean image respectively, and a corresponding sharpened image is generated; the corresponding input and output images of the image blur module are combined together as input to the arbiter module. Wherein, the original image refers to the input image of the image blurring module called by the discriminator module each time, namely, the original image comprises an image for predicting the artifact removal and a clean image.

Further, the arbiter module may take the combined image of the image blur module as an input to the arbiter to combat learning artifacts and characteristic information of the clean image. In this embodiment, a four-layer convolution structure is used, the convolution kernels are respectively 5x5, and 3x3, the step sizes are respectively 4, and 2, and the channel numbers are respectively 64, 128, 256, and 512. Further, 1 probability value can be output through two full-connection layers and Sigmoid function, the parameters of the first full-connection layer ((3 x3x 512), 1024) and the parameters of the second full-connection layer (1024, 1) can be calculated according to the data size in the training data set (the data width and height after preprocessing are 400 and 400 respectively in the embodiment).

Optionally, the initial demetallization artifact model includes a loss function calculation module, where the loss function calculation module includes a generator loss function sub-module and a discriminator loss function sub-module, and the loss function adopted by the loss function calculation module is as follows:

Further, when the generator loss function submodule is fixed, model training is performed on the discriminator loss function submodule, and the loss function of the discriminator loss function submodule is as follows:

namely, when the input is real clean data, the discriminator loss function submodule needs to return a high score; when the input is artifact data, the arbiter loss function submodule needs to return a low score. That is, the training process of the arbiter loss function submodule is to L _GAN And (5) taking the maximum value.

Further, when the discriminator loss function submodule is fixed, model training is carried out on the generator loss function submodule, and the loss function of the generator loss function submodule is as follows:

namely, when the output of the generator loss function sub-module is the artifact removal prediction graph and the discriminator loss function sub-module outputs the return value, the corresponding label is required to be identical to 1, so that the min is reached _G L _GAN Is effective in (1).

It should be appreciated that E in the above formula represents the desired function.

And step S204, carrying out reduction processing on the predicted artifact removal data output by the initial artifact removal model to obtain original data corresponding to the predicted artifact removal data.

Optionally, the restore process includes de-normalizing the data, removing padding operations, restoring the width and height of the data, restoring the original size of the data, and converting and saving the data types.

Specifically, before denormalizing the data, adjusting the output data range of the model to 0-1 through normalization operation; further, the normalized data is denormalized by using an inverse k-sigma transformation formula so as to recover the subsequent data size. In this embodiment, the inverse k-sigma transform formula is as follows:

more specifically, the data expansion is removed, and the data expansion can be performed according to p calculated in step S202 _h And p _w The 0 fill size of the outer edge of the data after the inverse k-s i gma is removed.

More specifically, restoring the width and height of the data can restore the 4-channel data to single-channel data. In this embodiment, the data dimensions are first swapped, i.e., (4, H/2, W/2) → (H/2, W/2, 4); further, the data can be split and converted into four-dimensional data, namely (H/2, W/2, 4) → (H/2, W/2, 2); further, the width and height of the restored data, four-dimensional data can be converted into two-dimensional data, i.e., (H/2, W/2, 2) → (H, W).

More specifically, the restored data original size may be resized to the data original size using a resizing technique; further, the readjustment data range may be adjusted to 0 to 255 using an inverse normalization operation.

More specifically, the data type is converted and stored, and the readjusted gray scale image can be converted into a CBCT image according to the CT value range (namely, the CT value range is-1000-5500) and stored as D ICOM data.

It should be noted that, because the present application performs the preprocessing operation on the input data, the output data needs to undergo the inverse preprocessing operation to recover the size of the original data, so as to be visualized, so as to facilitate the subsequent comparison of the difference between the learned data and the original data, so that the optimization of the model is completed in step S205.

Step S205, optimizing the initial demetallization artifact model by using the predicted demetallization artifact data and the original data, to obtain a target demetallization artifact model.

Step S206, inputting the CT image from which the metal artifact is to be removed into the target metal artifact removal model to obtain a target CT image from which the metal artifact is removed.

In summary, the application has the following beneficial effects:

(1) Aiming at the problem that artifact and artifact-free paired data are difficult to collect, the application trains an end-to-end metal artifact removal model from an unpaired artifact image and a clean image through a unpaired learning technology, thereby improving the practicability and the value of the method.

(2) Aiming at the problem of high complexity of a model structure and high inference efficiency, the application utilizes a nonlinear non-activated network, reduces the number of model parameters, improves the effect of model artifact removal, and simultaneously achieves the inference efficiency of an average single CBCT image of 54ms, thereby basically meeting the actual application requirements.

(3) Aiming at the problem of high training difficulty of the GAN model, the application provides an image blurring method, so that the GAN model focuses on the characteristic information in the image, and the convergence of the model is accelerated.

Third embodiment:

referring to fig. 4, a CBCT metal artifact removal system based on unpaired learning 500 includes: a data acquisition unit 510, a data processing unit 520, a model training unit 530, a data reduction unit 540, a model optimization unit 550, and a metal artifact removal unit 560. Wherein, the specific functions of each unit are as follows:

a data acquisition unit 510, configured to acquire an artifact data set and a clean data set, where the artifact data set and the clean data set are unpaired data sets;

a data processing unit 520, configured to perform data processing on the artifact data set and the clean data set, to obtain a processed training data set;

the model training unit 530 is configured to train the demetallization artifact model by using the training data set, so as to obtain an initial demetallization artifact model;

the data reduction unit 540 is configured to perform reduction processing on the predicted artifact data output by the initial artifact model, so as to obtain original data corresponding to the predicted artifact data;

a model optimization unit 550, configured to optimize the initial demetallization artifact model by using the predicted demetallization artifact data and the original data, so as to obtain a target demetallization artifact model;

and the metal artifact removing unit 560 is configured to input the CT image from which the metal artifact is to be removed into the target demetallization artifact model, and obtain a target CT image from which the metal artifact is removed.

It should be noted that, the specific functions of the above units are described with reference to the method embodiments, and are not repeated here.

Further, the present embodiment further provides a computer readable storage medium, on which a computer program is stored, where the computer program when executed by a processing device performs the steps of any of the CBCT metal artifact removal methods based on unpaired learning provided in the second embodiment.

The computer program product of the CBCT metal artifact removal method and system based on unpaired learning provided in the embodiments of the present application includes a computer readable storage medium storing program code, where the program code includes instructions for executing the method described in the foregoing method embodiment, and specific implementation may refer to the method embodiment and will not be described herein.

It should be noted that the foregoing embodiments may be implemented in whole or in part by software, hardware (such as a circuit), firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more sets of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state disk.

It should be understood that the term "and/or" is merely an association relationship describing the associated object, and means that three relationships may exist, for example, a and/or B may mean: there are three cases, a alone, a and B together, and B alone, wherein a, B may be singular or plural. In addition, the character "/" herein generally indicates that the associated object is an "or" relationship, but may also indicate an "and/or" relationship, and may be understood by referring to the context.

In the present application, "at least one" means one or more, and "a plurality" means two or more. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b, or c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or plural.

It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The above description is only of the preferred embodiments of the present application and is not intended to limit the present application, but various modifications and variations can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application. It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.

Claims

1. A CBCT metal artifact removal method based on unpaired learning, the method comprising:

2. The method of claim 1, wherein said data processing said artifact dataset and said clean dataset to obtain a processed training dataset, comprising:

and carrying out standardization processing on the expanded two-dimensional image data by using a k-sigma conversion formula to obtain a processed training data set.

3. The method of claim 2, wherein said converting each three-dimensional CBCT image in said artifact dataset and said clean dataset into a plurality of two-dimensional image data comprises:

4. The method of claim 2, wherein said separating the width and height of each of said new two-dimensional image data to obtain multi-channel two-dimensional image data comprises:

5. The method of claim 4, wherein expanding the multi-channel two-dimensional image data using a fill operation to obtain expanded two-dimensional image data comprises:

6. The method of claim 5, wherein the k-sigma satisfies:

wherein the method comprises the steps ofX and f represent the input artifact image and the output artifact image, k and σ, respectively ² And fitting the obtained parameters to the artifact images in the training data set by using a linear equation.

7. The method of any of claims 1-6, wherein the initial demetallization artifact model comprises: the device comprises a generator module, an image blurring module and a discriminator module; the generator module, the image blurring module and the discriminator module are sequentially connected;

8. The method of claim 7, wherein the initial demetallization artifact model comprises a loss function calculation module comprising a generator loss function sub-module and a arbiter loss function sub-module, the loss function calculation module employing a loss function as follows:

9. The method of claim 8, wherein the step of determining the position of the first electrode is performed,

when the generator loss function submodule is fixed, model training is carried out on the discriminator loss function submodule, and the loss function of the discriminator loss function submodule is as follows:

10. a CBCT metal artifact removal system based on unpaired learning, the system comprising: