CN112085829A - Spiral CT image reconstruction method and equipment based on neural network and storage medium - Google Patents

Abstract

The present disclosure provides a neural network-based spiral CT image reconstruction device and method. Wherein, the device includes: a memory for storing instructions and three-dimensional projection data from a helical CT device to an object under inspection, the object under inspection is preset as a multi-layer section; a processor, configured to execute the instructions, so as to : Perform image reconstruction on each layer section respectively. The reconstruction of each layer section includes: inputting the three-dimensional projection data related to the section to be reconstructed into the trained neural network model to obtain a section reconstruction image; according to the reconstruction of the multi-layer section The images form a three-dimensional reconstructed image. By combining the advantages of the deep neural network and the particularity of the helical CT imaging problem, the device of the present disclosure can reconstruct the three-dimensional data projection data into a three-dimensional reconstructed image with more information and less noise.

Description

Reconstruction method, equipment and storage medium of spiral CT image based on neural network

technical field

The present disclosure relates to radiation imaging, in particular to a neural network-based spiral CT image reconstruction method and device, and a storage medium.

Background technique

X-ray CT (Computed-Tomography) imaging systems are widely used in medical, security, industrial non-destructive testing and other fields. The ray source and detector collect a series of projection data according to a certain orbit, and after the restoration of the image reconstruction algorithm, the three-dimensional spatial distribution of the linear attenuation coefficient of the object under the ray energy can be obtained. CT image reconstruction is to restore the linear attenuation coefficient distribution from the projection data collected by the detector, which is the core step of CT imaging. At present, the analytical reconstruction algorithms such as Filtered Back-Projection (FBP), Feldkmap-Davis-Kress (FDK) and iterative reconstruction methods such as Algebra Reconstruction Technique (ART) and Maximum APosterior (MAP) are mainly used in practical applications. .

As people pay more and more attention to the issue of radiation dose, how to obtain conventional quality or higher quality images under the condition of low dose and fast scanning has become a hot research topic in the field. In terms of reconstruction methods, the analytical reconstruction speed is fast, but it is limited to the traditional system architecture, and cannot well solve the problems of missing data and large noise. Compared with the analytical algorithm, the iterative reconstruction algorithm has a wide range of applicable conditions in terms of system architecture, and can achieve better reconstruction results for various problems such as non-standard scanning orbits, low dose and large noise, and missing projection data. However, iterative reconstruction algorithms often require multiple iterations, and the reconstruction takes a long time. It is even more difficult for practical application of 3D helical CT with larger data scale. For the helical CT widely used in medical and industrial fields, increasing the pitch can reduce the scanning time, improve the scanning efficiency, and reduce the radiation dose. However, increasing the pitch means less effective data. The image quality obtained by the conventional analytical reconstruction method is poor; and the iterative reconstruction method is difficult to be practically applied due to its time-consuming.

Deep learning has made significant progress in computer vision, natural language processing, etc. In particular, convolutional neural networks have become popular in image classification, detection, etc. The mainstream network structure of the application. However, there is no related research on the application of neural network to the reconstruction of spiral CT images.

SUMMARY OF THE INVENTION

According to the embodiments of the present disclosure, a spiral CT image reconstruction method and device and a storage medium are provided.

According to an aspect of the present disclosure, a neural network-based spiral CT image reconstruction device is provided, comprising:

a memory for storing instructions and three-dimensional projection data from the helical CT apparatus to the inspected object, the inspected object being preset as a multi-layer section;

a processor configured to execute the instructions to:

Image reconstruction is performed on each layer section respectively, and the reconstruction of each layer section includes: inputting the three-dimensional projection data related to the section to be reconstructed into the trained neural network model to obtain a section reconstruction image;

A three-dimensional reconstructed image is formed from the reconstructed image of the multilayer cross-section.

According to another aspect of the present disclosure, there is provided a spiral CT image reconstruction method, comprising:

The inspected object is preset as a multi-layer section;

Image reconstruction is performed on each layer section respectively, and the reconstruction of each layer section includes: inputting the three-dimensional projection data related to the section to be reconstructed into the trained neural network model to obtain a section reconstruction image;

A three-dimensional reconstructed image is formed from the reconstructed image of the multilayer cross-section.

According to yet another aspect of the present disclosure, there is provided a method for training a neural network, the neural network comprising:

The projection domain sub-network is used to process the input three-dimensional projection data of the spiral CT related to the cross-section to be reconstructed, and obtain the two-dimensional projection data;

The domain conversion sub-network is used to analyze and reconstruct the two-dimensional projection data to obtain the cross-sectional image to be reconstructed;

The sub-network in the image domain is used to process the cross-section image in the image domain to obtain an accurate reconstructed image of the cross-section to be reconstructed;

Wherein, the method includes:

The parameters in the neural network are adjusted by using the consistency cost function of the data model based on the input 3D projection data, the ground truth of the image, and the plane reconstructed image of the set section.

According to yet another aspect of the present disclosure, there is provided a computer-readable storage medium in which computer instructions are stored, and when the instructions are executed by a processor, implement the above-described spiral CT image reconstruction method.

The neural network-based spiral CT image reconstruction device of the present disclosure, by combining the advantages of the deep network and the particularity of the spiral CT imaging problem, provides a device that can reconstruct the three-dimensional projection data into a relatively accurate three-dimensional image;

The present disclosure trains the network through a targeted neural network model architecture, combined with simulation and actual data, so as to reliably, effectively and comprehensively cover all system information and the collection information of the imaged object, accurately reconstruct object images, and suppress low-dose Noise and artifacts caused by missing data;

Although the training process of the neural network model of the present disclosure requires a large amount of data and operations, the actual reconstruction process does not require iteration, and the amount of computation required for reconstruction is comparable to the analytical reconstruction method and much faster than the iterative reconstruction algorithm.

Description of drawings

In order to better understand the embodiments of the present disclosure, the embodiments of the present disclosure will be described in detail according to the following drawings:

FIG. 1 shows a schematic structural diagram of a helical CT system according to an embodiment of the present disclosure;

Fig. 2A is a schematic diagram of the trajectory of the helical motion of the detector relative to the object to be inspected in the helical CT system shown in Fig. 1; Fig. 2B is a schematic diagram of the three-dimensional projection data corresponding to the signal detected by the detector in the helical CT system.

Fig. 3 is the structural representation of control and data processing device in the spiral CT system as shown in Fig. 1;

Fig. 4 shows a schematic diagram of the principle of a device for reconstructing a spiral CT image based on a neural network according to an embodiment of the present disclosure;

FIG. 5 shows a schematic structural diagram of a neural network according to an embodiment of the present disclosure;

6 is a visual network structure diagram of a neural network according to an embodiment of the disclosure;

FIG. 7 shows an exemplary network structure of the projection domain sub-network;

Fig. 8 is a schematic flow chart describing a method for reconstructing a spiral CT image according to an embodiment of the present disclosure.

Detailed ways

Specific embodiments of the present disclosure will be described in detail below. It should be noted that the embodiments described herein are only used for illustration and are not used to limit the embodiments of the present disclosure. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It will be apparent, however, to one of ordinary skill in the art that these specific details need not be employed to practice embodiments of the present disclosure. In other instances, well-known structures, materials, or methods have not been described in detail in order to avoid obscuring the disclosed embodiments.

Throughout this specification, references to "one embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in the present disclosure in at least one embodiment. Thus, appearances of the phrases "in one embodiment," "in an embodiment," "one example," or "an example" in various places throughout the specification are not necessarily all referring to the same embodiment or example. Furthermore, the particular features, structures or characteristics may be combined in any suitable combination and/or subcombination in one or more embodiments or examples. Further, those of ordinary skill in the art should understand that as used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

For the helical CT widely used in medical and industrial fields, increasing the pitch can reduce the scanning time, improve the scanning efficiency, and reduce the radiation dose. However, increasing the pitch means less effective data. The image quality obtained by the conventional analytical reconstruction method is poor; while the iterative reconstruction method is difficult to be practically applied due to the time-consuming.

From the perspective of deep learning, the present disclosure proposes a reconstruction method based on convolutional neural network for helical CT equipment under large-pitch scanning, deeply mines data information, and combines the physical laws of the helical CT system to design a unique network architecture, and training methods to reconstruct higher quality images in less time.

Embodiments of the present disclosure provide a neural network-based spiral CT image reconstruction method and device, and a storage medium. The neural network is used to process the three-dimensional projection data of the inspected object from the helical CT equipment to obtain the volume distribution of the inspected object's linear attenuation coefficient. The neural network may include: a projection domain sub-network, a domain transformation sub-network, and an image domain sub-network. The projection domain subnet processes the input 3D projection data to obtain 2D projection data. The domain transformation sub-network analyzes and reconstructs the two-dimensional projection data, and obtains the cross-sectional image of the image domain setting. The image domain sub-network inputs the cross-sectional image, and through the convolutional neural network including several layers, the features of the data in the image domain are collected, and the image features are further extracted and coupled with each other to obtain an accurate reconstructed image of the set cross-section. Using the solutions of the above embodiments of the present disclosure, it is possible to reconstruct the three-dimensional projection data of the object being inspected by the helical CT device to obtain higher quality results.

FIG. 1 shows a schematic structural diagram of a helical CT system according to an embodiment of the present disclosure. As shown in FIG. 1 , the CT helical system according to the present embodiment includes an X-ray source 20, a mechanical motion device 30, and a detector and a data acquisition system 10, and performs helical CT scanning on an object 60 to be inspected.

The X-ray source 10 can be, for example, an X-ray machine, and an appropriate focus size of the X-ray machine can be selected according to the resolution of imaging. In other embodiments, instead of using an X-ray machine, a linear accelerator or the like may be used to generate the X-ray beam.

The mechanical movement device includes the stage 60 and the frame 30 . The stage can move along the axis direction of the cross section (the direction perpendicular to the paper surface), and the gantry 30 can also rotate, simultaneously driving the detector on the gantry and the X-ray source 10 to rotate synchronously. In this embodiment, the stage, the detector, and the X-ray source are rotated synchronously, so that the detector performs a helical motion relative to the object to be inspected.

The detector and data acquisition system 10 includes an X-ray detector, a data acquisition circuit, and the like. The X-ray detector may use a solid-state detector, and may also use a gas detector or other detectors, and the embodiments of the present disclosure are not limited thereto. The data acquisition circuit includes readout circuit, acquisition trigger circuit and data transmission circuit, etc. The detector usually collects analog signals, which can be converted into digital signals through the data acquisition circuit. In one example, the detectors may be one row of detectors or multiple rows of detectors, and for multiple rows of detectors, different row spacings may be set.

For example, the control and data processing device 60 includes a spiral CT image reconstruction device installed with a control program and a neural network, and is responsible for completing the control of the operation process of the spiral CT system, including mechanical rotation, electrical control, safety interlocking control, etc., and training the neural network. (i.e. a machine learning process), and reconstruct CT images etc. from projection data using a trained neural network.

Fig. 2A is a schematic diagram of the trajectory of the helical motion of the detector relative to the object to be inspected in the CT helical system shown in Fig. 1 . As shown in Figure 2A, the stage can be translated back and forth (that is, in the direction perpendicular to the paper in Figure 1), and the object to be inspected is moved synchronously during this process; The relative motion relationship between the set section corresponding to the image to be reconstructed μ of the inspection object and the detector is that the detector makes a helical motion around the set section.

FIG. 3 shows a schematic structural diagram of the control and data processing device 60 shown in FIG. 1 . As shown in FIG. 3 , the data collected by the detector and the data collection system 10 are stored in the storage device 310 through the interface unit 370 and the bus 380. The read only memory (ROM) 320 stores configuration information and programs of the computer data processor. The random access memory (RAM) 330 is used to temporarily store various data during the operation of the processor 350 . In addition, the storage device 310 also stores computer programs for data processing, such as a program for training a neural network, a program for reconstructing CT images, and the like. The internal bus 380 connects the storage device 310, the read-only memory 320, the random access memory 330, the input device 340, the processor 350, the display device 360 and the interface unit 370 described above. The neural network-based spiral CT image reconstruction device and the control and data processing device 60 in the embodiment of the present disclosure share the storage device 310, the internal bus 380, the read only memory (ROM) 320, the display device 360, the processor 350, etc., for Realize the reconstruction of spiral CT images.

The instruction code of the computer program instructs the processor 350 to execute an algorithm for training a neural network and/or an algorithm for reconstructing a CT image after an operation command entered by the user through the input device 340 such as a keyboard and a mouse. After the reconstruction result is obtained, it is displayed on a display device 360 such as an LCD display, or the processed result is output directly in hard copy such as a print.

According to an embodiment of the present disclosure, the above-mentioned system is used to perform a helical CT scan on the inspected object to obtain the original attenuation signal. Such attenuation signal data is a three-dimensional data, denoted as P, then P is a matrix of size C × R × A, where C represents the number of columns of the detector (the column direction marked in Fig. 2A and Fig. 2B ) , R represents the number of rows of detectors (the row direction as marked in Figure 2A and Figure 2B, corresponding to the rows of multiple rows of detectors), A represents the number of angles of projections collected by the detectors (dimension marked in Figure 2B) ), that is, the spiral CT projection data is organized into a matrix form. The original attenuation signal is preprocessed into 3D projection data (see Figure 2B). For example, the projection data can be obtained by performing preprocessing such as negative logarithmic transformation on the projection data by a helical CT system. Then, the processor 350 in the control device executes the reconstruction program, uses the trained neural network to process the projection data, obtains the two-dimensional projection data of the set section, and further analyzes and reconstructs the two-dimensional projection data to obtain the image domain setting The cross-sectional image can then be further processed by setting the cross-sectional image in the image domain to obtain a plane reconstructed image of the set cross-section. For example, a trained convolutional neural network (such as a U-net type neural network) can be used to process images to obtain feature maps of different scales, and the feature maps of different scales can be merged to obtain the result.

In a specific example, a convolutional neural network may include convolutional layers, pooling, and fully connected layers. The convolutional layers identify the characteristic representation of the input data set, and each convolutional layer operates with a nonlinear activation function. The pooling layer refines the representation of features, and typical operations include average pooling and max pooling. One or more fully-connected layers implement high-order nonlinear signal synthesis operations, and the fully-connected layers also have nonlinear activation functions. Commonly used nonlinear activation functions are Sigmoid, Tanh, ReLU, etc.

During helical CT scanning, increasing the pitch can reduce the scanning time, improve the scanning efficiency, and reduce the radiation dose, resulting in a corresponding reduction in effective data. You can choose to perform interpolation on the helical CT projection data. Regarding the interpolation, that is, the missing data interpolation in the detector row direction, the interpolation methods include but are not limited to linear interpolation and cubic spline interpolation.

Further referring to FIG. 1 , after the X-rays emitted by the X-ray sources 20 at different positions are transmitted through the inspected object 60, they are received by the detector, converted into electrical signals, and then converted into digital signals representing attenuation values, which are preprocessed as Project data for reconstruction by a computer.

Fig. 4 shows a schematic diagram of the principle of a device for reconstructing a spiral CT image based on a neural network according to an embodiment of the present disclosure. As shown in FIG. 4 , in the neural network-based spiral CT image reconstruction device of the embodiment of the present disclosure, by inputting the three-dimensional projection data into the trained neural network model, the reconstructed image of the set section of the object under inspection is obtained. Among them, the neural network model is trained, and the parameters in the network are optimized, and the network learns through the data of the training set, including training and generalization processing, including training and optimizing the parameters in the neural network model through simulated data and/or actual data. ; and performing generalization processing on the optimized parameters by using part of the actual data, and the generalization processing includes fine-tuning the parameters.

FIG. 5 shows a schematic structural diagram of a neural network according to an embodiment of the present disclosure. As shown in FIG. 5 , the neural network in the embodiment of the present disclosure may include three cascaded sub-networks, which are independent neural networks, namely, a projection domain sub-network, a domain transformation sub-network, and an image domain sub-network. FIG. 6 is a visual network structure diagram of a neural network according to an embodiment of the present disclosure. Among them, the types of data before and after processing by sub-networks at all levels can be visually understood. Hereinafter, the three-level sub-network will be specifically described with reference to FIG. 5 and FIG. 6 .

The projection domain sub-network inputs the 3D projection data, which is the data received by the detector in the helical CT system. As the first part of the neural network structure, this sub-network is used to convert the 3D spiral projection to the 2D plane projection. The network takes the spiral projection data related to a certain section of the object to be reconstructed (the set section, which is the section of the image to be reconstructed) as input. In an example, the projection domain sub-network may include several layers of convolutional neural networks, and the spiral projection data After passing through the convolutional neural network, the equivalent two-dimensional fan beam (or parallel beam) projection data of the object is output. This part of the network aims to extract the features of the original spiral CT projection data through the convolutional neural network, and then estimate the fan beam (or parallel beam) projections between the sections that are independent of each other. The main task is to simplify the high-complexity problem of helical CT projection into a two-dimensional in-plane projection, which not only eliminates the influence of cone angle effects, but also simplifies subsequent reconstruction problems. 2D reconstruction requires far less resources and computation than helical CT reconstruction.

FIG. 7 shows an exemplary network structure of the projection domain sub-network. As shown in Figure 7, for the set cross-sectional image to be reconstructed, the projection data related to P is selected from the above-mentioned projection data and rearranged, which is denoted as P' as the input of the projection domain subnet. The specific operations are as follows: Select the axial coordinate of the reconstructed section as the center, cover the data of the front and rear helical scanning angles of 180 degrees, find the number of detector rows corresponding to the reconstructed section under each scanning angle, and rearrange them into C×A' A matrix of size ×R'. Among them, A' represents the angle of the spiral projection that we choose, that is, 360 degrees, and R' represents the maximum number of rows corresponding to the detector at all angles.

用H表示扇束扫描的系统矩阵， 所以有

作为投影域子网络残差。如图8所示，包括但不限于 采用一个U-net型神经网络结构作为投影域子网络，该部分网络以重 排后的投影数据P’作为输入，此部分网络作用是估计该二维截面内 线衰减系数μ的扇束投影p。这部分网络由多个卷积层组成，卷积层 配置K个尺度的2维卷积核。对于某一个尺度的2维卷积核有两个 维度，此处定义第一维度为探测器方向，第二维度为扫描角度方向。 两个维度的卷积核长度不必相同，例如取3*1，3*5，7*3的卷积核。 每个尺度可以设置多个卷积核。所有的卷积核为待定的网络参数。在 网络的池化部分，卷积层之间经过池化，图像尺度逐层减小，在上采 样部分，卷积层之间通过上采样，逐层恢复图像尺度。为保留更多图 像细节信息，池化部分之前的网络输出和上采样部分之后的网络输出 中同等尺度的图像在第三维度方向进行拼接，详见图7。用φ^P-net(P)表 示投影域子网络对应的算子，最后一层卷积层的输出结果使用残差方 式：

若先单独训练投影域子网络，则代价函数可设置 为l范数，以l＝2为例：In addition, the projection data of the linear attenuation coefficient distribution of the set section (plane to be reconstructed) under the condition of fan beam projection is denoted as p, where p is a C×A′-sized matrix. Before training the network, analytical reconstruction methods, including but not limited to PI-original, can be used to reconstruct the cross-sectional image corresponding to the input spiral projection data.

Let H denote the system matrix scanned by the fan beam, so we have

as the projection domain subnet residual. As shown in Figure 8, including but not limited to using a U-net type neural network structure as the projection domain sub-network, this part of the network takes the rearranged projection data P' as input, and the function of this part of the network is to estimate the two-dimensional cross-section Fan beam projection p of inner line attenuation coefficient μ. This part of the network consists of multiple convolutional layers, and the convolutional layers are configured with 2-dimensional convolution kernels of K scales. For a 2-dimensional convolution kernel of a certain scale, there are two dimensions. Here, the first dimension is defined as the detector direction, and the second dimension is the scanning angle direction. The lengths of the convolution kernels in the two dimensions do not have to be the same, for example, take 3*1, 3*5, and 7*3 convolution kernels. Multiple convolution kernels can be set for each scale. All convolution kernels are undetermined network parameters. In the pooling part of the network, after pooling between convolutional layers, the image scale is reduced layer by layer. In the upsampling part, the image scale is restored layer by layer through upsampling between convolutional layers. In order to retain more image details, the images of the same scale in the network output before the pooling part and the network output after the upsampling part are spliced in the third dimension, as shown in Figure 7. The operator corresponding to the projection domain subnet is represented by φ ^P-net (P), and the output of the last convolutional layer uses the residual method:

If the projection domain sub-network is trained separately first, the cost function can be set to the l norm, taking l=2 as an example:

为投影标签。鉴于实际应用无法获 得标签，可以使用小螺距下完备数据的重建结果或领域内的先进迭代 重建方法的重建结果做扇束投影作为投影标签。where k is the index number of the training sample,

is the projection label. In view of the fact that the labels cannot be obtained in practical applications, the reconstruction results of complete data under small pitch or the reconstruction results of advanced iterative reconstruction methods in the field can be used for fan beam projection as projection labels.

Although FIG. 7 exemplifies the projection domain sub-network as a specific structural example of a U-shaped network, those skilled in the art can imagine that the technical solutions of the present disclosure can also be implemented with networks with other structures. In addition, those skilled in the art can also think of using other networks as image domain networks, such as auto-encoder network (Auto-Encoder), fully convolution neural network (Fully convolution neural network), etc., can also realize the technical solutions of the present disclosure .

The domain conversion sub-network inputs the two-dimensional projection data output by the above-mentioned projection domain sub-network, and after analyzing and reconstructing, obtains the image domain setting cross-section image. As the second part of the neural network structure, this sub-network is used for the domain conversion from the projection domain to the image domain. This part of the network realizes the operation from the two-dimensional fan beam (or parallel beam) projection domain data to the image domain cross-sectional image. The weighting coefficients between network nodes (neurons) in the network can be determined by the scan geometry in the relation of 2D fan beam (or parallel beam) CT scans. The input of this layer is the fan beam (or parallel beam) projection data output by the first part, and the output is the preliminary CT reconstruction image (that is, the image domain setting cross-sectional image). Since the sub-network of the first part has transformed the reconstruction problem into two dimensions, the domain transformation network of this part can be directly completed by using the matrix operator of the two-dimensional analytical reconstruction. The operators of this part can also be implemented by a fully connected network. Trained using simulated or real projection data and reconstructed image pairs. The output of this part of the network can be used as the final output result, or it can be output after being processed by the image domain sub-network.

这里W完 成投影域数据的加权，F对应于一个斜坡滤波卷积运算，

完成带 权重的反投影。In an exemplary embodiment, the domain conversion sub-network specifically obtains the image domain output by performing reverse calculation on the above-mentioned p from the projection domain to the image domain. Use Siddon or other methods available in the field to calculate the projection matrix, and the elements of this system matrix correspond to the connection weights of the analytically reconstructed connection layer. Taking FBP fan beam analytical reconstruction as an example,

Here W completes the weighting of the projection domain data, F corresponds to a ramp filter convolution operation,

Complete backprojection with weights.

The image domain sub-network inputs the set cross-section image in the image domain, and after further extraction and fusion of image features, a plane reconstructed image of the set cross-section is formed. The image domain sub-network is the third part. This part of the network takes the image domain set cross-sectional image output by the aforementioned domain conversion sub-network as input, and collects the characteristics of the data in the image domain through the action of the convolutional neural network including several layers. Taking the target image as the learning target, the image features are further extracted and coupled with each other, so as to optimize the image quality in the image domain. The output of this part is the final output of the entire network.

作为输入，作用是实现图像域优 化。类似于第一部分网络，在前半部分，卷积层之间经过池化，图像 尺度逐层减小，在后半部分，卷积层之间通过上采样，逐层恢复图像 尺度。此部分网络可以采用残差训练方式，即最后一层卷积层的输出 结果加上

等于对二维重建图像的估计μ。与投影域子网络类似的， 在图像域子网络包括但不限于选择3×3的卷积核，池化和上采样均 采用2×2的尺寸。选择ReLu作为激活函数。In an illustrative example, the image domain sub-network adopts a U-net type neural network structure similar to the first part, and this part of the network is composed of

As input, the role is to achieve image domain optimization. Similar to the first part of the network, in the first half, the convolutional layers are pooled, and the image scale is reduced layer by layer, and in the second half, the image scale is restored layer by layer through upsampling between the convolutional layers. This part of the network can use the residual training method, that is, the output of the last convolutional layer plus the

is equal to the estimate μ for the 2D reconstructed image. Similar to the projection domain sub-network, the image domain sub-network includes but is not limited to selecting a 3×3 convolution kernel, and both pooling and upsampling use a 2×2 size. Choose ReLu as the activation function.

In the embodiment of the present disclosure, a cost function can be used as the objective function to be optimized, and the cost function of the overall network can be, but not limited to, the l-norm, RRMSE, SSIM, etc., which are commonly used in the field, and the synthesis of multiple cost functions.

f_i，

分别 为第i幅输出图像与第i幅目标图像。

是f_i，

的平均值；

是 f_i，

的方差；

是f_i，

的协方差；c₁，c₂为常数。where f={f ₁ , f ₂ ,..., f _n } is the output image, and the target image is

f _i ,

are the ith output image and the ith target image, respectively.

is f _i ,

average of;

is f _i ,

Variance;

is f _i ,

covariance of ; c ₁ , c ₂ are constants.

In the embodiment of the present disclosure, the neural network parameters can be trained, and the training data includes simulated data and actual data. For the simulated data, the basic mathematical model of the scanned object is established, and the spiral projection data is generated according to the actual system modeling. For example, the simulation data can be lung simulation data, including a total of 30 cases, each case contains 100 slices, a total of 3000 samples, and data augmentation is performed. Augmentation methods include, but are not limited to, rotation, flipping, and the like. For actual data, objects can be scanned on the actual system to obtain helical projection data, which are input to this network after preprocessing to obtain preliminary reconstruction. These preliminary reconstruction results are then subjected to targeted image processing, such as local smoothing of known local smooth regions to obtain label images. Labeled images can also be reconstructed using state-of-the-art iterative reconstruction methods. The network is further trained using the labeled images to achieve fine-tuning of network parameters. In some embodiments, during training, the sub-network that converts the spiral projection to the two-dimensional plane projection can be trained first, and then the overall training is performed; or the overall training can be directly performed. For the training of the projection sub-network and the image domain sub-network, they can be trained separately; the parameters of the domain transformation sub-network can be calculated in advance without post-training, or the parameters of the domain transformation sub-network can be trained.

If the projection domain subnet is trained separately first, the cost function is

为投影标签。鉴于实际应用无法获 得标签，我们使用小螺距下完备数据的重建结果或领域内的先进迭代 重建方法的重建结果做扇束投影作为投影标签。where k is the index number of the training sample,

is the projection label. In view of the fact that labels cannot be obtained in practical applications, we use the reconstruction results of complete data under small pitch or the reconstruction results of advanced iterative reconstruction methods in the field to do fan beam projection as projection labels.

If training image domain subnetworks separately, define the cost function as 2 norm:

where k is the training sample index number and μ ^* is the image label. Since labels cannot be obtained in practical applications, we use the reconstruction results of complete data under small pitch or the reconstruction results of advanced iterative reconstruction methods in the field as labels. Other tags may also be used if other means of obtaining high-quality images are available.

According to an embodiment of the present invention, a direct training method can be adopted. In the direct training process, the convolution kernel weights of the projection domain sub-network and the image domain sub-network are randomly initialized, and the training is performed on the actual collected data set. After the training is completed, another set of actual collected data is used as the test set to verify the network training effect. .

For the actual CT scanning process, the collected data is input into the above training process to obtain a trained network (the network parameters have been machine-learned at this time), and the reconstructed image is obtained.

Fig. 8 is a schematic flow chart describing a method for reconstructing a spiral CT image according to an embodiment of the present disclosure. As shown in Figure 8, in step S10, three-dimensional projection data is input; in step S20, after the neural network model inputs the three-dimensional projection data, a plane reconstruction image of the set section of the inspected object is obtained, wherein the neural network model is trained.

A neural network according to an embodiment of the present disclosure may include a projection domain sub-network, a domain transformation sub-network, and an image domain sub-network. The projection domain sub-network processes the input 3D projection data to obtain 2D projection data. The domain transformation sub-network analyzes and reconstructs the two-dimensional projection data, and obtains the image domain setting cross-section image. The image domain sub-network inputs the cross-sectional image in the image domain, and through the action of the convolutional neural network including several layers, the features of the data in the image domain are extracted, and the image features are further coupled to obtain the plane reconstructed image of the set cross-section. Using the solutions of the above embodiments of the present disclosure, it is possible to reconstruct the three-dimensional projection data of the object being inspected by the helical CT device to obtain higher-quality results.

The machine learning according to the embodiment of the present disclosure may include: training and optimizing parameters in the neural network model by using simulated data and/or actual data; performing generalization processing on the optimized parameters by using part of the actual data, the generalization processing Including fine-tuning of parameters.

The method of the present disclosure can be flexibly applied to different CT scanning modes and system architectures, and can be used in the fields of medical diagnosis, industrial non-destructive testing and security inspection.

The foregoing detailed description has set forth numerous embodiments of methods and apparatus for training neural networks using schematic diagrams, flowcharts, and/or examples. Where such diagrams, flowcharts and/or examples include one or more functions and/or operations, those skilled in the art will understand that each function and/or operation in such diagrams, flowcharts or examples may be Implemented individually and/or collectively by various structures, hardware, software, firmware, or virtually any combination thereof. In one embodiment, portions of the subject matter described in embodiments of the present disclosure may be implemented by application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), or other integrated formats. Those skilled in the art will recognize, however, that some aspects of the embodiments disclosed herein may equally be implemented, in whole or in part, in an integrated circuit, as one or more computers running on one or more computers. Computer programs (eg, implemented as one or more programs running on one or more computer systems), implemented as one or more programs (eg, implemented as one or more programs) running on one or more processors One or more programs running on multiple microprocessors), implemented as firmware, or substantially implemented as any combination of the above, and those skilled in the art will have the ability to design circuits and/or write software and and/or firmware code capability. Furthermore, those skilled in the art will recognize that the mechanisms of the subject matter of this disclosure can be distributed as program products in many forms, and that regardless of the specific type of signal bearing medium actually used to perform the distribution, the mechanisms of the subject matter of this disclosure can be distributed as program products in many forms Exemplary embodiments apply. Examples of signal bearing media include, but are not limited to: recordable-type media, such as floppy disks, hard drives, compact disks (CDs), digital versatile disks (DVDs), digital magnetic tapes, computer memory, etc.; and transmission-type media, such as digital and /or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

Although embodiments of the present disclosure have been described with reference to several exemplary embodiments, it is to be understood that the terms used are of description and illustration, and not of limitation. Since the disclosed embodiments can be embodied in various forms without departing from the spirit or essence of the disclosed embodiments, it should be understood that the above-described embodiments are not limited to any of the foregoing details, but should be within the spirit and scope defined by the appended claims be interpreted broadly within the scope of the claims, and therefore all changes and modifications that come within the scope of the claims or their equivalents are intended to be covered by the appended claims.

Claims (18)

1. A neural network-based spiral CT image reconstruction device, comprising: a memory for storing instructions and three-dimensional projection data from the helical CT apparatus to the inspected object, the inspected object being preset as a multi-layer section; a processor configured to execute the instructions to: Image reconstruction is performed on each layer section respectively, and the reconstruction of each layer section includes: inputting the three-dimensional projection data related to the section to be reconstructed into the trained neural network model to obtain a section reconstruction image; A three-dimensional reconstructed image is formed from the reconstructed image of the multilayer cross-section. 2. The neural network-based spiral CT image reconstruction device according to claim 1, wherein the neural network model comprises: The projection domain sub-network is used to process the input 3D projection data related to the section to obtain 2D projection data. 3. The neural network-based spiral CT image reconstruction device according to claim 2, wherein the neural network model further comprises: The domain conversion sub-network is used for analyzing and reconstructing the two-dimensional projection data to obtain a cross-sectional image in the image domain. 4. The neural network-based spiral CT image reconstruction device according to claim 3, wherein the neural network model further comprises: The image domain sub-network is used to input the cross-sectional image of the image domain. After the convolutional neural network including several layers, the features of the data in the image domain are extracted and the image features are further coupled to finally obtain a cross-sectional reconstructed image. 5. The neural network-based spiral CT image reconstruction device according to claim 1, wherein the three-dimensional projection data related to the cross-section to be reconstructed input by the neural network model is selected from all projection data of the spiral CT device and is related to the to-be-reconstructed. Section-correlated and rearranged projection data. 6 . The neural network-based spiral CT image reconstruction device according to claim 1 , wherein the three-dimensional projection data of the inspected object stored in the memory is the data after interpolation and preprocessing of all projection data of the spiral CT device. 7 . 7. The device for reconstructing a spiral CT image based on a neural network according to claim 1, wherein the learning data used in the training comprises: simulated data and/or actual data, and the simulated data includes numerical simulation of the scanned object The data obtained by the helical projection and the true value of the image of the scanned object are used as labels; the actual data includes the spiral projection data obtained by the helical CT equipment scanning the object and the label image obtained by iterative reconstruction according to the projection data; or known materials The projection data is obtained by helical scanning with the actual phantom of the structure, and the label image is formed with the known material and structure information. 8. The neural network-based spiral CT image reconstruction device according to claim 2, wherein the projection domain sub-network is a convolutional neural network structure, and the partial network takes three-dimensional projection data as input, and the partial network is used for estimation Sets the fan-beam and/or parallel-beam projection of the line attenuation coefficients within the section as the output of the projection domain subnet. 9. The neural network-based spiral CT image reconstruction device according to claim 4, wherein the image domain sub-network is a convolutional neural network structure, and the partial network takes the output of the domain conversion sub-network as an input, and outputs an optimized Rebuild the image. 10. A spiral CT image reconstruction method, comprising: The inspected object is preset as a multi-layer section; Image reconstruction is performed on each layer section respectively, and the reconstruction of each layer section includes: inputting the three-dimensional projection data related to the section to be reconstructed into the trained neural network model to obtain a section reconstruction image; A three-dimensional reconstructed image is formed from the reconstructed image of the multilayer cross-section. 11. The method of claim 10, wherein the neural network model comprises: The projection domain sub-network is used to process the 3D projection data related to the section to be reconstructed to obtain the 2D projection data; The domain conversion sub-network is used to analyze and reconstruct the two-dimensional projection data to obtain the image domain cross-section image; The image domain sub-network is used to input the cross-sectional image of the image domain. After the convolutional neural network including several layers, the features of the data in the image domain are extracted, and the image features are further optimized to obtain an accurate reconstructed image of the cross-section to be reconstructed. . 12. The method of claim 10, further comprising: The attenuation signal data is obtained by the CT scanning equipment, and the three-dimensional projection data is obtained after processing the attenuation signal data; The projection data related to the section to be reconstructed and rearranged are selected from all the three-dimensional projection data as the input of the neural network model. 13. The method of claim 11, wherein the training comprises: Train and optimize the parameters in the neural network model through simulated data and/or actual data; The optimized parameters are generalized by using part of the actual data, and the generalization includes fine-tuning the parameters. 14. The method of claim 13, wherein the training comprises: The projection sub-network and the image domain sub-network are separately trained or the neural network is trained as a whole; or the parameters of the domain transformation sub-network are calculated in advance, or the parameters of the domain transformation sub-network are trained. 15. The method according to claim 10, wherein the three-dimensional projection data is data preprocessed by interpolation from the overall projection data of the helical CT apparatus. 16. The method according to claim 11, wherein the projection domain sub-network is a convolutional neural network structure, the part of the network takes three-dimensional projection data as input, and the function of this part of the network is to estimate the sector for setting the attenuation coefficient of the inner line of the section. beam and/or parallel beam projection, the fan beam and/or parallel beam projection is used as the output of the projection domain sub-network; the image domain sub-network is a convolutional neural network structure, and this part of the network takes the image domain setting cross-sectional image as input , in the first half of the network structure, the convolutional layers are pooled, and the image scale is reduced layer by layer. In the second half, the image scale is restored layer by layer through upsampling between the convolutional layers. 17. A method for training a neural network comprising: The projection domain sub-network is used to process the input three-dimensional projection data of the spiral CT related to the cross-section to be reconstructed, and obtain the two-dimensional projection data; The domain conversion sub-network is used to analyze and reconstruct the two-dimensional projection data to obtain the cross-sectional image to be reconstructed; The image domain sub-network is used to process the cross-section image in the image domain to obtain an accurate reconstructed image of the cross-section to be reconstructed; Wherein, the method includes: The parameters in the neural network are adjusted by using the consistency cost function of the data model based on the input 3D projection data, the ground truth of the image, and the plane reconstructed image of the set section. 18. A computer-readable storage medium having stored therein computer instructions which, when executed by a processor, implement the method of any of claims 10-16.

Images