CN107154064B - Natural image compressed sensing method for reconstructing based on depth sparse coding - Google Patents

Abstract

The invention discloses a kind of natural image compressed sensing method for reconstructing based on depth sparse coding mainly solves the problems, such as that existing method is difficult to fast, accurately with coefficient reconstruction natural image.Its implementation is: 1) to image block and in orthogonal transform domain up conversion, the observation vector of calculation of transform coefficients；2) the recovery transformation coefficient of observation vector is found out with iteration method, and updates calculating parameter；3) calculate 2) in transformation coefficient observation vector, find out its with 1) in observation vector residual error amount；4) step 1) -3 is repeated) obtain trained model, and preservation model parameter；5) test observation data and model parameter are input to trained model, obtain image transform coefficients corresponding with test observation data；6) inverse transformation, the natural image finally rebuild are carried out to the transformation coefficient in 5).The natural image that the present invention is rebuild is clear, and reconstructed velocity is quickly, can be used for the recovery to natural image.

Description

Natural image compressed sensing reconstruction method based on depth sparse coding

Technical Field

The invention belongs to the field of image processing, and particularly relates to a natural image compressed sensing reconstruction method which can be used for sampled natural image restoration.

Background

With the development of media technology, massive image data faces huge challenges in real-time transmission and storage. The proposal of the compressed sensing technology opens up a new idea in theory for solving the problems effectively. The compressive sensing theory considers that if a signal has sparsity under a certain transformation base, random projection observation can be carried out on the signal, the signal is accurately reconstructed through priori information of the signal by using fewer observed values, and a model of the signal is a norm optimization problem under the constraint of observation data fidelity.

For the above compressed sensing model, different norm constraints represent different reconstruction algorithms and reconstruction performance. If norm is used, the orthogonal matching pursuit OMP algorithm is usually used for reconstruction. Although the norm satisfies the original idea of a compressed sensing model, namely finding a solution of an optimization problem with the minimum sparsity, the solution is often not accurate because it is an NP-Hard problem. Therefore, the scholars propose to replace the norm by the norm to make the norm become a convex optimization problem, and propose a series of reconstruction algorithms based on iteration: an iterative threshold shrinkage algorithm ISTA, a fast iterative threshold shrinkage algorithm FISTA, an approximate information transfer algorithm AMP, and the like. Although the problems are further solved by the proposed algorithms, the algorithms are all solution algorithms based on an optimization theory, and still have the problems of limited reconstruction precision, complex iteration process, low convergence speed and the like.

With the development of deep learning technology in recent years, researchers have proposed a compressed sensing reconstruction method Recon-Net based on a convolutional neural network, which solves the problem of complicated iteration process, but has two problems: 1) the reconstructed image obtained by directly applying the model has larger noise, and the reconstructed image with better quality can be obtained only by denoising; 2) the convergence rate of the model is slow during training, and even on high performance computers, the time of day is required for training the model.

Disclosure of Invention

The invention aims to provide a natural image compressed sensing reconstruction method based on depth sparse coding aiming at the problems of the traditional reconstruction method based on optimization and the current reconstruction method based on depth learning, so as to simplify the complexity of a model, reduce the training time of the model and improve the reconstruction speed and the reconstruction effect of an image.

The technical scheme of the invention is as follows: the compressed sensing coding is carried out on the natural image by utilizing the sparse prior information on the transform domain, and the recovery and reconstruction of the image coding information are realized by combining the learning-based approximate information transfer algorithm LAMP and the recurrent neural network RNN model, wherein the realization steps comprise the following steps:

(1) model training:

(1a) inputting a plurality of pictures, and taking n training image blocks X from the pictures;

(1b) carrying out compressed sensing observation on n training image blocks X in the step (1a) to obtain n observation coefficients Y, and forming n training sample pairs by using the training image blocks and the corresponding observation coefficients: { (X ═ X)₁,x₂,...,x_n-1,x_n),(Y＝y₁,y₂,...,y_n-1,y_n)}；

(1c) Setting the model training times K to be 100, and randomly selecting r training samples y from Y, X each time_r、x_rTraining by adopting a gradient descent method, wherein a cutoff condition flag of each training is that model error values are not attenuated after 50 times of iteration;

(1d) setting parameters of a sparse coding algorithm, initializing the iteration number T to 10, and enabling an initial transformation systemResidual v of observed data^tY, whereinTransform coefficients, v, for the t-th iteratively calculated training image block^tIs the observed residual error of the t iteration;

(1e) calculating the transformation coefficient of the training image block of the t +1 th iteration:whereinTransform coefficients for the t-th iteration, A^TC^tv^tTransform coefficients for the observed residual of the t-th iteration, A^TIs the transpose of the observation matrix A, C^tIs the parameter matrix to be optimized for the t-th time,threshold for the t-th iteration, α^tIs the scalar parameter value of the t-th update, M is the dimension of the observed data yDegree, | | v^t||₂Denotes v^tη_st(. is a threshold contraction function;

(1f) the observed residual of the t +1 th iteration is calculated:wherein, b^t+1v^tIs the Onsagercorrection item;

(1g) circularly executing T times (1e) -1 f) to obtain transformation coefficient

(1h) The transform coefficient obtained when (1g)When the iteration cutoff condition flag is met, the model parameters of the training are stored;

(1i) performing the model training for K times (1d) - (1h) in a circulating manner to finish the model training;

(2) the testing steps are as follows:

(2a) will observe the data y_testAnd T parameter matrixes obtained by model trainingAnd scalar parameter valuesInputting the data into a trained model to obtain and input observation data y_testCorresponding test image transform coefficients

(2b) For the transformation coefficientCarrying out PCA inverse transformation to obtain and observe data y_testCorresponding toImage Img:therein, Ψ^-1Representing a PCA inverse transformation matrix;

(2c) Ψ existing according to the orthogonality of the PCA transform^-1＝ΨΨ^TRelation to observed data y_testThe corresponding image Img is rewritten as:finish to the observation data y_testIn which Ψ is^TRepresenting the transpose of the PCA forward transform matrix Ψ.

Compared with other prior art, the invention has the following advantages:

the invention introduces sparse prior information of a natural image, combines the advantages of a deep neural network and sparse coding, and reduces the complexity of a model, thereby shortening the reconstruction time of the image, realizing the rapid reconstruction of compressed sensing and improving the reconstruction effect of the natural image;

secondly, the model training speed is improved by adopting a model training method of transfer learning, so that the model can be trained quickly.

Drawings

FIG. 1 is a general flow chart of an implementation of the present invention;

FIG. 2 is a model training sub-flow diagram of the present invention;

FIG. 3 is a sub-flowchart of image reconstruction in the present invention;

FIG. 4 is an original natural image of Barbara used in the simulation experiment of the present invention;

fig. 5 is a graph showing the effect of the conventional TVAL3 method on the reconstruction of a Barbara image at a compression ratio of 0.25;

FIG. 6 is a graph showing the effect of reconstruction of a Barbara image at a compression ratio of 0.25 using a conventional Recon-Net method;

fig. 7 is a graph showing the effect of the present invention on the reconstruction of a Barbara image at a compression ratio of 0.25.

The specific implementation mode is as follows:

the embodiments and effects of the present invention are described in detail below with reference to the accompanying drawings:

referring to fig. 1, the natural image reconstruction method based on depth sparse coding of the present invention includes two parts, namely model training and testing. Firstly inputting n training image blocks X and constructing a training sample pair, then carrying out model training to obtain a trained model, and inputting test observation data into the trained model to carry out testing to obtain a reconstructed natural image.

The following describes the two parts of model training and testing of the present invention in detail:

first, model training part

Referring to fig. 2, the implementation steps of this section are as follows:

step 1: inputting n training image blocks X to obtain training sample pairs,

(1a) inputting n training image blocks X, and for each input training image block data X_iCarrying out Principal Component Analysis (PCA) transformation to obtain transformation coefficientWherein Ψ represents a PCA forward transformation matrix of principal component analysis, and Ψ is obtained according to the orthogonality of the PCA transformation of the principal component analysis^-1＝ΨΨ^TIs obtained according to the relationTherein, Ψ^-1And Ψ^TSeparately representing principal component analysis PCA inversionTransposing the transform matrix and the positive transform matrix Ψ;

(1b) observing each training image block according to a compressed sensing observation model to obtain observation data y_i：Wherein, A ═ phi Ψ^TPhi is an undersampled Gaussian random observation matrix, and the vector w is Gaussian white noise with a zero mean value;

(1c) n training image blocks X and corresponding n observation coefficients Y form n training sample pairs: { (X ═ X)₁,x₂,...,x_n-1,x_n),(Y＝y₁,y₂,...,y_n-1,y_n)}。

Step 2: setting the training times of the model, randomly selecting r training sample pairs,

(2a) setting the model training times K to be 100;

(2b) at each training time, r training sample pairs x are randomly selected from Y, X_r,y_r}。

And step 3: setting parameters of a sparse coding algorithm, and initializing relevant variables in the algorithm.

(3a) Initializing the iteration time T as 10;

(3b) let the initial transform coefficientResidual v of observed data^tY, whereinImage block transform coefficients, v, calculated for the t-th iteration^tIs the observed residual for the t-th iteration.

And 4, step 4: calculating transformation coefficient of training image block of t +1 th iteration

(4a) Calculating the observed residual v of the t-th iteration^tSparse coefficient A of^TC^tv^tWherein A is^TIs the transpose of the observation matrix A, C^tIs a parameter matrix to be optimized for the t time and is initialized to be an identity matrix;

(4b) calculating sparse coefficients of training image blocks of the (t + 1) th iteration,wherein,for the transform coefficients of the t-th iteration,threshold for the t-th iteration, α^tIs the scalar parameter value of the t-th update, M is the dimension of the observed data y, | | v^t||₂Denotes v^tA second norm of (d);

(4c) calculating the transformation coefficient of the training image block of the (t + 1) th iteration,

wherein, η_st(. is) a threshold contraction function for contracting the threshold λ^tAnd the sparse coefficient mu^t+1Making comparison by making the ratio less than a threshold value lambda^tCoefficient of sparsity mu^t+1Set to zero and will be greater than a threshold lambda^tCoefficient of sparsity mu^t+1Is set to mu^t+1And a threshold lambda^tThe absolute value of the difference.

And 5: calculating the observed residual v of the t +1 th iteration^t+1。

(5a) Firstly, the product obtained in the step 4Setting the coefficient larger than zero as 1, and then averaging the coefficients according to the columns to obtainZero norm ofRecalculating the weight b^t+1：

Where N is a transform coefficientM is the observation residual v^tDimension (d);

(5b) weight b^t+1And the observed residual v^tPerforming dot multiplication to obtain a matrix b of the Onsager correction item^t+1v^t；

(5c) Subjecting the matrix b obtained in (5b) to^t+1v^tSubstitution formulaCalculating the observed residual v of the t +1 th iteration^t+1Wherein y is the observation data of the training image block, and A is the observation matrix.

Step 6: and (4) circularly executing the steps from the step 4 to the step 5 for 10 times to obtain the transformation coefficient

And 7: and saving the model parameters of the training.

The transform coefficient obtained in step 6Satisfy iteration cutoffAnd when the flag is set, storing the model parameters of the training, wherein the cutoff condition flag of each training is that the model error value is not attenuated after iterating for 50 times.

And 8: and (5) circularly executing the step 3 to the step 7 for K times to finish model training.

Second, test part

Referring to fig. 3, the implementation steps of this section are as follows:

and step 9: inputting the test observation data and the parameters stored by the model training part to obtain input observation data y_testCorresponding image transform coefficient

Taking out T matrixes C stored in the model training part from the model training part^tAnd scalar α^tAre respectively marked asAndand the two parameters and the test observation data y collected in the real scene_testAs input, inputting into trained model, outputting and observing data y_testCorresponding image transform coefficient

Step 10: obtaining observation data y by Principal Component Analysis (PCA) inverse transformation_testCorresponding natural image Img.

(10a) For the transformation coefficientCarrying out Principal Component Analysis (PCA) inverse transformation to obtain and observe data y_testCorresponding natural images:therein, Ψ^-1Representing a Principal Component Analysis (PCA) inverse transformation matrix;

(10b) Ψ) transformed from Principal Component Analysis (PCA)^-1＝ΨΨ^TRelation to the observed data y_testThe corresponding natural image Img is rewritten as:finish to the observation data y_testIn which Ψ is^TRepresenting the transpose of the principal component analysis PCA forward transform matrix Ψ.

The effect of the invention can be specifically illustrated by the following simulation experiment:

1. simulation conditions are as follows:

1) the programming platform used for the simulation experiment is Pycharm v 2016;

2) the natural image data used by the simulation experiment come from standard training and testing data sets;

3) the size of a training image block used in the simulation experiment is 25 multiplied by 25, and the number n of training samples is 52650;

4) in the simulation experiment, a peak signal-to-noise ratio (PSNR) index is adopted to evaluate a compressed sensing experiment result, wherein the PSNR index is defined as:

wherein, MAX_iAnd MSE_iN is the number of pixels for the maximum pixel value and the mean square error of the reconstructed high-resolution natural image Img.

2. Simulation content:

simulation 1, reconstructing the natural image Barbara at a compression rate of 0.25 by using the TVAL3 method, and the reconstruction result is shown in fig. 5.

Simulation 2, by using the Recon-Net method, reconstructs the natural image Barbara at a compression rate of 0.25, and the reconstruction result is shown in fig. 6.

Simulation 3, adopting the method of the invention to reconstruct the natural image Barbara when the compression rate is 0.25, and the reconstruction result is shown in FIG. 7.

As can be seen from the results of the natural image Barbara reconstruction shown in FIGS. 5-7, the image reconstructed by the method is clearer, the image edge is sharper and the visual effect is better than the images reconstructed by other methods.

The peak signal-to-noise ratio PSNR obtained by respectively carrying out reconstruction simulation on a natural image Barbara by using the conventional TVAL3 method, NLR-CS method, Recon-Net method and the method disclosed by the invention is shown in Table 1.

TABLE 1 PSNR values for different reconstruction methods

As can be seen from Table 1, the peak signal-to-noise ratio PSNR of the present invention is 3.73dB higher than that of the existing TVAL3 method when the compression ratio is 0.25, and 2.00dB higher than that of the existing Recon-Net.

Claims (1)

1. The natural image reconstruction method based on the depth sparse coding comprises the following steps:

(1) model training:

(1a) inputting a plurality of pictures, and taking n training image blocks X from the pictures;

(1b) carrying out compressed sensing observation on n training image blocks X in the step (1a) to obtain n observation coefficients Y, and forming n training sample pairs by using the training image blocks and the corresponding observation coefficients:

{(X＝x₁,x₂,...,x_n-1,x_n),(Y＝y₁,y₂,...,y_n-1,y_n)}，

the compressed sensing observation of the n training image blocks X in (1a) is implemented as follows:

(1b1) for each image block data x_iCarrying out PCA transformation to obtain a transformation coefficient:wherein, Ψ and Ψ^-1Respectively representing the PCA forward transform and the inverse transform thereof, and satisfying Ψ due to the orthogonality of the PCA transform^-1＝ΨΨ^TThen, there is,Ψ^Ta transpose representing the PCA positive transformation matrix Ψ;

(1b2) observing each image block according to a compressed sensing observation model to obtain observation data:wherein A ═ phi Ψ^TThe matrix phi is an undersampled Gaussian random observation matrix, and the vector w is Gaussian white noise with zero mean;

(1c) setting the model training times K to be 100, and randomly selecting r training samples y from Y, X each time_r、x_rTraining by adopting a gradient descent method, wherein a cutoff condition flag of each training is that model error values are not attenuated after 50 times of iteration;

(1d) setting parameters of a sparse coding algorithm, initializing the iteration number T to 10, and enabling an initial transformation systemResidual v of observed data^tY, whereinTransform coefficients, v, for the t-th iteratively calculated training image block^tIs the observed residual error of the t iteration;

(1e) calculate t +1 th iterationTraining image block transform coefficients of (1):whereinTransform coefficients for the t-th iteration, A^TC^tv^tTransform coefficients for the observed residual of the t-th iteration, A^TIs the transpose of the observation matrix A, C^tIs the parameter matrix to be optimized for the t-th time,threshold for the t-th iteration, α^tIs the scalar parameter value of the t-th update, M is the dimension of the observed data y, | | v^t||₂Denotes v^tη_st(. is a threshold contraction function;

(1f) the observed residual of the t +1 th iteration is calculated:wherein, b^t+1v^tIs the Onsagercorrection term:

(1f1) firstly, the method is carried outSetting the coefficient larger than zero as 1, and then averaging the coefficients according to the columns to obtainZero norm ofFinally, the weight b is calculated^t+1：

Where N is a transform coefficientM is the observation residual v^tDimension (d);

(1f2) weight b^t+1And the observed residual v^tPerforming dot multiplication to obtain a matrix b of the Onsager correction item^t+1v^t；

(1g) Circularly executing T times (1e) -1 f) to obtain transformation coefficient

(1h) The transform coefficient obtained when (1g)When the iteration cutoff condition flag is met, the model parameters of the training are stored;

(1i) performing the model training for K times (1d) - (1h) in a circulating manner to finish the model training;

(2) the testing steps are as follows:

(2a) will observe the data y_testAnd T parameter matrixes obtained by model trainingAnd scalar parameter valuesInputting the data into a trained model to obtain and input observation data y_testCorresponding test image transform coefficients

(2b) For the transformation coefficientCarrying out PCA inverse transformation to obtain and observe data y_testCorresponding image Img:therein, Ψ^-1Representing a PCA inverse transformation matrix;

(2c) Ψ existing according to the orthogonality of the PCA transform^-1＝ΨΨ^TRelation to observed data y_testThe corresponding image Img is rewritten as:finish to the observation data y_testIn which Ψ is^TRepresenting the transpose of the PCA forward transform matrix Ψ.