CN110097116A - A kind of virtual sample generation method based on independent component analysis and Density Estimator - Google Patents

Abstract

The invention discloses a virtual sample generation method based on independent component analysis and kernel density estimation. In the initial stage of system operation, when the number of training samples is insufficient, the invention uses the kernel density estimation method to estimate the probability density function of a small number of samples The probability density function of the sample as a whole, when there is a correlation between the attributes of the original sample, the independent component analysis method is used to remove the correlation between the attributes of the original sample, and then the kernel density is estimated, which is generated according to the estimated probability density function dummy samples. The invention can alleviate the problem of insufficient training samples when training the machine learning model, and improve the accuracy of the machine learning model. Compared with other virtual sample generation methods, the present invention introduces an independent component analysis method to solve the problem of correlation among various attributes of the sample, thus broadening the application of the present invention.

Description

A Virtual Sample Generation Method Based on Independent Component Analysis and Kernel Density Estimation

technical field

The invention belongs to the field of computers, in particular to a virtual sample generation method based on independent component analysis and kernel density estimation.

Background technique

At present, machine learning methods are increasingly used in various fields. For problems that cannot be solved by classical statistics, people hope that machine learning can be used to solve them. The sample size has a great influence on the accuracy of machine learning methods. However, in many cases, due to the limitation of sampling time and cost, there is often the problem of insufficient number of samples.

The virtual sample generation technology was first proposed by Niyogi et al. Wang Xu and others divided virtual sample generation methods into three categories, based on prior knowledge, based on disturbance and based on the distribution function of the research field. The virtual sample generation technology is applied to the construction process of the energy prediction model, and the virtual sample generation technology has significantly improved the accuracy of the energy prediction model. Lee et al. used latent information functions to generate virtual samples to improve the performance of demand forecasting models based on neural networks. Arora et al. generated virtual samples through empirical formulas, and successfully constructed an artificial neural network-based calculation model using the data set with virtual samples to estimate the heating rate of the battery.

The existing virtual sample generation methods are mainly aimed at samples whose attributes are independent of each other, without considering the correlation between sample attributes.

Contents of the invention

The purpose of the present invention is to overcome the above shortcomings, provide a virtual sample generation method based on independent component analysis and kernel density estimation with wider application and simpler operation, and improve the accuracy of machine learning models.

In order to achieve the above object, the present invention comprises the following steps:

Step 1: Carry out independent component analysis on the original sample data, remove the correlation between attributes, and judge whether the analysis structure is convergent;

Step 2, if convergent, estimate the probability density function by multi-kernel density estimation method for independent samples, and sample; if not convergent, estimate the probability density function by multi-kernel density estimation method for the original sample, and sample;

Step 3, using the results of the independent component analysis in step 1 to restore the correlation of the convergently sampled data in step 2, so that the data after convergent sampling are mapped back to the original sample space to obtain virtual samples;

Step 4, mixing the virtual samples with the original samples to obtain the final expanded sample set.

When there is a correlation between the various attributes of the sample data, the results obtained by the independent component analysis are as follows:

Assume that a small number of samples are collected as,

x=(x ₁ ,x ₂ ,…,x _n ), x∈R ⁿ

Assuming that x is obtained by linear transformation of n independent random variables s, then,

s=(s ₁ ,s ₂ ,…,s _n ), s∈R ⁿ

Assuming that A is a mixing matrix, there are,

x ⁽ⁱ⁾ = As ⁽ⁱ⁾ , i = (1,2,...,m), A is a constant;

Among them, x is the sample that has been collected, and s is the independent random variable obtained after independent component analysis.

The method of estimating the probability density function using the multi-kernel density estimation method is as follows:

The mathematical expression for kernel density estimation is,

Solve the smooth coefficient h according to the mean square integral error function, where f(s) is the true probability density function of s, is an estimate of f(s);

After solving the smooth coefficient h, the estimation of the probability density function of s is completed.

Using the Gaussian function as the kernel function, the expression of the Gaussian function is,

In step 2, the sampling method is as follows:

s _v ＝s _i +hs _r ，where 1≤i≤n,s _r ～N(0,1);

Among them, s _v is the sampling value.

The original sample data is the data with insufficient training samples.

Compared with the prior art, in the initial stage of system operation, when the number of training samples is insufficient, the present invention uses the method of kernel density estimation to estimate the probability density function of the whole sample through the probability density function of a small number of samples, when the attributes of the original sample When there is a correlation among them, the method of independent component analysis is used to remove the correlation between the attributes of the original sample, and then the kernel density is estimated, and the virtual sample is generated according to the estimated probability density function. The invention can alleviate the problem of insufficient training samples when training the machine learning model, and improve the accuracy of the machine learning model. Compared with other virtual sample generation methods, the present invention introduces an independent component analysis method to solve the problem of correlation among various attributes of the sample, thus broadening the application of the present invention.

Description of drawings

Fig. 1 is a flowchart of the present invention;

Fig. 2 is a schematic diagram of an embodiment of the present invention.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings.

Referring to Fig. 1, the present invention comprises the following steps:

Step 1: Perform independent component analysis on the original sample data with insufficient training samples, remove the correlation between attributes, and judge whether the analysis structure is convergent;

Step 2, if convergent, estimate the probability density function by multi-kernel density estimation method for independent samples, and sample; if not convergent, estimate the probability density function by multi-kernel density estimation method for the original sample, and sample;

Step 3, using the results of the independent component analysis in step 1 to restore the correlation of the convergently sampled data in step 2, so that the data after convergent sampling are mapped back to the original sample space to obtain virtual samples;

Step 4: Mix the virtual sample with the data after non-convergent sampling to obtain the final virtual sample.

When there is a correlation between the various attributes of the sample data, the results obtained by the independent component analysis are as follows:

Assume that a small number of samples are collected as,

x=(x ₁ ,x ₂ ,…,x _n ), x∈R ⁿ

Assuming that x is obtained by linear transformation of n independent random variables s, then,

s=(s ₁ ,s ₂ ,…,s _n ), s∈R ⁿ

Assuming that A is a mixing matrix, there are,

x ⁽ⁱ⁾ = As ⁽ⁱ⁾ , i = (1,2,...,m);

Among them, x is the sample that has been collected, s is the independent random variable obtained after independent component analysis, and A is a constant.

The method of estimating the probability density function using the multi-kernel density estimation method is as follows:

The mathematical expression for kernel density estimation is,

Solve the smooth coefficient h according to the mean square integral error function, where f(s) is the true probability density function of s, is an estimate of f(s);

After solving the smooth coefficient h, the estimation of the probability density function of s is completed.

Using the Gaussian function as the kernel function, the expression of the Gaussian function is,

The sampling method is as follows:

s _v =s _i +hs _r , where 1≤i≤n, s _r ~N(0,1), s _v is the sampling value. .

Example:

Suppose s=[-4,-3.5,-2,-1,-0.75,1,3,3.2,4,4.2,4,6], after kernel density estimation, the solid line in Figure 2 is Dashed lines depict the Gaussian kernel applied to each original sample. When generating a virtual sample, first select an original sample. In Figure 2, the original sample at s=3 is selected, and the blue curve depicts the Gaussian kernel function at s=3. Then generate a one-dimensional random number s _r conforming to the normal distribution, where s _r =0.29. Finally, the virtual independent sample s _v is obtained according to h calculated by kernel density estimation. In this example, s=3, s _r =0.29, h=1.4614, according to the formula, s _v =3+0.29*1.461=3.4237.

Sampling is carried out according to the above steps until a satisfactory number of virtual independent samples s _v is obtained, and finally according to formula (3), the independent virtual samples are mapped back to the original sample space to obtain virtual samples,

x _v ⁽ⁱ⁾ = As _v ⁽ⁱ⁾ , i = 1, 2, . . . , m.

Claims (6)

1. A virtual sample generation method based on independent component analysis and kernel density estimation, is characterized in that, comprises the following steps: Step 1: Carry out independent component analysis on the original sample data, remove the correlation between attributes, and judge whether the analysis structure is convergent; Step 2, if convergent, estimate the probability density function by multi-kernel density estimation method for independent samples, and sample; if not convergent, estimate the probability density function by multi-kernel density estimation method for the original sample, and sample; Step 3, using the results of the independent component analysis in step 1 to restore the correlation of the convergently sampled data in step 2, so that the data after convergent sampling are mapped back to the original sample space to obtain virtual samples; Step 4, mixing the virtual samples with the original samples to obtain the final expanded sample set. 2. a kind of virtual sample generation method based on independent component analysis and kernel density estimation according to claim 1, it is characterized in that, when there is correlation between each attribute of sample data, the obtained result of independent component analysis is as follows: Assume that a small number of samples are collected as, x=(x ₁ ,x ₂ ,…,x _n ), x∈R ⁿ Assuming that x is obtained by linear transformation of n independent random variables s, then, s=(s ₁ ,s ₂ ,…,s _n ), s∈R ⁿ Assuming that A is a mixing matrix, there are, x ⁽ⁱ⁾ = As ⁽ⁱ⁾ , i = (1,2,...,m); Among them, x is the sample that has been collected, s is the independent random variable obtained after independent component analysis, and A is a constant. 3. a kind of virtual sample generation method based on independent component analysis and kernel density estimation according to claim 1, is characterized in that, adopts the method for estimation probability density function of multikernel density estimation method as follows: The mathematical expression for kernel density estimation is, Solve the smooth coefficient h according to the mean square integral error function, where f(s) is the true probability density function of s, is an estimate of f(s); After solving the smooth coefficient h, the estimation of the probability density function of s is completed. 4. a kind of virtual sample generation method based on independent component analysis and kernel density estimation according to claim 3, it is characterized in that, adopt Gaussian function as kernel function, Gaussian function expression is, 5. a kind of virtual sample generation method based on independent component analysis and kernel density estimation according to claim 1, is characterized in that, in step 2, the method for sampling is as follows: s _v ＝s _i +hs _r ，where1≤i≤n,s _r ～N(0,1) Among them, s _v is the sampling value. 6. a kind of virtual sample generation method based on independent component analysis and kernel density estimation according to claim 1, is characterized in that, original sample data is the data that training sample quantity is insufficient.