JP5728534B2 - Integrated classifier learning apparatus, integrated classifier learning method, and integrated classifier learning program - Google Patents

Description

  The present invention relates to an integrated discriminator learning device, an integrated discriminator learning method, and an integrated discriminator learning program, and more particularly to constructing an integrated discriminator that integrates identification results of a plurality of discriminators to identify classes. The present invention relates to an integrated classifier learning device, an integrated classifier learning method, and an integrated classifier learning program.

  Generally, in supervised learning, observation data and a class label representing a class to which each observation data belongs are given as learning data. The observation data is extracted by some method, and is usually expressed as a feature vector. Then, based on a set (learning data) of a set of feature vectors and their class labels, the discriminator learns so as to correctly identify the learning data. The identification here means that in the case of K classes, the feature vector is classified into one of the K classes.

  Then, the test data whose class label is unknown is identified using a learned classifier. At this time, in the classifier, not only learning data but also identification accuracy (generalization performance) for test data is important. That is, a discriminator that is excessively adapted to the learning data and has low test data discrimination accuracy is impractical. This problem is called overfitting and is an important practical issue in the design of classifiers.

  So far, classifiers have been intensively studied in the fields of pattern recognition and machine learning, and many classifiers are currently available. However, when faced with a new identification task, it is difficult to design an appropriate feature vector for observation data if there is no prior knowledge or domain knowledge of what features are effective for identification. In addition, when the number of classes is large, there is a limit to designing a feature vector that is effective for identifying all classes. That is, with a single type of feature vector or single classifier, the discrimination performance is high only for a certain class of learning data, and the discrimination performance is insufficient for other classes of learning data. In many cases, it becomes a biased classifier such as.

  Therefore, it has been performed to obtain a test data identification result (class label) using an integrated classifier that integrates class labels that are identification results of a plurality of classifiers.

  The technique of the integrated identification method that integrates the identification results (class labels) of a plurality of classifiers is called ensemble learning, and several techniques have been proposed so far. The simplest method is a method of taking a majority vote of identification results (class labels) of a plurality of classifiers (see, for example, Non-Patent Document 1 and Non-Patent Document 2). Specifically, a plurality of subsets (partial learning data sets) are prepared by random extraction with respect to a labeled learning data set including a set of feature vectors and class labels. When there are J partial learning data sets, the discriminator is learned with each partial learning data set. Thereby, J discriminators are obtained. In identifying test data with unknown class labels, J test results (class labels) are obtained by identifying certain test data using J classifiers. Next, the majority class of the J class labels is taken and the class that appears most frequently is set as the identification result (class label) of the test data. It has been shown by theory and experiment that this method improves the reliability of classification results by taking a majority vote when the classification results of a single classifier are unstable. However, this method is based on the premise that the discrimination performance of each discriminator is relatively good. When many discriminators with low discrimination performance are included, the discrimination performance of the single discriminator is comparable. It becomes discrimination performance.

  As another method, there is a method of learning a meta-level discriminator using a discrimination result (class label) of the discriminator as a new feature (for example, see Non-Patent Document 3). Specifically, in the case of J discriminators, a J-dimensional vector having the discrimination result (class label) of each discriminator as an element can be configured for certain learning data.

  The jth element corresponds to the identification result of the jth classifier. By enumerating the discrimination results of the J discriminators for each learning data with respect to the J discriminators after learning, one J-dimensional feature vector is obtained for one learning data. That is, for N pieces of learning data consisting of a set of class labels and original feature vectors (hereinafter referred to as primary learning data), N pieces of sets consisting of a set of class labels and J-dimensional feature vectors are used. Learning data (hereinafter referred to as secondary learning data) is obtained. Non-Patent Document 3 proposes a Bayesian Classifier Combination (BCC) for constructing a discriminator using secondary learning data as new learning data, and the discrimination performance for test data compared to the majority method. Has been experimentally shown to improve. The idea of constructing a new classifier using secondary learning data composed of the classification results of a plurality of classifiers is known as stackedregression proposed in Non-Patent Document 4 and the like. Non-Patent Document 3 is positioned as a Bayesian model of this stacked regression.

Breiman, L., 1996. Bagging predictors, Machine Learning, 24, 123-140. Dietterich, T. G., 2000. Ensemble methods in machine learning.In Proceedingsof the First International Workshop on Multiple Classifier Systems, Springer-Verlag, LondonUK, 1-15. Kim, H.C. & Ghahramani, Z., 2012.Bayesian classifier combination.In Proceedingsof International Conference on Artificial Intelligence and Statistcs, AISTATS2012, http: //www.aistats.org/papers.php. Wolpert, D. H., 1992. Stacked generalization, Neural Networks, 5, 241-259.

  Since the conventional integrated identification method is based on the premise that the discrimination performance of each discriminator is good to some extent, there is a problem that the effect of integration cannot be expected if a discriminator with low discrimination performance exists. In fact, if feature extraction effective for identification cannot be realized in the target identification task, the performance of a single classifier is limited. Therefore, even if the identification performance of one class is high, the classification performance of the classification of another class is low. This phenomenon becomes more prominent as the number of classes increases.

  When such a phenomenon occurs, there is a problem that the identification accuracy of the integrated classifier that integrates the identification results of a plurality of classifiers may decrease.

  The present invention has been made in consideration of the above problems, and provides an integrated discriminator learning device, an integrated discriminator learning method, and an integrated discriminator learning program capable of improving the identification accuracy of the integrated discriminator. For the purpose.

  In order to achieve the above object, an integrated discriminator learning apparatus according to the present invention learns an integrated discriminator that integrates the discrimination results output from each of a plurality of discriminators to identify a class. Each of the plurality of discriminators for each of the plurality of pieces of primary learning data comprising a set of a feature vector of the data for each of a plurality of pieces of data and a true class to which the data belongs. Each of the secondary learning data consisting of a set of the identification result of identifying the class to which the data belongs and the true class to which the data belongs is acquired, and based on each of the acquired secondary learning data, Whether each of the plurality of classifiers outputs an identification result consistent with the true class is used to identify the class to which the data belongs by each of the plurality of classifiers. It was based on the identification result to learn the combined discriminator identifying a class by integrating the identification result.

Further, the integrated discriminator learning device according to the present invention provides data in which the class ω _k is a true class for each combination of the discriminator and the class ω _k based on each of the acquired secondary learning data. It is preferable to learn the integrated classifier including a binary latent variable r _j ^(k) indicating whether or not the j-th classifier outputs a consistent classification result.

The integrated classifier learning apparatus of the present invention, based on each of the front Symbol secondary learning data the acquired following formula (I) ~ (V) of the generation model of the secondary learning data represented by formula The parameters α, β, a, and b are learned by the following equation (VI), and are calculated by the following equations (VII) and (VIII) for each combination of the classifier and the class ω _k. that according to the probability distribution of the values of the latent variable r _j ^(k), it is preferable to determine the value of the latent variables r _j ^(k).

However, c _{i, j} ^(k) shows the identification result of the j-th identifier contained classes omega _k to the i-th pre-Symbol secondary learning data to true class, R, the potential Indicates a set of variables r _j ^(k) , where r _j ^(k) = 1 indicates that the j th discriminator outputs a consistent identification result for data whose class ω _k is a true class. , R _j ^(k) = 0 indicates that the j th discriminator outputs a consistent discrimination result for data whose class ω _k is a true class, and r _{\ (k, j)} is represents the set of latent variables excluding r _j ^(k) , C represents the set of secondary learning data, and n _{j, l} ^(k) represents the true class ω by the j-th discriminator. Indicates the number of data identified as class ω _l for data belonging to _k , and β _{j, l} ^(k) is a consistent identifier for data for which class ω _k is a true class. A parameter representing a directory distribution that generates a probability of identifying data belonging to the true class ω _k as class ω _l by the j th discriminator that outputs another result, and α _l is a consistent identification N ^(k) is a parameter representing a directory distribution that generates a probability of identifying the data as class ω _l by the classifier that does not output a result, and N ^(k) indicates the number of data in which class ω _k is a true class , Δ (x, y) is a delta function, and Γ (x) is a gamma function.

  The integrated discriminator learning method of the present invention is an integrated discriminator learning method in an integrated discriminator learning device that learns an integrated discriminator that identifies and discriminates the discrimination results output from each of a plurality of discriminators, The plurality of classifiers for each of the plurality of pieces of primary learning data composed of a set of a feature vector of the data for each of a plurality of pieces of data and a true class to which the data belongs. Each of which acquires secondary learning data consisting of a set of an identification result obtained by identifying a class to which the data belongs and a correct class to which the data belongs, and based on each of the acquired secondary learning data The data is attributed to each of the plurality of classifiers using whether or not each of the plurality of classifiers outputs an identification result consistent with a true class. The identification of classes based on the identification result of including the step of learning the combined discriminator for integrating the identification result.

  The integrated discriminator learning program of the present invention is for causing a computer to function as the integrated discriminator learning device of the present invention.

  According to the integrated discriminator learning device, the integrated discriminator learning method, and the integrated discriminator learning program of the present invention, the effect that the identification accuracy of the integrated discriminator can be improved is obtained.

It is the schematic which shows the outline of an example of the identification system of this Embodiment. It is a flowchart showing the flow of an example of the learning operation | movement of the integrated discriminator by the integrated discriminator learning device of this Embodiment. It is explanatory drawing which showed the specific example of the secondary learning data in the integrated discriminator learning device of the identification system of this Embodiment. It is a flowchart showing the flow of an example of the identification operation | movement of the test data by the integrated identification result output device of this Embodiment. It is explanatory drawing which showed the identification rate of HMM (single discriminator) learned by the feature expression whose window size is 48 in a present Example. It is explanatory drawing which showed the identification rate of the conventional integrated discriminator and the integrated discriminator of this Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that this embodiment does not limit the present invention.

  FIG. 1 shows a schematic configuration diagram showing an outline of an example of an identification system according to the present embodiment. The identification system 10 according to the present embodiment includes a function of generating secondary learning data 22 based on the identification result obtained by identifying the class of the primary learning data 12 by the classifier 14, and secondary learning by the integrated classifier learning device 20. The integrated discriminator 24 has a function of learning using the data 22 as teacher data, and a function of discriminating the test data 30 using the integrated discriminator 24 by the integrated discrimination result output device 32.

  In the identification system 10 of the present embodiment, the integrated discriminator learning device 20, the integrated identification result output device 32, and the like are configured to implement a CPU (Central Processing Unit), a RAM (Random Access Memory), This is realized by a computer having a ROM (Read Only Memory) or the like. When the CPU executes a program stored in the ROM, each operation described in detail later is executed. The primary learning data 12, the secondary learning data 22, and the test data 30 are stored in a storage unit such as an HDD (Hard Disk Drive), a storage medium, or the like. Note that the primary learning data 12, the secondary learning data 22, and the test data 30 are stored in a storage unit, a storage medium, or the like provided outside the identification system 10, and appropriately according to the execution of each function. You may make it acquire in the identification system 10. FIG.

First, the function of generating the secondary learning data 22 based on the identification result obtained by identifying the class of the primary learning data 12 by the classifier 14 will be described. Each of the primary learning data 12 is composed of a set of a feature vector and a true class label. Using this set of primary learning data 12, J discriminators 14 (first discriminator 14 ₁ to J th discriminator 14 _J ) are independently learned. In the following, the collective name is simply referred to as the discriminator 14, and the individual discriminator is referred to by an individual code such as the _jth discriminator 14j. In the present embodiment, the type of each classifier 14 and the learning method of each classifier 14 are not particularly limited and may be arbitrary.

The set of primary learning data 12 is D = {d ₁ ,..., D _N } consisting of N learning data. However, d _i = (x _i , y _i ) represents the i-th learning data, x _i represents a feature vector, and y _i represents a class label. When the number of classes to which the learning data belongs is K, the class label has a value of 1, 2,. Further, if D _−i represents a set of learning data obtained by removing the i-th learning data d _i from the set D of primary learning data, after learning J discriminators 14 with D _−i , By inputting the feature vector x _i of the i-th learning data d _i that is not used for learning to the learned J discriminators 14, J discrimination results are obtained. By executing this process for i = 1,..., N, a set of secondary learning data 22 is obtained. In this case, the secondary learning data 22 includes a set of a J-dimensional feature vector (each element is a class label) and a true class label. That is, the set of secondary learning data 22 includes N sets of learning data in which J-dimensional feature vectors and true class labels are paired.

  Next, the function of performing learning of the integrated classifier 24 using the secondary learning data 22 as teacher data by the integrated classifier learning device 20 of the present embodiment will be described. The technique for performing the learning is the identification according to the present embodiment for learning the integrated classifier 24 for correctly identifying as many test data 30 as possible from the observation data C that is a set of known identification results. This is a main technique in the integrated classifier learning device 20 of the system 10. FIG. 2 is a flowchart showing an example of the learning operation of the integrated discriminator 24 by the integrated discriminator learning device 20.

In the integrated discriminator learning device 20 of the present embodiment, first, in step S100, secondary learning based on the discrimination results of the J discriminators 14 (first discriminator 14 ₁ to J th discriminator 14 _J ). A set of data 22 is acquired.

  In the next step S102, parameters (α, β, a, b, details will be described later) of the generation model of the secondary learning data 22 are estimated and stored based on the acquired set of secondary learning data 22. In the present embodiment, the storage of these parameters is stored in a storage unit (not shown) provided in the integrated discriminator learning device 20 or the like.

In the next step S104, a set of latent variables R = {r _j ^(k) } is determined and stored based on the estimated parameters, and this operation is terminated. In the present embodiment, the storage of the set R of latent variables is performed by a storage unit (not shown) provided in the integrated classifier learning device 20 and the like, similar to the parameters of the generation model of the secondary learning data 22 described above. And so on.

  In the integrated discriminator learning apparatus 20 of the present embodiment, a plurality of (this embodiment) are obtained by estimating the parameters of the generation model of the secondary learning data 22 and determining and storing the set R of latent variables. In this case, among the J classifiers 14, which classifier 14 is effective for identifying which class is effective. In the integrated classifier learning device 20 of the present embodiment, the criterion for determining whether the classifier 14 is effective for identification is not limited to whether the identification result is correct. In the integrated discriminator learning device 20, a discriminator 14 that outputs a consistent discrimination result for a true class (a class to which learning data should belong originally) is used as a discriminator 14 effective for discriminating the class. Deciding. Intuitively, when a certain classifier 14 frequently identifies the primary learning data 12 of class 1 as class 2, if the classification result of the classifier 14 is class 2, the true class is actually , Learning by modeling the idea that it is likely to be class 1.

  Hereinafter, details of an example of each operation of the learning operation (FIG. 2, steps S100 to S104) will be described.

A matrix C ^(k) of size N ^(k) × J represents a set of secondary learning data 22 for a true class ω _k, and primary learning data in which all classes are true classes. A set of secondary learning data 22 consisting of a set of identification results for 12 and its class label is written as C = U _{k = 1} ^K C ^(k) . K represents the total number of classes. N ^(k) represents the number of learning data in which the class ω _k is a true class. C the the (i, j) th component c _{i, j} ^(k) of the matrix ^(k) is the j-th discriminator 14 _j of the identification result for the i-th training data class omega _k and true class It corresponds to. That is, it can be expressed as c _{i, j} ^(k) ∈ {1,..., K}.

FIG. 3 shows a specific example of the secondary learning data 22. In the secondary learning data 22 shown in FIG. 3, since the number of classes is 3 (3-class problem), K = 3, J = 10, N = 15, N ^(k) = 5 (k = 1, 2, This is an example in the case of 3). In FIG. 3, the true class label is not explicitly shown, but it is assumed that true class labels 1, 2, and 3 are assigned in order from the top with five data delimiters. For example, the first row expressed in the matrix format of FIG. 3 shows the discrimination results (classes) by the ten discriminators 14 for the first primary learning data 12 having class 1 (class label 1) as a true class. Label). As described above, according to the single discriminator 14, the identification result may be a biased result that data of a certain class can be identified relatively correctly, but data of another class is easily misidentified. . This tendency becomes more prominent as the number of classes increases. Therefore, it is not appropriate to treat the discrimination results of all the discriminators 14 on an equal basis.

  Therefore, in the integrated classifier learning device 20 of the present embodiment, a latent variable indicating whether or not a certain classifier 14 is effective for identifying each class is introduced, and the latent variable is learned. However, the discriminator 14 having many misidentifications in identifying a certain class is not always effective (invalid) for identifying the class. In other words, if the classifier 14 is consistent with the true class (correct classification result) even if almost all of the classes are erroneously identified, the classifier 14 can be said to be effective for classifying the class. In FIG. 3, for each discriminator 14, the correct identification result having consistency in each class is shaded, which is misidentification, but the identification result having consistency with the true class is displayed. Is shaded.

For example, the second discriminator 14 ₂ represented by the discriminator id = 2 in FIG. 3, in the identification of all classes, the identification result is correct, since it has a consistent, effective to identify all classes It is.

Further, the third classifier 14 ₃ indicated by the discriminator id = 3 in FIG. 3, in the identification of Class 2 and Class 3, the identification result is correct, since it has a consistent, Class 2 and Effective for class 3 identification. However, since, the third classifier 14 _3, where in the identification of class 1, and a misidentification, also the identification result does not have a consistency against true class "1", the class 1 is invalid for identification.

Further, the fifth discriminator 14 ₅ represented by the identifier id = 5 in Figure 3, in the identification of Class 2 and Class 3, the identification result is correct, since it has a consistent, Class 2 and Effective for class 3 identification. Further, the fifth discriminator 14 _5, in the identification of class 1, although the misidentification, consistency of the identification result identifies the class "2" to the true class "1" Therefore, it is also effective for class 1 identification.

Also, the 10 th classifier 14 ₁₀ represented by the identifier id = 10 in FIG. 3, in the identification of class 2, although the misidentification with respect to the identification result is true class "2" Therefore, it is effective for class 2 identification. However, the tenth discriminator 14 ₁₀ misclassifies class 1 and class 3, and the identification result is consistent with the true classes “1” and “3”. It is invalid for class 1 and class 3 identification.

Here, c _i ^(k) = (c _{i, 1} ^(k) ,..., C _{i, J} ^(k) ) is the J dimension for the i-th learning data in which the class ω _k is a true class. Can be regarded as discrete feature vectors. If J discriminators are statistically independent, the probability distribution of class ω _k is expressed by the following equation (1).

However, as described above, since the identification result is not always correct, such a simple modeling is not sufficient. Therefore, in the integrated discriminator learning device 20, a binary latent variable r _j ^(k) indicating whether or not the j-th discriminator 14 _j outputs a discrimination result consistent with the true class ω _k . ⁾ . When r _j ^(k) = 1, the j-th discriminator 14 _j outputs a consistent discrimination result for the true class ω _k , and the j th discriminator for the learning data with the class ω _k as the true class. It is assumed that the j-dimensional feature is generated according to a distribution unique to the class ω _k . On the other hand, when r _j ^(k) = 0, the j-th identifier 14 _j is not output the identification result with a consistency against true class omega _k, the class omega _k and true class learning Assume that the jth dimension features for the data are generated according to a distribution that does not depend on the class ω _k . Further, assume that c _{i, j} ^(k) is generated by the following probability model.

Here, θ _j ^(k) = {θ _{j, l} ^(k) } _{l = 1} ^K and φ = {φ _l } _{l = 1} ^K. Θ _{j, l} ^(k) is the probability that when r _j ^(k) = 1, the discrimination result of the j-th discriminator 14 _j for the data with the class ω _k as a true class is the class ω _l. Represents. Φ _l represents the probability that the identification result of the j-th discriminator 14 _j for the data having the class ω _k as a true class is the class ω _l when r _j ^(k) = 0. As described above, r _j ^(k) = 0 means that the j-th discriminator 14 _j is not effective in class ω _k discrimination, and therefore φ _l does not depend on j and k. a and b are parameters of the beta distribution (beta). α = {α _l } _{l = 1} ^K and β _j ^(k) = {β _{j, l} ^(k) } _{l = 1} ^K are parameters of the Dirichlet distribution (Dirichlet).

The meanings of the formulas (2) to (6) are as follows. First, λ is generated from a beta distribution (Beta) having parameters a and b. Next, φ is generated from a Dirichlet distribution (Dirichlet) having a parameter α. Also, θ _j ^(k) is generated from a Dirichlet distribution with parameter β _j ^(k) . Then, the latent variable r _j ^(k) is generated from the Bernoulli distribution (Bernoulli) of the parameter λ. Then, when r _j ^(k) = 1, from the discrete distribution (Discrete) with θ _j ^(k) = (θ _{j, 1} ^(k) ,..., Θ _{j, K} ^(k) ) as a parameter, c _{i, j} ^(k) is generated. In addition, when r _j ^(k) = 0, c _{i, j} ^(k) is generated from a discrete distribution whose parameter is φ = (φ ₁ ,... Φ _K ).

  The above equations (2) to (6) correspond to a generation model of the secondary learning data 22. That is, if the parameters appearing in the equations (2) to (6) are appropriately given, it means that data substantially similar to the actually observed secondary learning data 22 can be obtained. Of course, since the parameter is unknown, the parameter is estimated using the observation data C which is the actually observed identification result. This estimation corresponds to learning in the integrated classifier learning device 20.

Learning can be realized by the following Bayesian estimation framework. {C _i ^(k) } _{i = 1 From} the statistical independence of ^K , the likelihood of the observation data C is expressed by the following equation (7).

In the equation (7), R = {r _j ^(k) }. Further, n _{j, l} ^(k) represents the number of data in which data having the class ω _k as a true class is identified as the class ω _l by the j-th discriminator 14 _j . That is, n _{j, l} ^(k) = Σ _{i = 1} ^{N (k)} δ (c _{i, j} ^(k) , l). However, δ (x, y) is a delta function, and takes a value of 1 when x = y, and takes a value of 0 otherwise. Let Θ = {θ _j ^(k) }. Furthermore, Θ, φ, and λ can all be integrated and eliminated due to the conjugate nature of the prior distribution.

In the above equation (8), Γ (x) represents a gamma function. Also, α _● = Σ _{l = 1} ^K α _l and β _{j, ●} ^(k) = Σ _{l = 1} ^K β _{j, l} ^(k) . Similarly, the following equation (9) can be obtained.

    From the above formulas (7) and (8), a posterior distribution represented by the following formula (10) is obtained.

  The empirical Bayes method is used to calculate the marginal likelihood regarded as a function of each parameter (α, β, a, b) based on the set C of the secondary learning data 22 by taking the logarithm of the right side of equation (10). The parameters (α, β, a, b) are learned by estimating the parameters (α, β, a, b) so as to maximize the parameters.

If P (R | C; α, β, a, b) is obtained, R = {r _j ^(k) } can be determined stochastically by the Gibbs sampling method. r _{\ (k, j)} and (s, u) ≠ (k , j) if it is assumed to represent all of R = other than ^{_{{r s (u)},}} P (r j (k) = 1 | r \ _{(k, j)} , C) + P ( _rj ^(k) = 0 | r _{\ (k, j)} , C) = 1, the ratio ν = P ( _rj ^(k) = 1 | r _{\ (k, j)} , C) / P (r _j ^(k) = 0 | r _{\ (k, j)} , C) to obtain the probability distribution of the value of r _j ^(k) by the following equation (11) Can do.

  However, (nu) is calculated | required as following (12) Formula.

For each pair of the jth discriminator 14j and class ωk, the value of r _j ^(k) is determined according to the probability distribution obtained by the above equation (11). Note that the determination of the value of r _j ^(k) is repeated T times.

  In this manner, in the identification system 10 according to the present embodiment, the integrated classifier learning device 20 learns the integrated classifier 24.

  The integrated identification result output device 32 uses the learned integrated classifier 24 to identify the test data 30 whose class label is unknown. Next, a function for identifying the test data 30 using the integrated identifier 24 by the integrated identification result output device 32 will be described.

  FIG. 4 shows a flowchart showing an example of the operation of identifying the test data 30 by the integrated identification result output device 32.

In the integrated identification result output apparatus 32 according to the present embodiment, first, in step S200, the test data 30 is identified by the J classifiers 14 (the first classifier 14 ₁ to the Jth classifier 14 _J ). Get the identification result. Note that the discriminator 14 used at this time is the same as the discriminator 14 used in the learning of the integrated discriminator 24.

In the next step S202, the optimal class ω _{k *} of the test data 30 is derived using the integrated classifier 24 learned by the integrated classifier learning device 20. Specifically, since J class labels are obtained in step S200, the optimal class ω _{k *} is derived by inputting a J-dimensional feature vector representing the identification result to the integrated classifier 24. .

In the next step S204, the optimum class ω _{k *} derived in step S202 is output as the identification result of the integrated discriminator 24 to, for example, the outside of the identification system 10, etc., and then this operation is terminated.

  Hereinafter, details of an example of each operation of the identification operation (FIG. 4, steps S200 to S204) will be described.

The discrimination result by each discriminator 14 for the mth test data of M test data 30 whose class labels are unknown is represented by _cm ^* = ( _{cm, l} ^* ,..., Cm _{, J} ^*. ). In this case, the identification system 10 according to the present embodiment recognizes the class of the mth test data 30 for each of m = 1,... This is performed using the discriminator 24.

If R = {r _j ^(k) } is obtained in the integrated discriminator learning device 20 as described above, the test data 30 can be identified by the integrated discriminator 24 in the following procedure. Bayes optimal class omega _{k *} is for c _{m ^*,} prediction posterior distribution for ^{_{c m *: P (ω k}} | c m *, C) corresponds to a k that maximizes. From the Bayes' theorem, the optimal class ω _{k *} is obtained by the following equation (13).

In the equation (13), P (ω _k ) is a class prior distribution and is usually a uniform distribution. At this time, P (ω _k ) can be approximately calculated using the Monte Carlo approximation of the following equation (14).

(14) In the equation, _{r j} ^(k) (t) ^{_{is, P (r j (k)}} | r \ (k, j), C) r j (k in the t-th iteration in the Gibbs sampling based on ⁾ Represents the value of (t). As described above, since the determination process of the value of r _j ^(k) is repeated T times, r _j ^(k) (t) represents the determined value in the t-th iteration. t ₀ represents the burn-in time. That is, the value of r _j ^(k) (t) (t = 1,..., T ₀ ) is rejected as an initial sampling value of Gibbs sampling that has low reliability.

Further, the right side of the above equation (14) can be analytically calculated as in the following equations (15) and (16) depending on the value of r _j ^(k) .

In the equations (15) and (16), n _{j, l} ^(●) = Σ _{k = 1} ^K n _{j, l} ^(k) .

Finally, the optimum class ω _{k *} obtained by the integrated discriminator 24 for the m-th test data is obtained according to the following equation (17).

Expression (17) represents an identification rule of the test data 30 by the integrated classifier 24. From equation (17), it can be confirmed that the identification rule depends on the value of r _j ^(k) .

(Example)
A specific embodiment using the integrated classifier 24 constructed by the identification system 10 as described above will be described.

  As an identification task that is difficult to identify with a single classifier 14, an experiment result when 22 types and 14 types of nurse behavior data are used as test data 30 will be described. The primary learning data 12 was created by the following procedure.

  Data on 22 types of nursing patients and 14 types of nursing behaviors (hereinafter simply referred to as “behavior”) obtained from a total of four 3-axis accelerometers attached to the nurse's wrists, chest pockets, and waist. In contrast, a sliding window with a certain time width was set, and features were extracted while shifting in the time axis direction with 50% overlap. The average value, standard deviation, and energy and entropy in the frequency domain were adopted as features. For these features, see Bao, L., and Intille, S., 2004.Activity recognition from user-annotated acceleration data, In Proceedings of International Conference on Pervasive Computing, Pervasive2004, Springer-Verlag, 1-17. It is described in detail. By connecting these features, primary learning data 12 comprising a sequence of 48-dimensional feature vectors and a true class label that is an action class of the sequence is obtained. Next, a hidden Markov model (HMM) is learned using the primary learning data 12. Note that the feature vector of the primary learning data 12 here is time-series information, and thus becomes a time-series feature vector.

  The nurse behavior used in this embodiment is not uniform in operation, so it is difficult to determine the optimal window size for each behavior class. That is, it is desirable to reduce the sliding window time width for fast-moving behavior. Conversely, for actions with slow movements, it is desirable to increase the sliding window in terms of improving the action class identification rate. For this reason, it is difficult to optimize the time width of a single sliding window. Therefore, in this embodiment, the primary learning data 12 composed of a time series of 50 types of feature vectors and their class labels is configured with a time width of 4, 6,. Learned. That is, the integrated discriminator learning device 20 acquires the discrimination results obtained by the HMMs of 50 (J = 50) discriminators 14 for learning by the integrated discriminator 24. Among them, the discrimination rate of the HMM having the best discrimination rate in the test data (HMM learned by the feature expression having a window size of 48) was about 42% as shown in FIG. FIG. 5 shows the discrimination rate of the HMM (single discriminator 14) trained by the feature expression having a window size of 48. In FIG. 5, “()” represents a standard deviation. Then, the integrated discriminator learning device 20 creates secondary learning data 22 from the discrimination results obtained by these 50 HMMs, and uses this to learn the integrated discriminator 24.

The discrimination rates of discrimination by a conventional discriminator and discrimination by the integrated discriminator 24 of the present embodiment were compared. The total number of learning data items of the primary learning data 12 is 1,097. From this, five data sets composed of learning data and test data 30 were created, and the average value and standard deviation of the discrimination rate in test data 30 were compared. As a conventional method, in addition to the above-mentioned two models of IBCC (Independent Bayesian Classifier Combination) and EBCC (Enhanced Bayesian Classifier Combination) which are the BCC models of Non-Patent Document 3, the majority decision classification which is a representative integrated classifier is used. LR + Lasso with L1 regularization term added to logistic regression (Logistic Regression) based on the above mentioned non-patent document 4 (MajorityVoting: MV), naive Bayes model (NB), support vector machine (linear kernel) : SVM (L)), and polynomial kernel: SVM (P)). The NB model corresponds to the case where r _j ^(k) = 1 is set for all the k, j pairs in the identification system 10 (integrated classifier 24) of the present embodiment. Neither LR + Lasso nor SVM is an integrated identification method, but secondary learning data is used as learning data, and is applied in the framework of the stacked regression of Non-Patent Document 4.

  FIG. 6 shows the identification rates of the conventional integrated classifier and the integrated classifier 24 of the present embodiment. In FIG. 6, “()” represents a standard deviation. As is clear from FIG. 6, it can be confirmed that the integrated discriminator 24 of the present embodiment has obtained a significantly superior discrimination result. As described above, the learning method of the integrated discriminator 24 by the integrated discriminator learning device 20 according to the present embodiment can be regarded as an extension of the latent variable of the NB model. This can be confirmed experimentally. In the learning method by the integrated discriminator learning device 20 of the present embodiment, the meta discriminator is learned based on Gibbs sampling. However, since it converges at about 100 iterations, there is a problem in calculation time. In addition, the identification for the test data 30 can be calculated in real time per sample.

As described above, in the integrated classifier learning device 20 in the identification system 10 of the present embodiment, the identification results and true classes of each of the plurality of classifiers 14 are used as the secondary learning data 22 (teacher data), Learning of the integrated discriminator 24 is performed. In the learning, the integrated discriminator learning device 20 does not simply enable only a discriminator that performs correct discrimination in the learning, but even if it is a discriminator that performs misrecognition, the discrimination result is obtained for a true class. If the classifier 14 determines that the classifier 14 outputs a consistent identification result when the class is a true class, the latent variable r _j Learn as ^(k) .

  As a result, in the present embodiment, it is possible to realize the integrated discriminator 24 having a high discrimination performance even if it is difficult to discriminate with a single type of feature or with a single type of discriminator 14. it can. Further, when integrating a plurality of classifiers 14, it is possible to effectively integrate classifiers 14 having a low discrimination capability for all classes but having a high discrimination capability for some classes.

  Therefore, according to the integrated classifier learning device 20 of the present embodiment, the identification accuracy of the integrated classifier can be improved.

  Further, by using the integrated classifier learning device 20 of the present embodiment, it is not necessary to select the best classifier for practical use, and the identification system 10 can be constructed more easily.

Further, by learning the integrated discriminator 24 by the integrated discriminator learning apparatus 20 of the present embodiment, the identification system 10 has versatility and robustness. Specifically, in order to integrate the discrimination results of a plurality of discriminators 14, a general-purpose that does not depend on what kind of feature vector each discriminator 14 has learned and what kind of discriminator 14 is used. Have sex. That is, it is sufficient that only the identification results (class labels) of a plurality of classifiers with respect to the learning data are given. In other words, this is realized by being applicable to an arbitrary classifier 14. Also, in the conventional method, when the classifiers to be integrated include classifiers with poor identification accuracy, this implementation is performed for the problem that the identification accuracy is not improved by integrating the identification results. In this form, even if the discriminator 14 with poor discrimination accuracy is included in the discriminator 14 into which the discrimination results are integrated, if the discriminator 14 is consistent in misclassification, the integrated discriminator 24 is configured. In the present embodiment, the set of latent variables R = {r _j ^(k) } is obtained by the Gibbs sampling method. However, the present invention is not limited to this, but may be determined by the Markov chain Monte Carlo method (MCMC), the metropolis method, or the like. However, since there are R (aggregate elements) as many as k and j, it is preferable to determine by Gibbs sampling.

  Although the present embodiment has been described in detail with reference to the drawings, the present embodiment is an example, and the specific configuration is not limited to the present embodiment, and departs from the gist of the present invention. Needless to say, the range of designs that are not included is included and can be changed according to the situation.

DESCRIPTION OF SYMBOLS 10 Identification system 12 Primary learning data 14 Classifier 20 Integrated discriminator learning apparatus 22 Secondary learning data 24 Integrated discriminator 30 Test data 32 Integrated discrimination result output apparatus

Claims (5)

An integrated discriminator learning device that learns an integrated discriminator that integrates discrimination results output from each of a plurality of discriminators to identify a class,
For each of the plurality of data of the primary learning data consisting of a set of a feature vector of the data for each of a plurality of data and a true class to which the data belongs, the data is attributed to each of the plurality of classifiers. Secondary learning data each consisting of a set of identification results obtained by identifying a class to be identified and a true class to which the data belongs are acquired, and the plurality of identifications are based on each of the acquired secondary learning data Based on the identification result obtained by identifying the class to which the data belongs by each of the plurality of classifiers using whether or not each classifier outputs an identification result consistent with the true class. Learning an integrated classifier that identifies the class by integrating the identification results;
Integrated classifier learning device. Based on each of the acquired secondary learning data, for each combination of the classifier and class ω _k , the jth classifier is consistent for data in which the class ω _k is a true class. The integrated discriminator learning device according to claim 1, wherein the integrated discriminator includes a binary latent variable r _j ^(k) indicating whether to output an identification result. Based on each of the front Symbol secondary learning data the acquired, which is a parameter of generation model of the secondary learning data represented by the following formula (I) ~ (V) Formula alpha, beta, a, and b , Learn by the following formula (VI)
Wherein for each combination of classifiers and class omega _k, according to the probability distribution of the values of the following (VII) wherein and said latent variables r _j calculated by (VIII) Formula ^(k), wherein latent variables r _j ⁽ The integrated discriminator learning device according to claim 2, wherein a value of ^k) is determined.

However, c _{i, j} ^(k) shows the identification result of the j-th identifier contained classes omega _k to the i-th pre-Symbol secondary learning data to true class, R, the potential Indicates a set of variables r _j ^(k) , where r _j ^(k) = 1 indicates that the j th discriminator outputs a consistent identification result for data whose class ω _k is a true class. , R _j ^(k) = 0 indicates that the j th discriminator outputs a consistent discrimination result for data whose class ω _k is a true class, and r _{\ (k, j)} is represents the set of latent variables excluding r _j ^(k) , C represents the set of secondary learning data, and n _{j, l} ^(k) represents the true class ω by the j-th discriminator. Indicates the number of data identified as class ω _l for data belonging to _k , and β _{j, l} ^(k) is a consistent identifier for data for which class ω _k is a true class. A parameter representing a directory distribution that generates a probability of identifying data belonging to the true class ω _k as class ω _l by the j th discriminator that outputs another result, and α _l is a consistent identification N ^(k) is a parameter representing a directory distribution that generates a probability of identifying the data as class ω _l by the classifier that does not output a result, and N ^(k) indicates the number of data in which class ω _k is a true class , Δ (x, y) is a delta function, and Γ (x) is a gamma function. An integrated discriminator learning method in an integrated discriminator learning device that learns an integrated discriminator that identifies and discriminates the discrimination results output from each of a plurality of discriminators,
The plurality of classifiers for each of the plurality of pieces of primary learning data composed of a set of a feature vector of the data for each of a plurality of pieces of data and a true class to which the data belongs. Each of which acquires secondary learning data consisting of a set of an identification result obtained by identifying a class to which the data belongs and a correct class to which the data belongs, and based on each of the acquired secondary learning data The classifier to which the data belongs is determined by each of the plurality of classifiers using whether or not each of the plurality of classifiers outputs an identification result consistent with the true class. Learning an integrated classifier that integrates the identification results based on the identified results.
Integrated classifier learning method. An integrated discriminator learning program for causing a computer to function as the integrated discriminator learning device according to any one of claims 1 to 3.