CN102609731A - Image classifying method for combining vision vocabulary books of different sizes - Google Patents

Abstract

The invention discloses an image classifying method for combining vision vocabulary books of different sizes and relates to the technical field of model recognition, computer vision and image understanding. According to the image classifying method disclosed by the invention, multi-resolution information is used for quantizing images and a plurality of available clues from different comprehensive layers are used for classifying the images in parallel. In order to utilize information of different particles to classify the images, the images are quantized based on the vision vocabulary books of different sizes; and the vision vocabulary books of different sizes can be used for capturing different image characteristics. Then, the images are trained based on the vision vocabulary books of different sizes to obtain different quantization vector sets so as to learn different classifiers; and each classifier can obtain different models of objects according to the information of different granularities of the images to integrate classifier models to classify new images to obtain a better effect. The experimental result shows that the performance of the vision vocabulary books of single size can be obviously improved and the robustness is very strong, so that the image classifying method can be used for achieving a good classifying effect on the different images.

Description

An Image Classification Method Combining Visual Lexicons of Different Sizes

technical field

The invention belongs to the technical fields of pattern recognition, computer vision and image understanding, and in particular relates to an image classification method.

Background technique

The difficulty of image classification lies in the need to establish a class model that can accommodate the height variation within a class and distinguish between different classes. The "Constellation" model attempts to locate different object parts and determine their spatial relationship. Although these methods may be expressive, such spatially constrained models cannot handle or recognize large deformations, such as rotations and occlusions that do not lie in a plane, nor do they consider objects with an uncertain number of localities, such as buildings and trees. Many popular methods for image classification represent images using collections of independent blocks described by local visual descriptors, most typical of which are "bag-of-words" models. It determines the specific proportion of parts in each class, while ignoring the spatial relationship between parts. After detecting interest points (individual patches) of an image and describing them with descriptors (i.e. feature representations), their distribution must be represented for training and test images. A popular representation method, also known as image quantization, is to obtain a visual vocabulary by clustering the described set of interest points. The image is then represented as a histogram of visual word labels. But almost all popular clustering algorithms require the user to input the number of clusters. In order to provide this parameter, the user must have some prior knowledge of the image or choose an appropriate parameter through many verification experiments. Recently, many methods based on the "bag-of-words" model are devoted to fusing multiple features for performance improvement. A popular trend in computer vision to combine multiple features is to use Multiple Kernel Learning (MKL). From the perspective of time complexity, the MKL method cannot learn multiple features in parallel.

The present invention attempts to apply the advantages of ensemble learning techniques to image classification. The idea of ensemble learning is to apply multiple learners and combine their predictions. Image classification is very difficult for traditional machine learning algorithms because the dimensionality of the vectors describing images is very high. To classify images using cues from different information integration layers, visual vocabulary members of different sizes are used to form a visual vocabulary collective. Performance gains can be obtained when applying a ensemble of classifiers learned on the corpus of visual vocabulary to classify new images. Moreover, from the perspective of time complexity, the present invention can learn member visual vocabulary and corresponding member classifiers in parallel, and has good parallelism and scalability.

The main contribution of this invention is to propose an image classification method combining visual vocabulary books of different sizes. The invention can effectively reduce the supervision degree of image classification, comprehensively utilize various effective information, learn object models in parallel, and effectively improve the efficiency and accuracy of image classification.

Contents of the invention

In order to solve the problem that image classification cannot effectively integrate multiple information and due to the high dimensionality of vectors describing images, traditional machine learning methods tend to produce models that are very unstable and have poor generalization ability, the present invention provides a method that combines different sizes Image Classification Approaches for Visual Lexicons.

The invention applies the advantages of integrated learning to image classification, and the features of different comprehensive levels are used to form a visual vocabulary collection. Based on the visual vocabulary, different quantization vectors can be obtained for the same image. Therefore, an ensemble of classifiers can be learned on different sets of representation vectors from the same training image set. Since each member utilizes one kind of image information, when using this classifier collectively to classify new images, unexpected satisfactory results can be obtained. Ensembling methods improve the performance of existing algorithms by combining the predictions of multiple models.

Similar to classifier collectives, visual vocabulary collectives are used to improve the quality and robustness of visual vocabulary. The vocabulary is generally learned from the training image set using a standard clustering algorithm, so the use of the vocabulary can also achieve the purpose of improving the quality of the clustering algorithm. This collective of visual vocabularies is used to express different types of image information. After constructing a differential visual vocabulary collective, a highly differentiated classifier collective can be obtained, in which each member classifier builds object models according to different image features. So when using this classifier collectively to classify new images, better and more robust results can be obtained. High diversity ensembles are also very effective in reducing the amount of supervision needed to build an accurate model.

The present invention quantizes images directly using multi-resolution information, classifying images in parallel using multiple available cues from different integration layers. To classify objects using information at different granularities, images are quantized based on visual vocabularies of different sizes that can capture image features at different granularities. Then based on the visual vocabulary of different sizes, the training image set obtains different quantization vector sets, so that different classifiers can be learned. Each classifier obtains different models of objects according to the information of different granularity of the image, and integrates these classifier models. Classify new images, including the following steps:

Step 1. Use the interest point detector to extract the interest points of the training image, and then use the descriptor to describe the extracted interest points;

Step 2. Randomly select a part of the well-described interest points, run the clustering algorithm on it to obtain a member visual vocabulary, and set different cluster numbers as the parameters of the clustering algorithm to obtain member visual vocabulary with different sizes;

Step 3. Quantize the training image set based on this member visual vocabulary;

Step 4. Learn a classifier on the quantized training data set;

Step 5. Repeat steps 2 to 4 to generate a preset size of visual vocabulary and classifier collectives;

Step 6. Based on a member visual vocabulary, quantify the new image;

Step 7. Use the corresponding member classifier to classify the new image to obtain the classification result;

Step 8. Repeat steps 6 to 7 until each member classifier gets its own classification result;

Step 9. Use the integration technique to integrate the classification results of the member classifiers to obtain the final image classification label.

Experimental results show that the method proposed by the present invention can increase robustness, because it is difficult to evaluate the quality of the classifier in high-dimensional problems, so users usually do not know which method to choose, the integration method can use many models, and then combine They produce stable results, and ensemble methods automatically focus on the information that best fits the given data.

The beneficial effect of the present invention is that it has the advantages of better average performance on images in different fields and strong robustness, and the model is simple, which is very suitable for general operators, does not require complex parameter adjustment, and has a low degree of supervision. The requirements are low; using the inherent parallelism of integrated learning, a small amount of training data can be used for parallel learning on multiple processors, so the efficiency of the present invention is relatively high.

Detailed ways

Preferred specific embodiments of the present invention:

，反例样本的权值设为

，这里#pos表示训练集中正例样本个数，#neg是训练集中反例样本个数。为了应用SVM (Support Vector Machine)分类器到多类问题，应用了一对多（one-against-all）的方法。 A descriptor corresponds to its nearest word in Euler space. After forming a member vocabulary, in order to quantify the image, all detected interest points are used to build a histogram based on this member vocabulary. To make the histogram independent of the number of descriptors, the histogram vector is normalized to sum to 1. The visual vocabulary is obtained by applying a clustering algorithm to a set of 200,000 descriptors randomly selected from the training image set. Weighted LibSVM is used to train the classifier. In the training phase, the weight of positive samples is set to

, and the weight of the negative sample is set to

, where #pos represents the number of positive samples in the training set, and #neg is the number of negative samples in the training set. In order to apply the SVM (Support Vector Machine) classifier to multi-class problems, a one-against-all method is applied.

For interest point detectors and descriptors, use Koen E. A. van de Sande et al. al.'s color descriptor software, this software performs the detection of interest points and the calculation of color descriptors. In order to improve the performance of members, a spatial pyramid structure 1x1+2x2+1x3 is used.

，则SVM的决策函数是：

，

是训练图像

和测试图像

间的核函数，

是

的类标签（+1 或 −1），是训练图像

的权值，b是决策阈值。

在这里设定为基于

距离的核函数：

，A是规范化距离的尺度参数，在这里设置成所有训练图像间的平均

距离。 For a test image, its feature vector is assumed to be

, then the decision function of SVM is:

,

is the training image

and a test image

The kernel function between

yes

class labels (+1 or −1), is the training image

The weight of b is the decision threshold.

set here based on

Kernel function for distance:

, A is the scale parameter of the normalized distance, here it is set to be the average of all training images

distance.

时，分类器集体的输出通过将所有成员分类器的决策函数值平均得到： To obtain the AP value, the output of the SVM decision function value is used directly. When testing a new image

When , the output of the classifier ensemble is obtained by averaging the decision function values of all member classifiers:

，这里S是集体大小，

是第i个成员分类器的输出值。通过设置不同的阈值，从而获得AP值和precision-recall曲线。

, where S is the collective size,

is the output value of the i -th member classifier. By setting different thresholds, the AP value and precision-recall curve are obtained.

The size of the member's visual vocabulary is set to 200, 400, 800, 1200, 1300, 1500, 1600, 1700, 1900, 2000, 2200, 2600, 2800, 3000, 3600, 4000, 4500, and 5000, and a size of A collective of 18, using Harris-Laplace interest point detectors and OpponentSift descriptors. The clustering algorithm uses the k-means clustering algorithm of the Euclidean distance measure.

Experimental results show that the preferred embodiment of the present invention has better performance than the traditional recognition method based on a single visual vocabulary, and even surpasses the performance of some complex models whose parameters have been carefully adjusted.

Claims (3)

1. An image classification method based on a visual vocabulary book, characterized in that multi-resolution information is used to quantify images, and multiple available clues from different comprehensive layers are used to classify images in parallel, in order to utilize information of different granularities to classify images , images are quantified based on visual vocabularies of different sizes, which can capture different image features, including the following steps: (1) Use the interest point detector to extract the interest points of the training image, and then use the descriptor to describe the extracted interest points; (2) Randomly select a part of the well-described interest points, run the clustering algorithm on it to obtain a member visual vocabulary, and set different cluster numbers as the parameters of the clustering algorithm to obtain member visual vocabulary with different sizes; (3) Quantize the training image set based on this member visual vocabulary; (4) Learn a classifier on the quantized training data set; (5) Repeat steps 2 to 4 to generate a preset size of visual vocabulary and classifier collectives; (6) Quantify new images based on a member visual vocabulary; (7) Use the corresponding member classifier to classify the new image to obtain the classification result; (8) Repeat steps 6 to 7 until each member classifier gets its own classification result; (9) Use the integration technique to integrate the classification results of the member classifiers to obtain the final image classification label. 2. The method according to claim 1, wherein the size of the members' visual vocabulary is set to 200, 400, 800, 1200, 1300, 1500, 1600, 1700, 1900, 2000, 2200, 2600, 2800 , 3000, 3600, 4000, 4500, and 5000 to get a collective of size 18. 3. The method according to claim 1, characterized in that in order to integrate the collective visual vocabulary and corresponding classifier collective classification images, directly use the output of the SVM decision function value, when testing a new image

When , the output of the classifier ensemble is obtained by averaging the decision function values of all member classifiers:

, where S is the collective size,

is the output value of the i- th member classifier, and the precision-recall curve is obtained by setting different thresholds.