Hyperspectral tree crown classification using the multiple instance adaptive cosine estimator

MI-ACE target characterization

MI-ACE (Zare, Jiao & Glenn, 2018) is a discriminative target characterization method that is based on ACE detector and multiple instance concept learning. In multiple instance learning, sets of data points (termed bags) are labeled as either positive or negative. Specifically, let $\mathbf{X}=\left[{\mathbf{x}}_1,\dots ,{\mathbf{x}}_N\right]\in {\mathbb{R}}^{d\times N}$ be training data where d is the dimensionality of an instance, x_i, and N is the total number of training instances. The data are grouped into K bags, $\mathbf{B}=\left\{{\mathbf{B}}_1,\dots ,{\mathbf{B}}_K\right\}$, with associated binary bag-level labels, $L=\left\{L_1,\dots ,L_K\right\}$ where $L_j\in \left\{0,1\right\}$ and x_ji ∈ B_j denotes the ith instance in bag B_j. Positive bags (i.e.,  B_j with L_j = 1, denoted as ${\mathbf{B}}_j^{+}$) contain at least one instance composed of some target: (1)${\displaystyle \text{if}{L}_j=1,\exists {\mathbf{x}}_{ji}\in {\mathbf{B}}_j^{+}\text{s.t.}{\mathbf{x}}_{ji}\sim \mathcal{N}\left({\alpha }_{it}\mathbf{s}+{\mu }_b,{\sigma }_1^2{\Sigma }_b\right),{\alpha }_{it}\ne 0}$ where Σ_b is the background covariance, μ_b is the mean of the background, s is the known target signature which is scaled by a target abundance, a, and ${\sigma }_1^2=\frac{1}{d}{\left(\mathbf{x}-a\mathbf{s}\right)}^T{\Sigma }_b^{-1}\left(\mathbf{x}-a\mathbf{s}\right)$. However, the number of instances in a positive bag with a target component is unknown. If B_j is a negative bag (i.e.,  L_j = 0, denoted as ${\mathbf{B}}_j^{-}$), then this indicates that ${\mathbf{B}}_j^{-}$ does not contain any target: (2)${\displaystyle \text{if}{L}_j=0,{\mathbf{x}}_{ji}\sim \mathcal{N}\left({\mu }_b,{\sigma }_1^2{\Sigma }_b\right)\forall {\mathbf{x}}_{ji}\in {\mathbf{B}}_j^{-}}$

Given this problem formulation, the goal of MI-ACE is to estimate the target signature, s, that maximizes the corresponding adaptive cosine estimator (ACE) detection statistic for the target instances in each positive bag and minimize the detection statistic over all negative instances. This is accomplished by maximizing the following objective: (3)${\displaystyle arg\underset{\mathbf{s}}{max}\frac{1}{N^{+}}\sum _{j:L_j=1}{D}_{ACE}\left({\mathbf{x}}_j^{\ast },\mathbf{s}\right)-\frac{1}{N^{-}}\sum _{j:L_j=0}\frac{1}{N_j^{-}}\sum _{{\mathbf{x}}_i\in B_j^{-}}{D}_{ACE}\left({\mathbf{x}}_i,\mathbf{s}\right)}$ where N⁺ and N⁻ are the number of positive and negative bags, respectively, $N_j^{-}$ is the number of instances in the jth negative bag, and ${\mathbf{x}}_j^{\ast }$ is the selected instance from the positive bag $B_j^{+}$ that is mostly likely a target instance in the bag. The selected instance is identified as the point with the maximum detection statistic given a target signature, s: (4)${\displaystyle {\mathbf{x}}_j^{\ast }=arg\underset{{\mathbf{x}}_i\in B_j^{+}}{max}{D}_{ACE}\left({\mathbf{x}}_i,\mathbf{s}\right)}$ Since the first term of the objective function relies only on the selected instance from each positive bag, the method is robust to outliers and incorrectly labeled samples. In other words, a target point will have a small spectral distance to the target signature and, thus, likely to be identified as the selected instance for a bag whereas outliers will be far from the target signature and disregarded. The D_ACE is the ACE detection statistic, (5)${\displaystyle D_{ACE}\left(\mathbf{x},\mathbf{s}\right)={\left(\frac{\hat{\mathbf{s}}}{\parallel \hat{\mathbf{s}}\parallel }\right)}^T\left(\frac{\hat{\mathbf{x}}}{\parallel \hat{\mathbf{x}}\parallel }\right)={\widehat{\stackrel{ˆ}{\mathbf{s}}}}^T\widehat{\stackrel{ˆ}{\mathbf{x}}}}$ where $\hat{\mathbf{x}}={\mathbf{D}}^{-\frac{1}{2}}{\mathbf{U}}^T\left(\mathbf{x}-{\mu }_b\right)$ and $\hat{\mathbf{s}}={\mathbf{D}}^{-\frac{1}{2}}{\mathbf{U}}^T\mathbf{s}$, U and D are the eigenvectors and eigenvalues of the background covariance matrix, respectively.

As outlined by  Zare, Jiao & Glenn (2018), the MI-ACE algorithm optimizes Eq. (3) using an alternating optimization strategy. After optimization, an estimate for a discriminative target signature (that is used to distinguish between two classes and perform pixel level classification using the D_ACE detector) is obtained. This iterative approach does not require any parameter tuning. The MI-ACE code is available and published on our GitHub site (Zare, Glenn & Gader, 2018).

Proposed one-vs-one MI-ACE

The original MI-ACE algorithm was designed for target detection. Target detection can also be viewed as a two-class classification problem with one class being target and the other class being non-target or background (often with heavily imbalanced class sizes). In this work, we investigate whether MI-ACE can be extended to multi-class classification problems and applied to tree crown species classification using a one-vs-one classification strategy. Two MI-ACE classifiers are trained for every pair of two classes in the multi-class classification problem. Two classifiers are trained so that each class can be considered as the target class once in this pair. An MI-ACE classifier consists of a trained discriminative target signature (estimated using the MI-ACE approach outlined in the previous section), a background mean and covariance computed using the training samples from the non-target class, and a threshold value used to assign a target or non-target label to individual data points given their ACE detection confidence computed using the estimated target signature, background mean and background covariance values, as is shown in Fig. 1.

During testing, each trained MI-ACE classifier is applied to an input test point. Since each classifier yields a classification result, the final classification for a testing bag is obtained by aggregating all of the individual results. Specifically, a test bag is assigned the class label associated with the class that had the largest number of votes associated with each class. The votes are tallied by, first, averaging the confidence values estimated from each of the individual classifiers applied to each test point in the bag and, then, thresholding these average confidence values to obtain a binary target vs. non-target label the test bag. Then, the class label with the largest number of votes from the binary classification results is assigned as the final class labe, as is shown in Fig. 2.

Pseudocode for the proposed method is shown below. In the pseudocode, let X and Y be the set of all training and testing bags, respectively, with X_L being the set of all bags assigned label L, Y_m being the mth testing bag, and Y_m,n being the nth data point in the mth testing bag. Let L and R be the corresponding bag level labels for the training and testing bags, respectively. C denotes the number of classes. M denotes the number of testing bags. The variables s_i,j and τ_i,j represent the estimated target signature and classification threshold for target class i and background class j, respectively, and z_m,i,j denotes the confidence value estimated by ACE detector given the estimated s_i,j and τ_i,j values for the mth test bag. The threshold value is set by determining the threshold that minimizes classification error on the training data. This approach does not have any parameters to tune as all parameters are estimated from the training data.

The threshold value, τ_i,j, is estimated from the training data by identifying the threshold that minmizes classification error on the training data. After training a set of one-vs-one classifiers as outlined in the psuedo-code, the estimated target signature s_i,j is used to apply the ACE detector to the training data to compute their associated confidence scores. Then, τ_i,j is varied between the largest and smallest computed confidence values by first sorting the confidence values and setting the threshold to be half-way between every pair of consecutive sorted confidence values. At each of these locations, the classification accuracy of the training data is computed. The threshold is set at the value with the maximum classification accuracy on the training data.

 
_____________________________________________________________________________________________________________ 
Procedure 1 One-vs-one MI-ACE classification 
_____________________________________________________________________________________________________________ 
  1: Train two MI-ACE classifiers for each pair of class labels: 
 Input: X, Y, L 
  2: for Every pair of classes c1 = 1 : C and c2 = 1 : C where c2 ⁄= c1   do 
  3:     Train MI-ACE: (sc1,c2,τc1,c2,μc2,Σc2) = MI-ACE(XL=c1,XL=c2) 
  4:     Train MI-ACE: (sc2,c1,τc2,c1,μc1,Σc1) = MI-ACE(XL=c2,XL=c1) 
  5: end for 
  6: Test using a one-vs-one voting scheme: 
  7: for Every test bag m = 1 : M do 
  8:     for Every pair of classes c1 = 1 : C and c2 = 1 : C where c2 ⁄= c1   do 
  9:         for Every data point, n = 1 : Nm in bag m do 
 10:              Apply the c1 vs. c2 classifier: zm,n,c1,c2  = ACE(Ym,n,sc1,c2) 
 11:         end for 
 12:         Average the confidence scores over all points in the bag: zm,c1,c2  =  1_N ∑Nm n=1 zm,n,c1,c2 
 13:         if zm,c1,c2  > τc1,c2  then 
 14:             Rm,c1,c2  gets one vote for class c1 
 15:         else 
 16:             Rm,c1,c2  gets one vote for class c2 
 17:         end if 
 18:      end for 
 19:      Rm is assigned to the class with the largest number of votes. 
 20:   end for 
 Output: R 
_______________________________________________________________________________________________________________________    

Data description

The training data released by Marconi et al. (2018) contains the hyperspectral signatures from 305 tree crowns collected over the Ordway-Swisher Biological Station (OSBS) by the National Ecological Observatory Network (NEON). The data from NEON included the following data products: (1) field data from years 2015-2017: Woody plant vegetation structure (NEON.DP1.10098); (2) hyperspectral data from year 2014: Spectrometer orthorectified surface directional reflectance - flightline (NEON.DP1.30008); (3) LiDAR point cloud data from year 2014: Ecosystem structure (NEON.DP3.30015); and RGB data from year 2014: High-resolution orthorectified camera imagery (NEON.DP1.30010). In this study, only hyperspectral data are used.

Figure 3 shows a region in OSBS containing various tree species from one NEON flight path. The left image in Fig. 3 is the high resolution RGB image collected from NEON AOP with a 0.25 m resolution. The middle image in Fig. 3 shows the maximum height of the canopy top from LiDAR point cloud with a 1 m resolution. The right image is the RGB bands (wavelength 668 nm, 548 nm, 473 nm for red, green and blue) extracted from hyperspectral image from NEON AOP using NEON imaging spectrometer with a 1 m spatial resolution and 5 nm spectral resolution. The training tree crowns in this challenge were extracted by the organizer from a series of regions similar to Fig. 3 from several flight paths. However, for this tree crown classification challenge, only the individual spectral signature of each pixel in each tree crown are stored and provided. Each tree crown usually contains dozens of pixels. Thus, no spatial information is given nor can any image processing approach be applied. Each training tree crown is associated with a genus and species label. The tree crown is a manually selected image patch which may contain, not only the labeled tree pixels, but also some noise pixels or pixels corresponding to sand, neighboring trees or grass growing underneath the labeled tree. Therefore, the given crown level label is imprecise for the individual pixels of which it is composed. The signatures provided contain 426 spectral bands ranging from 383 nm to 2,512 nm. Water absorption wavelengths, of which reflectance are set to be 1.5 as shown in Fig. 4, correspond to 1,345 nm to 1,430 nm, 1,800 nm to 1,0956 nm and 2,482 nm to 2,512 nm. Each training tree crown is paired with a genus class label and a species class label. The genus consisted of 5 classes which are Acer (AC), Liquidambar (LI), Pinus (PI), Quercus (QU) and OTHERS (OT). OTHERS represent the tree crowns that cannot be classified to any one of the four known genera. Each genus has a different number of associated species. AC and LI contains only one species, which are Acer rubrum (ACRU) and Liquidambar styraciflua (LIST), respectively. PI and QU contain more than three species individually, which are Pinus elliottii (PIEL), Pinus palustris (PIPA), Pinus taeda (PITA) and OTHERS (for PI), Quercus geminata (QUGE), Quercus laevis (QULA), Quercus nigra (QUNI) and OTHERS (for QU). The number of tree polygons for each species are shown in Table 1. There are both prevalent species, such as PIPA and QULA, and rare species, such as ACRU and LIST in the dataset, which make the competition more challenging. In the current implementation of this work, OTHERS in both the genus and species level are not used for training as the proposed approach did not have a mechanism to identify points that did not belong to any of the labeled training classes.

The testing data are also NEON tree crown hyperspectral data in the same format. There were 126 testing tree crowns. The test labels were not provided by the competition organizers.

Data preprocessing and MI-ACE training

Prior to application of the MI-ACE algorithm, the water bands were removed to minimize the influence of noise and reduce dimensionality of the dataset. There were in total 55 water bands that are removed, correspond to wavelength of 1,345 nm to 1,430 nm, 1,800 nm to 1,956 nm and 2,482 nm to 2,512 nm. Then, since in our current implementation data points labeled as OTHERS genus or species are not addressed, the signatures that were labeled as OTHERS are removed from the training set.

After removal of the water bands and the OTHERS data points, the target signatures and classification threshold values are trained using the one-vs-one MI-ACE classification technique. Training was conducted at two levels, the genus and the species levels. During the training phase, each training tree crown was considered a bag for MI-ACE, thus the training label (genus or species level) was the bag label. A one-vs-one MI-ACE was used in which a set of MI-ACE target signatures representing the difference between every two genera or species were estimated. For instance, a target signature was trained to distinguish between the genus PI and the genus QU where tree crown labeled as PI was labeled target (or ‘1’) and QU was labeled as non-target (or ‘0’). For this competition only one MI-ACE classifier was trained for each pair of classes (as opposed to two for each pair) because results were similar between the two approaches. Similarly, a set of target signatures were estimated between every two species (if there were at least two species) that belonged to the same genus. For example, a target signature was estimated using training data to distinguish between species PIEL and PITA where the tree crowns labeled as PIEL were labeled as target (or ‘1’) and PITA was labeled as non-target (or ‘0’).

Testing using ACE detector and voting

Testing was also conducted in two stages where a test tree crown was first classified at the genus level and, then, further classified at the species level. An ACE detector was used to estimate the confidence value indicating how similar a test signature is to a trained target signature. Classification of test tree crowns consisted of the following steps. First, the confidence values of each instance signature inside the a test tree crown were computed using each of the six genus-level trained MI-ACE classifiers. Second, the confidence value for each testing crown was estimated by taking the average value over all of the instance-level confidence values. These average confidence values were then thresholded using the trained threshold values to obtain a binary classification result. The classification thresholds were determined during training to minimize the number of misclassified tree crowns in the training data. The final classification of a tree crown is the class label with the highest number of corresponding binary classifications. After the genus level classification, a test tree crown can be classified at species level using the same approach among the species associated with the genus to which the tree crown assigned.

There are three types of experimental validation methods used in this study: test-on-train, test-on-test and cross validation. Suppose the training dataset and testing dataset are X, Y, respectively. Test-on-train experiments represent training the model using X and testing also using X. Test-on-test experiments represent training the model using X and testing using Y. Cross validation experiments represent training the model using d₁ and validate using d₂, then train using d₂ and validate using d₁, where X = {d₁, d₂}, d₁ and d₂ are the subsets of the training data X. Test-on-train and cross-validation experiments are included since groundtruth labels for the test data have not been released by competition organizers.

Evaluation metrics

The metrics used by competition organizers to evaluate performance are rank-1 accuracy, cross entropy, accuracy and specificity score, F1 score, precision and recall. The rank-1 accuracy is the fraction of crowns whose ground truth class is assigned the highest probability by the proposed algorithm. Intuitively, rank-1 accuracy only consider the percentage of crowns that are correctly predicted (assigned with highest probability). The rank-1 accuracy, r, is defined as: (6)${\displaystyle r=\frac{1}{N}\sum _{i=1}^N\delta \left(arg\underset{k}{max}\left(p_{i,k}\right),g_i\right)}$ where g_i is the ground truth class of crown i and p_i,k is the probability assigned that crown i belongs to class k. The δ(x, y) is an indicator function that takes value 1 if x = y and 0 otherwise. The cross entropy, c, is defined as: (7)${\displaystyle c=-\frac{\sum _{i,k}\delta \left(k,g_i\right)ln{p}_{i,k}}{N}}$ where g_i is the ground truth class of crown i and p_i,k is the probability assigned that crown i belongs to class k. The δ(x, y) is an indicator function that takes value 1 if x = y and 0 otherwise. The accuracy for kth class, a_k, is defined as: (8)${\displaystyle a_k=\frac{T{P}_k+T{N}_k}{T{P}_k+T{N}_k+F{P}_k+F{N}_k}}$ where TP_k, TN_k, FP_k and FN_k represents the true positive, true negative, false positive and false negative for class k, respectively. The specificity for kth class, s_k, is defined as: (9)${\displaystyle s_k=\frac{T{N}_k}{T{N}_k+F{P}_k}}$ where TN_k and FP_k represents the true negative and false positive for class k, respectively. The precision for kth class, p_k, is defined as: (10)${\displaystyle p_k=\frac{T{P}_k}{T{P}_k+F{P}_k}}$ where TP_k and FP_k represents the true positive and false positive for class k, respectively. The recall for kth class, e_k, is defined as: (11)${\displaystyle e_k=\frac{T{P}_k}{T{P}_k+F{N}_k}}$ where TP_k and FN_k represents the true positive and false negative for class k, respectively. The F-1 score for kth class, f_k, is defined as: (12)${\displaystyle f_k=2\frac{p_k{e}_k}{p_k+e_k}}$ where p_k and e_k represents the precision and recall for class k, respectively.

Genus level classification

Classification result when testing on training samples are shown via confusion matrix in Tables 2 and 3 showing results where one classifier is trained for each pair of species classes and two classifiers are trained for each pair of species classes, respectively. The overall classification accuracy on the training dataset is 97.31% for both the one classifier and two classifiers cases. The pixel confidence distributions and the aggregated crown confidence distributions for each classifier are shown in Figs. 5 and 6 respectively. In Figs. 5 and 6, each subfigure represents the confidence distribution of tree crowns from one ground truth genus type estimated by one of the six classifiers . The associated threshold value for each classifier is shown as a vertical blue line in each subfigure. For instance, the subfigure A in Fig. 5 shows the averaged ACE confidence value distribution of AC tree crowns detected using AC-vs-LI classifier and the subfigure A in Fig. 6 shows the corresponding pixel confidence value distributions. A high classification accuracy result for the AC-vs-LI classifier will have all AC confidence values to right of the threshold value and all LI confidence values (shown in the subfigure B) to the left of the threshold value. As can be seen, the AC-vs-LI classifier accurately distinguishes between these two classes on the aggregated crown-level scale. However, when considering the same plots in Fig. 6 for the pixel level confidences, we can see that there are many AC pixels to the left of the threshold causing significant overlap with the LI pixels confidences. This indicates that the aggregation procedure helps to improve results.

Since there were six classifiers trained and four classes in this data set, there are only a small set of possible voting cases. These cases are: a. (3,1,1,1) votes for each class; b. (3,2,1,0) votes for each class; c. (2,2,1,1) votes for each class and d. (2,2,2,0) votes for each class. For cases a. and b., there is a single class with the largest amount of votes, thus, labeling of the crown is straightforward. However, for voting cases c. and d., there are ties among 2 or 3 candidate classes. In our current implementation, we randomly assign the label of one of the tied classes. We found that cases c. and d. are rare in our training and testing results. In the testing on training data results, votes for all of the tree crowns fell into either case a. or b. When applying the trained approach to the testing dataset provided by the competition, there is only one tree crown in which there was a tie (and resulted in voting case is d). For this tree crown, we found that it has two votes for AC, two votes for LI and two votes for PI. Our implementation in test randomly assigned this tree crown to the AC class. Since the ground truth labels of testing data are not released by organizer, the true genus class of this testing tree crown is unknown. However, we found that in our classification results on testing data, there are three tree crowns that were predicted to be in class AC by our method. Two of these crowns were correctly classified into class AC and the other false positive tree crown actually belonged to the OTHER class. Thus, it is likely that this tree crown may be the tree crown with the tied result.

Cross validation studies on the training data were also conducted. There are a limited number of training tree crowns for the AC and LI classes (only six AC and four LI tree crowns). Due to this reason, cross validation experiments were not conducted for AC or LI. In the training phase, the PI training (pixel-level) samples and QU (pixel-level) training samples are considered target and background, respectively. The learned target signature is shown in Fig. 7A, which characterizes the spectral difference between PI and QU shown in Fig. 7B. In the testing phase, each pixel signature of PI and QU are compared with estimated target signature using the ACE detector resulting in a confidence value shown in Fig. 8. Since most confidence values of PI pixels are larger than QU pixels, a threshold value (of 0.05) can be selected such that the misclassified error is minimized.

Two-fold cross validation was applied to the PI and QU samples. The PI and QU classes were randomly split into two datasets (50% of PI and QU tree crowns are selected as d₁ and the rest as d₂). We train on d₁ and validate on d₂, followed by training on d₂ and validating on d₁. After training the PI-vs-QU target signature and corresponding threshold, the histogram of the average confidence values on the validation set is shown in Fig. 9. In this figure, the PI and QU tree crowns are colored by their ground truth classes, i.e., red for PI and blue for QU. The threshold value estimated from training can be directly applied to the validation set for classification of the validation training crowns.

The cross validation experiment was repeated ten times and the mean confusion matrix is shown in Tables 4 and 5 where one classifier is trained for each pair of species classes and two classifiers are trained for each pair of species classes, respectively. The average classification accuracy on the PI and QU given two-fold cross validation dataset was 95.8% and 96.78% for the one classifier and two classifiers cases, respectively, which is similar to the test-on-train accuracy indicating robust results. From the genus level classification, training two classifiers for each pair of classes can achieve classification accuracy slightly higher or equal to the training one classifier for each pair of classes.

Species level classification

After the genus level classification, the tree crowns were further classified into species. If a tree is classified as AC or LI, it is classified also as ACRU or LIST automatically. If a tree is classified as PI or QU, the one-vs-one MI-ACE method is used to classify it into one of the corresponding species. The confusion matrices for species level classification (testing on training data) are shown in Table 6. The classification (rank-1) accuracy is 95.62% on the training dataset in species level with a cross entropy value of 0.2649.

The confusion matrices for species level classification for testing data are shown in Fig. 10 as provided by the competition organizers. The classification (rank-1) accuracy is 86.40% and cross entropy is 0.9395 on the testing dataset. However, this accuracy includes data points labeled as OTHERS in the testing dataset which, using our approach, were all misclassified to one of the four genus types since we did not implement a mechanism to distinguish outliers in this approach. If the OTHERS tree crowns (three tree crowns) are excluded, the classification accuracy would come to 88.52% and cross entropy would be 0.7918 on the testing dataset.

The species level classification results are further evaluated using several metrics on the testing data by the organizer, including per-class accuracy, specificity, precision, recall and F1 score. For comparison, we also evaluated the classification performance using the same metrics on the training data. The accuracy and specificity score, F1 score, precision, recall for both training and testing dataset are shown in Figs. 11–14, respectively. As can be seen, accuracy and specificity results between training and testing data are similar whereas the F1, precision and recall curves highlight that challenging classes in the testing data were PIEL, PITA and QUGE.

Discussions

In the current implementation, only crisp binary classification results are estimated. However, competition organizers evaluated results using the cross entropy evaluation metric assuming probabilities of belonging to each class are estimated, following Eq. (7). Class probabilities given the one-vs-one scheme can be estimated in the future using approaches such as those proposed by Wu, Lin & Weng (2004). Furthermore, even if individual probabilities per data are not computed, an overall uncertainty value can be estimated from the training data. In other words, as opposed to assigning 0-1 probabilities for the crisp class labels. In our implementation data points were assigned to the estimated class label with probability 1 and all others with probability 0. Instead, we could pre-compute an optimal epsilon value, ϵ^∗, to add to the ‘0’ probabilities and subtract from the ‘1’ probabilities to ensure values sum to one across classes to minimize cross entropy on the training data. For instance, we found that when ϵ^∗ = 0.017, the cross entropy for our results comes to 0.68, which is a smaller (i.e., better) than the cross entropy of 0.94 obtained using crisp labels and calculated by the competition organizers. The relationship between the cross entropy and epsilon value for the training data provided shown in Fig. 15.

In addition, in the current proposed framework, each one-vs-one classifier is equally weighted in final voting. However, the classification accuracies and the applicability of each classifier varies. For instance, if a tree crown is in class PI, the PI-vs-QU classifier should be more heavily weighted than the AC-vs-LI classifier. Investigation into whether this could be determined by considering the average confidence values estimated from the individual ACE detectors is needed. Furthermore, since some of the classes are more spectrally distinct, some one-vs-one classifiers have better prediction performances. One possible solution is to weight the classifiers based on the difference between the average confidence values of target and background classes for training data. Another possible solution is to weight based on the difference between the confidence value of testing point and threshold value of the classifier. In some scenarios, these two solutions might be equivalent. Finally, data fusion is also a promising approach for boosting the classification performance. For example, height information from Lidar data could be also incorporated into the training phase since different species generally have different average heights. In the current implementation, only hyperspectral information was leveraged.