Affiliations: [a] Key Laboratory of Symbolic Computation and Knowledge Engineer, Ministry of Education, Changchun, Jilin, China | [b] College of Computer Science and Technology, Jilin University, Changchun, Jilin, China | [c] State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China | [d] CSIRO Data61, Sydney, NSW, Australia
Correspondence:
[*]
Corresponding author: Xiaoli Zhang, Key Laboratory of Symbolic Computation and Knowledge Engineer, Ministry of Education, Changchun, Jilin 130012, China. E-mail: [email protected].
Abstract: In classification, a decision tree is a common model due to its simple structure and easy understanding. Most of decision tree algorithms assume all instances in a dataset have the same degree of confidence, so they use the same generation and pruning strategies for all training instances. In fact, the instances with greater degree of confidence are more useful than the ones with lower degree of confidence in the same dataset. Therefore, the instances should be treated discriminately according to their corresponding confidence degrees when training classifiers. In this paper, we investigate the impact and significance of degree of confidence of instances on the classification performance of decision tree algorithms, taking the classification and regression tree (CART) algorithm as an example. First, the degree of confidence of instances is quantified from a statistical perspective. Then, a developed CART algorithm named C_CART is proposed by introducing the confidence of instances into the generation and pruning processes of CART algorithm. Finally, we conduct experiments to evaluate the performance of C_CART algorithm. The experimental results show that our C_CART algorithm can significantly improve the generalization performance as well as avoiding the over-fitting problem to a certain extend.
Keywords: Degree of confidence, CART algorithm, generalization, classification, machine learning