article

Extraction of fuzzy rules from fuzzy decision trees: An axiomatic fuzzy sets (AFS) approach

Authors:

Witold PedryczAuthors Info & Claims

Data & Knowledge Engineering, Volume 84

Pages 1 - 25

https://doi.org/10.1016/j.datak.2012.12.001

Published: 01 March 2013 Publication History

Abstract

In this study, we introduce a new type of coherence membership function to describe fuzzy concepts, which builds upon the theoretical findings of the Axiomatic Fuzzy Set (AFS) theory. This type of membership function embraces both the factor of fuzziness (by capturing subjective imprecision) and randomness (by referring to the objective uncertainty) and treats both of them in a consistent manner. Furthermore we propose a method to construct a fuzzy rule-based classifier using coherence membership functions. Given the theoretical developments presented there, the resulting classification systems are referred to as AFS classifiers. The proposed algorithm consists of three major steps: (a) generating fuzzy decision trees by assuming some level of specificity (detailed view) quantified in terms of threshold; (b) pruning the obtained rule-base; and (c) determining the optimal threshold resulting in a final tree. Compared with other fuzzy classifiers, the AFS classifier exhibits several essential advantages being of practical relevance. In particular, the relevance of classification results is quantified by associated confidence levels. Furthermore the proposed algorithm can be applied to data sets with mixed data type attributes. We have experimented with various data commonly present in the literature and compared the results with that of SVM, KNN, C4.5, Fuzzy Decision Trees (FDTs), Fuzzy SLIQ Decision Tree (FS-DT), FARC-HD and FURIA. It has been shown that the accuracy is higher than that being obtained by other methods. The results of statistical tests supporting comparative analysis show that the proposed algorithm performs significantly better than FDTs, FS-DT, KNN and C4.5.

References

[1]

Ichihashi, H., Shirai, T., Nagasaka, K. and Miyoshi, T., Neuro fuzzy ID3: a method of inducing fuzzy decision trees with linear programming for maximizing entropy and algebraic methods. Fuzzy Sets and Systems. v81 i1. 157-167.

[2]

Chandra, B. and Varghese, P.P., Fuzzifying gini index based decision trees. Expert Systems with Applications. v36. 8549-8559.

Digital Library

[3]

Serrurier, M., Dubois, D., Prade, H. and Sudkamp, T., Learning fuzzy rules with their implication operators. Data and Knowledge Engineering. v60. 71-89.

Digital Library

[4]

Ghazisaidi, N., Assi, C.M. and Maier, M., Intelligent wireless mesh path selection algorithm using fuzzy decision making. Wireless Networks. v18 i2. 129-146.

[5]

Cano, J.R., Herrera, F. and Lozano, M., Evolutionary stratified training set selection for extracting classification rules with trade off precision-interpretability. Data and Knowledge Engineering. v60. 90-108.

Digital Library

[6]

Jiang, X.R. and Gruenwald, L., Microarray gene expression data association rules mining based on BSC-tree and FIS-tree. Data and Knowledge Engineering. v53. 3-29.

[7]

Weber, R., Fuzzy-ID3: a class of methods for automatic knowledge acquisition. In: 2nd International Conf. on Fuzzy Logic and Neural Networks, Lizuka, Japan, pp. 265-268.

[8]

Xin, W., Liu, X.D., Pedryczc, W., Zhu, X.L. and Hu, G.F., Mining axiomatic fuzzy set association rules for classification problems. European Journal of Operational Research. v218 i1. 202-210.

[9]

Andreou, A.S. and Papatheocharous, E., Software cost estimation using fuzzy decision trees. In: ASE '08, pp. 371-374.

Digital Library

[10]

Chen, C.L., Tseng, F.S.C. and Liang, T., An integration of wordnet and fuzzy association rule mining for multi-label document clustering. Data and Knowledge Engineering. v69 i11. 1208-1226.

[11]

Xu, Y., Li, Y.F. and Shaw, G., Reliable representations for association rules. Data and Knowledge Engineering. v70 i6. 555-575.

[12]

Yeung, D.S., Wang, X.Z. and Tsang, E.C.C., Learning weighted fuzzy rules from examples with mixed attributes by fuzzy decision trees. In: IEEE Int. Conf. on Systems, Man, and Cybernetics, Tokyo, Japan, pp. 349-354.

[13]

Pedrycz, W. and Sosnowski, Z.A., C-fuzzy decision trees. IEEE Transactions on Systems, Man, and Cybernetics - Part B: Applications and Reviews. v35 i4. 498-511.

[14]

Liu, X.D. and Pedrycz, W., The development of fuzzy decision trees in the framework of axiomatic fuzzy set logic. Applied Soft Computing. v7. 325-342.

Digital Library

[15]

Janikow, C.Z., Fuzzy decision trees: issues and methods. IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics. v28 i1. 1-14.

Digital Library

[16]

Wang, X.Z., Yeung, D.S. and Tsang, E.C.C., A comparative study on heuristic algorithms for generating fuzzy decision trees. IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics. v31 i2. 215-226.

[17]

Chandra, B. and Varghese, P.P., Fuzzy SLIQ decision tree algorithm. IEEE Transaction on Systems, Man, Cybernetics - Part B: Cybernetics. v38 i5. 1294-1301.

[18]

Yuan, Y. and Shaw, M.J., Induction of fuzzy decision trees. Fuzzy Sets and Systems. v69. 125-139.

Digital Library

[19]

Olaru, C. and Wehenkel, L., A complete fuzzy decision tree technique. Fuzzy Sets and Systems. v138 i2. 221-254.

[20]

Lertworaprachaya, Y., Yang, Y.J. and John, R., Interval-valued fuzzy decision trees. In: IEEE Int. Conf. on Fuzzy Systems, Barcelona, Spain, pp. 1-7.

[21]

Boyen, X. and Wehenkel, L., Automatic induction of fuzzy decision trees and its application to power system security assessment. Fuzzy Sets and Systems. v102. 3-19.

Digital Library

[22]

Boyen, X. and Wehenkel, L., Fuzzy decision tree induction for power system security assessment. In: Proc. SIPOWER '95, 2nd IFAC Symp. on Control of Power Plants and Power Systems, Mexico, pp. 151-156.

[23]

Hayashi, I., Maeda, T., Bastian, A. and Jain, L.C., Generation of fuzzy decision trees by fuzzy ID3 with adjusting mechanism of and/or operators. In: Int. Conf. Fuzzy Syst., pp. 681-685.

[24]

Boyen, X. and Wehenkel, L., Automatic induction of continuous decision trees. In: Proc. IPMU '96, Information Processing and Management of Uncertainty in Knowledge-Based Systems, Granada, pp. 419-424.

[25]

Smith, J.F., Evolving fuzzy decision tree structure that adapts in real-time. In: GECCO '05, pp. 1737-1744.

[26]

Moustakidis, S., Mallinis, G., Koutsias, N., Theocharis, J.B. and Petridis, V., SVM-based fuzzy decision trees for classification of high spatial resolution remote sensing images. IEEE Transactions on Geoscience and Remote Sensing. v50 i1. 149-169.

[27]

Umanol, M., Okamoto, H., Hatono, I., Tamura, H., Kawachi, F., Umedzu, S. and Kinoshita, J., Fuzzy decision trees by fuzzy ID3 algorithm and its application to diagnosis systems. In: IEEE Conference on Fuzzy Systems, Orlando, USA, pp. 2113-2118.

[28]

A new partition criterion for fuzzy decision tree algorithm. In: Intelligent Information Technology Application (Work shop), pp. 43-46.

[29]

Chen, R.Y., Sheu, D.D. and Liu, C.M., Vague knowledge search in the design for outsourcing using fuzzy decision tree. Computers and Operations Research. v34. 3628-3637.

Digital Library

[30]

Quinlan, J.R., Induction of decision trees. Machine Learning. v1. 81-106.

[31]

Abu-halaweh, N.M. and Harrison, R.W., Practical fuzzy decision trees. In: IEEE Symposium on CIDM '09, Nashville, USA, pp. 211-216.

[32]

Meng, Q.W., He, Q., Li, N., Du, X.R. and Su, L.N., Crisp decision tree induction based on fuzzy decision tree algorithm. In: ICISE, pp. 4811-4814.

Digital Library

[33]

Breiman, L., Friedman, J., Olshen, R. and Stone, C., Classification and Regression Trees. 1984. Wadsworth, Belmont, CA.

[34]

Jang, J.S.R., Structure determination in fuzzy modeling a fuzzy CART approach. In: IEEE Conference on Fuzzy Systems, Orlando, USA, pp. 480-485.

[35]

Chandra, B., Mazumdar, S., Arena, V. and Parimi, N., Elegant decision tree algorithm for classification in data mining. In: Proceedings of the 3rd International Conference on Information Systems Engineering (work shops), pp. 160-169.

[36]

Chandra, B. and Varghese, P.P., A robust algorithm for classification using decision trees. In: IEEE Conference on Cybernetics and Intelligent Systems, pp. 1-5.

[37]

Pedrycz, W. and Sosnowski, Z.A., Designing decision trees with the use of fuzzy granulation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans. v30 i2. 151-159.

[38]

Fowdar, J., Crockett, K., Bandar, Z. and O'Shea, J., On the use of fuzzy trees for solving classification problems with numeric outcomes. In: IEEE Conference on Fuzzy Systems, pp. 436-441.

[39]

Constructing a multi-valued and multi-labeled decision tree. Expert Systems with Applications. v25. 199-209.

[40]

Apolloni, B., Zamponi, G. and Zanaboni, A.M., Learning fuzzy decision trees. Neural Networks. v11. 885-895.

Digital Library

[41]

Wang, X.Z., Zhai, J.H. and Lu, S.X., Induction of multiple fuzzy decision trees based on rough set technique. Information Sciences. v178. 3188-3202.

Digital Library

[42]

Cheng, Y., The incremental method for fast computing the rough fuzzy approximations. Data and Knowledge Engineering. v70 i1. 84-100.

[43]

Merz, C.J. and Murphy, P.M., UCI Repository for Machine Learning Data-Bases. 1996. Dept. of Information and Computer Science, University of California, Irvine, CA.

[44]

Platt, J., Fast Training of Support Vector Machines Using Sequential Minimal Optimization, Advances in Kernel Methods: Support Vector Learning. 1998. MIT Press.

[45]

Aha, D.W., Kibler, D. and Albert, M.K., Instance-based learning algorithms. Machine Learning. v6 i1. 37-66.

[46]

Quinlan, J.R., C4.5: Programs for Machine Learning. 1993. Morgan Kaufmann, San Mateo.

[47]

Alcala-Fdez, J., Alcala, R. and Herrera, F., A fuzzy association rule-based classification model for high-dimensional problems with genetic rule selection and lateral tuning. IEEE Transactions on Fuzzy Systems. v19 i5. 857-872.

[48]

Huhn, J.C. and Hullermeier, E., FURIA: an algorithm for unordered fuzzy rule induction. Data Mining and Knowledge Discovery. v19. 293-319.

[49]

Liu, X.D., Chai, T.Y., Wang, W. and Liu, W.Q., Approaches to the representations and logic operations for fuzzy concepts in the framework of axiomatic fuzzy set theory ¿. Information Sciences. v177. 1007-1026.

Digital Library

[50]

Liu, X.D., Pedrycz, W., Chai, T.Y. and Song, M.L., The development of fuzzy rough sets with the use of structures and algebras of axiomatic fuzzy sets. IEEE Transactions on Knowledge and Data Engineering. v21 i3. 443-462.

[51]

The fuzzy clustering analysis based on AFS theory. IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics. v35 i5. 1013-1027.

[52]

Liu, X.D., The fuzzy theory based on AFS algebras and AFS structure. Journal of Mathematical Analysis and Applications. v217. 459-478.

[53]

Witten, I.H. and Frank, E., Data Mining: Practical Machine Learning Tools and Techniques. 2005. 2nd ed. Morgan Kaufmann, San Mateo, CA.

[54]

Alcala-Fdez, J., Sanchez, L., Garcia, S., Jesus, M., Ventura, S., Garrell, J., Otero, J., Romero, C., Bacardit, J., Rivas, V., Fernandez, J. and Herrera, F., KEEL: a software tool to assess evolutionary algorithms to data mining problems. Soft Computing. v13 i3. 307-318.

[55]

Demsar, J., Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research. v7. 1-30.

Digital Library

[56]

Huhn, J.C. and Hullermeier, E., FR3: a fuzzy rule learner for Inducing reliable classifiers. IEEE Transactions on Fuzzy Systems. v17 i1. 138-139.

[57]

Ishibuchi, H. and Nakashima, T., Effect of rule weights in fuzzy rule-based classification systems. IEEE Transactions on Fuzzy Systems. v9 i4. 506-515.

[58]

Ross, T.J., Fuzzy Logic with Engineering Applications. 1995. McGraw-Hill, New York.

[59]

Kelly, W.E. and Painter, J.H., Hypertrapezoidal fuzzy membership functions. In: IEEE International Conference on Fuzzy Systems, pp. 1279-1284.

[60]

Halmos, P.R., Measure Theory. 1974. Springer-Verlag, New York.

[61]

Simono, J.S., Smoothing Methods in Statistics. 1996. Springer-Verlag, New York.

Cited By

Wang KZhou HCai JLi WYao J(2023)Time-domain adaptive attention network for single-channel speech separationEURASIP Journal on Audio, Speech, and Music Processing10.1186/s13636-023-00283-w2023:1Online publication date: 11-May-2023
https://dl.acm.org/doi/10.1186/s13636-023-00283-w
Porebski S(2022)Evaluation of fuzzy membership functions for linguistic rule-based classifier focused on explainability, interpretability and reliabilityExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.117116199:COnline publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1016/j.eswa.2022.117116
Mu YWang LLiu X(2020)Dynamic programming based fuzzy partition in fuzzy decision tree inductionJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-19149739:5(6757-6772)Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.3233/JIFS-191497
Show More Cited By

Extraction of fuzzy rules from fuzzy decision trees: An axiomatic fuzzy sets (AFS) approach
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches

Recommendations

The development of fuzzy decision trees in the framework of Axiomatic Fuzzy Set logic
Abstract
Decision trees are one among interesting and commonly encountered architectures used for learning, reasoning and organization of datasets. This study being positioned in the realm of decision trees is aimed at two main objectives. ...
Fuzzy Interpolative Reasoning for Sparse Fuzzy-Rule-Based Systems Based on the Areas of Fuzzy Sets

Fuzzy interpolative reasoning is an inference technique for dealing with the sparse rules problem in sparse fuzzy-rule-based systems. In this paper, we present a new fuzzy interpolative reasoning method for sparse fuzzy-rule-based systems based on the ...
Fuzzy interpolative reasoning based on ranking values of polygonal fuzzy sets and automatically generated weights of fuzzy rules

In this paper, we propose a new fuzzy interpolative reasoning method for sparse fuzzy rule-based systems based on ranking values of polygonal fuzzy sets and automatically generated weights of fuzzy rules. First, the proposed method gets the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Data & Knowledge Engineering

Data & Knowledge Engineering Volume 84, Issue

March, 2013

81 pages

ISSN:0169-023X

Issue’s Table of Contents

Copyright © Elsevier B.V. © 2012.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 March 2013

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang KZhou HCai JLi WYao J(2023)Time-domain adaptive attention network for single-channel speech separationEURASIP Journal on Audio, Speech, and Music Processing10.1186/s13636-023-00283-w2023:1Online publication date: 11-May-2023
https://dl.acm.org/doi/10.1186/s13636-023-00283-w
Porebski S(2022)Evaluation of fuzzy membership functions for linguistic rule-based classifier focused on explainability, interpretability and reliabilityExpert Systems with Applications: An International Journal10.1016/j.eswa.2022.117116199:COnline publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1016/j.eswa.2022.117116
Mu YWang LLiu X(2020)Dynamic programming based fuzzy partition in fuzzy decision tree inductionJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-19149739:5(6757-6772)Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.3233/JIFS-191497
Cai YZhang HHe QDuan J(2020)A novel framework of fuzzy oblique decision tree construction for pattern classificationApplied Intelligence10.1007/s10489-020-01675-750:9(2959-2975)Online publication date: 1-Sep-2020
https://dl.acm.org/doi/10.1007/s10489-020-01675-7
Cai YZhang HSun SWang XHe Q(2020)Axiomatic fuzzy set theory-based fuzzy oblique decision tree with dynamic mining fuzzy rulesNeural Computing and Applications10.1007/s00521-019-04649-032:15(11621-11636)Online publication date: 1-Aug-2020
https://dl.acm.org/doi/10.1007/s00521-019-04649-0
Mutlu BSezer E(2019)A New Fuzzy Rule Generation Scheme based on Multiple-Selection of Influencing Factors2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)10.1109/FUZZ-IEEE.2019.8858990(1-6)Online publication date: 23-Jun-2019
https://dl.acm.org/doi/10.1109/FUZZ-IEEE.2019.8858990
Duan XWang YPedrycz WLiu XWang CLi Z(2018)AFSNN: A Classification Algorithm Using Axiomatic Fuzzy Sets and Neural NetworksIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2017.278887526:5(3151-3163)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1109/TFUZZ.2017.2788875
Segatori AMarcelloni FPedrycz W(2018)On Distributed Fuzzy Decision Trees for Big DataIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2016.264674626:1(174-192)Online publication date: 1-Feb-2018
https://dl.acm.org/doi/10.1109/TFUZZ.2016.2646746
Mutlu BSezer EAli Akcayol M(2018)End-to-End Hierarchical Fuzzy Inference Solution2018 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)10.1109/FUZZ-IEEE.2018.8491481(1-9)Online publication date: 8-Jul-2018
https://dl.acm.org/doi/10.1109/FUZZ-IEEE.2018.8491481
Porebski SStraszecka E(2018)Extracting easily interpreted diagnostic rulesInformation Sciences: an International Journal10.1016/j.ins.2017.10.034426:C(19-37)Online publication date: 1-Feb-2018
https://dl.acm.org/doi/10.1016/j.ins.2017.10.034
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents