research-article

Human-In-The-Loop Construction of Decision Tree Classifiers with Parallel Coordinates

Authors:

Vladimir Estivill-Castro,

Eugene Gilmore,

René HexelAuthors Info & Claims

2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)

Pages 3852 - 3859

https://doi.org/10.1109/SMC42975.2020.9283240

Published: 11 October 2020 Publication History

Abstract

How can there be Human-In-the-Loop-Learning (HILL) if datasets aimed at building classifiers have ever more dimensions? We make two contributions. First, we examine the few early results on the effectiveness of HILL for building autonomous classifiers and report on our own experiment that validates the merits of HILL. Second, we introduce a HILL system (by using parallel coordinates) for learning of decision tree classifiers (DTCs). DTCs importantly emphasise the relevance of attributes and enable attribute selection, and therefore are appreciated for their transparency. The proposed system addresses a number of the shortcomings of the many HILL systems and allows for easy exploration of datasets. In particular, we incorporate parallel coordinates effectively in our tool for visualisation of high dimensional datasets. We can not only focus the learning on the accuracy of classifiers, but we can enhance performance in other important factors such as system’s interpretability and the ability to gain insight into datasets. Finally, we show the advantages of our HILL system in the application area of mobile robotics using the case study of image segmentation in robotic soccer.

References

[1]

A. A. Freitas, “Comprehensible classification models: a position paper,” SIGKDD Explorations, vol. 15, no. 1, pp. 1–10, 2013.

Digital Library

[2]

R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D. Pedreschi, “A survey of methods for explaining black box models,” ACM computing surveys (CSUR), vol. 51, no. 5, pp. 1–42, 2018.

Digital Library

[3]

A. Moore, V. Murdock, Y. Cai, and K. Jones, “Transparent tree ensembles,” in The 41st Int. ACM SIGIR Conf. on Research & Development in Information Retrieval, ser. SIGIR ’18. New York, NY, USA: Association for Computing Machinery, 2018, p. 1241–1244.

[4]

Q. V. Liao, M. Singh, Y. Zhang, and R. K. Bellamy, “Introduction to explainable ai,” in Extended Abstracts of the 2020 CHI Conf. on Human Factors in Computing Systems, ser. CHI EA ’20. New York, NY, USA: Association for Computing Machinery, 2020, pp. 1–4.

[5]

W. Samek and K.-R. Müller, “Towards explainable artificial intelligence,” in Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Cham: Springer International Publishing, 2019, pp. 5–22.

[6]

A. Blanco-Justicia and J. Domingo-Ferrer, “Machine learning explainability through comprehensible decision trees,” in Machine Learning and Knowledge Extraction. Springer, 2019, pp. 15–26.

[7]

W. Samek, A. Binder, G. Montavon, S. Lapuschkin, and K. Müller, “Evaluating the visualization of what a deep neural network has learned,” IEEE Trans. on Neural Networks and Learning Systems, vol. 28, no. 11, pp. 2660–2673, 2017.

[8]

H. Lakkaraju, E. Kamar, R. Caruana, and J. Leskovec, “Faithful and customizable explanations of black box models,” in Proc. of the 2019 AAAI/ACM Conf. on AI, Ethics, and Society, ser. AIES ’19. New York, NY, USA: Association for Computing Machinery, 2019, p. 131–138.

[9]

C. Rudin, “Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead,” Nature Machine Intelligence, vol. 1, pp. 206–215, 05 2019.

[10]

J. Huysmans, K. Dejaeger, C. Mues, J. Vanthienen, and B. Baesens, “An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models,” Decision Support Systems, vol. 51, no. 1, pp. 141–154, 2011.

Digital Library

[11]

S. Amershi, M. Cakmak, W. B. Knox, and T. Kulesza, “Power to the people: The role of humans in interactive machine learning,” AI Magazine, vol. 35, no. 4, pp. 105–120, Dec. 2014. [Online]. Available: https://www.aaai.org/ojs/index.php/aimagazine/article/view/2513

[12]

J. A. Fails and D. R. Olsen, “Interactive machine learning,” in Proc. of the 8th Int. Conf. on Intelligent User Interfaces, ser. IUI ’03. New York, NY, USA: Association for Computing Machinery, 2003, p. 39–45.

[13]

J. Quinlan, C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann Publishers, 1993.

Digital Library

[14]

X. Wu, V. Kumar, J. R. Quinlan, J. Ghosh, Q. Yang, H. Motoda, G. McLachlan, A. Ng, B. Liu, P. Yu, Z.-H. Zhou, M. Steinbach, D. Hand, and D. Steinberg, “Top 10 algorithms in data mining,” Knowledge and Information Systems, vol. 14, no. 1, pp. 1–37, January 2008.

Digital Library

[15]

E. Hunt, J. Martin, and P. Stone, Experiments in Induction. New York: Academic Press, 1966.

[16]

P.-N. Tan, M. Steinbach, and V. Kumar, Introduction to Data Mining. Reading, MA: Addison-Wesley Publishing Co., 2006.

Digital Library

[17]

L. Breiman, J. Friedman, R. Olshen, and C. Stone, Classification and Regression Trees. Monterrey, CA: Wadsworth and Brooks, 1984.

[18]

C. Mues, J. Huysmans, J. Vanthienen, and B. Baesens, “Comprehensible credit-scoring knowledge visualization using decision tables and diagrams,” in Enterprise Information Systems VI. Springer Netherlands, 2006, pp. 109–115.

[19]

A. Inselberg, Parallel Coordinates : Visual Multidimensional Geometry and its Applications. NY: Springer, 2009.

[20]

V. Estivill-Castro, “Collaborative knowledge acquisition with a genetic algorithm,” in 9th Int. Conf. on Tools with Artificial Intelligence, ICTAI ’97. Newport Beach, CA, USA: IEEE Computer Society, November 3rd-8th 1997, pp. 270–277.

[21]

M. Ankerst, C. Elsen, M. Ester, and H.-P. Kriegel, “Visual classification: An interactive approach to decision tree construction,” in Proc. of the Fifth ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, ser. KDD ’99. New York, NY, USA: ACM, 1999, pp. 392–396.

[22]

M. Ware, E. Frank, G. Holmes, H. M. A., and I. H. Witten, “Interactive machine learning: letting users build classifiers,” Int. J. Hum.-Comput. Stud., vol. 55, no. 3, pp. 281–292, 2001.

Digital Library

[23]

W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: Single shot multibox detector,” in Computer Vision – ECCV 2016. Cham: Springer International Publishing, 2016, pp. 21–37.

[24]

K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. Xiao, A. Prakash, T. Kohno, and D. Song, “Robust physical-world attacks on deep learning visual classification,” in 2018 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, 2018, pp. 1625–1634.

[25]

A. K. Mackworth, “On seeing robots,” University of British Columbia, Vancouver, BC, Canada, Canada, Tech. Rep., 1993.

[26]

K. Pulli, A. Baksheev, K. Kornyakov, and V. Eruhimov, “Real-time computer vision with OpenCV,” Commun. ACM, vol. 55, no. 6, pp. 61–69, Jun. 2012.

Digital Library

[27]

I. Schwarz, M. Hofmann, O. Urbann, and S. Tasse, “A robust and calibration-free vision system for humanoid soccer robots,” in RoboCup 2015: Robot World Cup XIX. Springer International Publishing, 2015, pp. 239–250.

[28]

D. Albani, A. Youssef, V. Suriani, D. Nardi, and D. D. Bloisi, “A deep learning approach for object recognition with nao soccer robots,” in RoboCup 2016: Robot World Cup XX. Cham: Springer International Publishing, 2017, pp. 392–403.

[29]

M. Javadi, S. M. Azar, S. Azami, S. S. Ghidary, S. Sadeghnejad, and J. Baltes, “Humanoid robot detection using deep learning: A speed-accuracy tradeoff,” in RoboCup International Symposium. Cham: Springer International Publishing, 2018, pp. 338–349.

[30]

S. O’Keeffe and R. Villing, “A benchmark data set and evaluation of deep learning architectures for ball detection in the robocup spl,” in RoboCup Int. Symposium. Cham: Springer International Publishing, 2017, pp. 398–409.

[31]

A. Gabel, T. Heuer, I. Schiering, and R. Gerndt, “Jetson, where is the ball? using neural networks for ball detection at robocup 2017,” in RoboCup 2018: Robot World Cup XXII. Cham: Springer International Publishing, 2019, pp. 181–192.

[32]

D. Speck, P. Barros, C. Weber, and S. Wermter, “Ball localization for robocup soccer using convolutional neural networks,” in RoboCup 2016: Robot World Cup XX. Cham: Springer International Publishing, 2017, pp. 19–30.

[33]

T. Houliston and S. K. Chalup, “Visual mesh: Real-time object detection using constant sample density,” in RoboCup 2018: Robot World Cup XXII [Montreal, QC, Canada, June 18-22, 2018], ser. Lecture Notes in Computer Science, vol. 11374. Springer, 2018, pp. 45–56.

[34]

M. Ankerst, M. Ester, and H.-P. Kriegel, “Towards an effective cooperation of the user and the computer for classification,” in Proc. of the Sixth ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, ser. KDD ’00. New York, NY, USA: ACM, 2000, pp. 179–188.

[35]

A. Inselberg and T. Avidan, “Classification and visualization for high-dimensional data,” in Proc. of the sixth ACM SIGKDD Int. Conf. on Knowledge discovery and data mining. Boston, MA, USA: ACM, August 20th-23rd 2000, pp. 370–374.

[36]

P. L. Lai, Y. J. Liang, and A. Inselberg, “Geometric divide and conquer classification for high-dimensional data,” in DATA 2012 - Proc. of the Int. Conf. on Data Technologies and Applications. SciTePress, July 25th-27th July 2012, pp. 79–82.

[37]

E. Hunt, Concept Learning — An Information Processing Problem, second printing ed. New York: John Wiley, 1962.

[38]

P. R. Cohen and E. A. Feigenbaum, The Handbook of artificial intelligence, volume III. Stanford, CA: HeurisTech Press, 1982.

[39]

A. Inselberg, “Parallel coordinates: Visualization, exploration and classification of high-dimensional data,” 2008, iII.14 Parallel Coordinates: Visualization, Exploration and Classification of High-Dimensional Data.

[40]

S. T. Teoh and K. Ma, “Starclass: Interactive visual classification using star coordinates,” in Proc. of the Third SIAM Int. Conf. on Data Mining, vol. 112. SIAM, 2003, pp. 178–185.

[41]

S. T. Teoh and K.-L. Ma, “Paintingclass: Interactive construction, visualization and exploration of decision trees,” in Proc. of the Ninth ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, ser. KDD ’03. New York, NY, USA: Association for Computing Machinery, 2003, p. 667–672.

[42]

J. Choo, H. Lee, J. Kihm, and H. Park, “ivisclassifier: An interactive visual analytics system for classification based on supervised dimension reduction,” in 2010 IEEE Symposium on Visual Analytics Science and Technology, 2010, pp. 27–34.

[43]

B. A. Mustafa, “An experimental comparison of use case models understanding by novice and high knowledge users,” in New Trends in Software Methodologies, Tools and Techniques - Proc. of the 9th SoMeT 10, ser. Frontiers in Artificial Intelligence and Applications, vol. 217. IOS Press, September 29th - October 1st 2010, pp. 182–199.

[44]

M. Lichman, “UCI machine learning repository,” 2013. [Online]. Available: http://archive.ics.uci.edu/ml

[45]

V. Estivill-Castro, E. Gilmore, and R. Hexel, “Constructing interpretable decision trees using parallel coordinates,” 2020, to appear in, Proc. of the 19th Int. Conf. on Artifical Intelligence and Soft Computing.

[46]

S. K. Murthy, S. Kasif, and S. Salzberg, “A system for induction of oblique decision trees,” J. Artif. Int. Res., vol. 2, no. 1, pp. 1–32, Aug. 1994.

Digital Library

[47]

E. Cantú-Paz and C. Kamath, “Inducing oblique decision trees with evolutionary algorithms,” IEEE Trans. Evolutionary Computation, vol. 7, no. 1, pp. 54–68, 2003.

Digital Library

[48]

N. T. D., T. Ho, and H. Shimodaira, “Interactive visualization in mining large decision trees,” in Knowledge Discovery and Data Mining, Current Issues and New Applications, 4th Pacific-Asia Conf. PADKK 2000, ser. Lecture Notes in Computer Science, vol. 1805. Kyoto, Japan: Springer, April 18th-20th 2000, pp. 345–348.

Cited By

Bianchi MDe Santis ATocchetti ABrambilla MLarson K(2024)Interpretable network visualizationsProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/411(3715-3723)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/411
Good JKovach TMiller KDubrawski AOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Feature learning for interpretable, performant decision treesProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3669029(66571-66582)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3669029
Sarailidis GWagener TPianosi F(2023)Integrating scientific knowledge into machine learning using interactive decision treesComputers & Geosciences10.1016/j.cageo.2022.105248170:COnline publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.cageo.2022.105248

Recommendations

Comparing ensemble learning methods based on decision tree classifiers for protein fold recognition

In this paper, some methods for ensemble learning of protein fold recognition based on a decision tree DT are compared and contrasted against each other over three datasets taken from the literature. According to previously reported studies, the ...
Hybrid decision tree and naïve Bayes classifiers for multi-class classification tasks

In this paper, we introduce two independent hybrid mining algorithms to improve the classification accuracy rates of decision tree (DT) and naive Bayes (NB) classifiers for the classification of multi-class problems. Both DT and NB classifiers are ...
Classification of major construction materials in construction environments using ensemble classifiers

The automatic detection of construction materials in images acquired on a construction site has been regarded as a critical topic. Recently, several data mining techniques have been used as a way to solve the problem of detecting construction materials. ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)

Oct 2020

4507 pages

Copyright © 2020.

Publisher

IEEE Press

Publication History

Published: 11 October 2020

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 30 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bianchi MDe Santis ATocchetti ABrambilla MLarson K(2024)Interpretable network visualizationsProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/411(3715-3723)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/411
Good JKovach TMiller KDubrawski AOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Feature learning for interpretable, performant decision treesProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3669029(66571-66582)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3669029
Sarailidis GWagener TPianosi F(2023)Integrating scientific knowledge into machine learning using interactive decision treesComputers & Geosciences10.1016/j.cageo.2022.105248170:COnline publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.cageo.2022.105248

View Options

View options

Figures

Tables

Media

View Table of Conten