Lessons from Past, Current Issues, and Future Research Directions in Extracting the Knowledge Embedded in Artificial Neural Networks

1. Alexander, J.A., Mozer, M.C.: Template-Based Procedures for Neural Network Interpretation. Neural Networks 12, 479–498 (1999)

   Article  Google Scholar 

2. Andrews, R., Diederich, J., Tickle, A.B.: A Survey and Critique of Techniques for Extracting Rules from Trained Artificial Neural Networks. Knowledge-Based Systems 8(6), 373–389 (1995)

   Article  Google Scholar 

3. Andrews, R., Cable, R., Diederich, J., Geva, S., Golea, M., Hayward, R., Ho-Stuart, C., Tickle, A.B.: An Evaluation and Comparison of Techniques for Extracting and Refining Rules from Artificial Neural Networks. Technical report, Queensland University of Technology, Australia (1996)

   Google Scholar 

4. Banerjee, M., Mitra, S., Pal, S.K.: Rough Fuzzy MLP: Knowledge Encoding and Classification. IEEE Transactions on Neural Networks 9(6), 1203–1216 (1998)

   Article  Google Scholar 

5. Bologna, G., Pellegrini, C.: Symbolic Rule Extraction from Modular Transparent Boxes. In: Proceedings of the Conference of Neural Networks and their Applications (NEURAP), pp. 393–398 (1998)

   Google Scholar 

6. Bologna, G., Pellegrini, C.: Rule-Extraction from the Oblique Multi-layer Perceptron. In: Proceedings of the Australian Conference on Neural Networks (ACNN 1998), University of Queensland, Australia, pp. 260–264 (1998)

   Google Scholar 

7. Bologna, G.: Symbolic Rule Extraction from the DIMLP Neural Network. In: Sun, R., Wermter, S. (eds.) Hybrid Neural Systems 1998. LNCS, vol. 1778, pp. 240–254. Springer, Heidelberg (2000)

   Chapter  Google Scholar 

8. Carpenter, G.A., Tan, A.W.: Rule Extraction: From Neural Architecture to Symbolic Representation. Connection Science 7(1), 3–27 (1995)

   Article  Google Scholar 

9. Craven, M., Shavlik, J.W.: Using Sampling and Queries to Extract Rules From Trained Neural Networks. In: Machine Learning: Proceedings of the Eleventh International Conference, pp. 73–80. Morgan-Kaufmann, Amherst MA (1994)

   Google Scholar 

10. Craven, M.: Extracting Comprehensible Models from Trained Neural Networks. PhD Thesis, University of Wisconsin, Madison Wisconsin (1996)

    Google Scholar 

11. Fu, L.M.: Rule Generation from Neural Networks. IEEE Transactions on Systems, Man and Cybernetics 28(8), 1114–1124 (1994)

    Google Scholar 

12. Fu, L.M.: A Neural Network Model for Learning Domain Rules Based on its Activation Function Characteristics. IEEE Transactions on Neural Networks 9(5), 787–795 (1998)

    Article  Google Scholar 

13. Fu, L.M.: A Neural Network for Learning Domain Rules with Precision. In: Proceedings of the International Joint Conference on Neural Networks, IJCNN 1999 (1999) (to appear)

    Google Scholar 

14. Giles, C.L., Miller, C.B., Chen, D., Chen, H., Sun, Z., Lee, Y.C.: Learning and Extracting Finite State Automata with Second-order Recurrent Neural Networks. Neural Computation 4, 393–405 (1992)

    Article  Google Scholar 

15. Giles, C.L., Omlin, C.W.: Rule Refinement with Recurrent Neural Networks. In: Proceedings of the IEEE International Conference on Neural Networks, San Francisco, CA, pp. 801–806 (1993)

    Google Scholar 

16. Giles, C.L., Omlin, C.W.: Extraction, Insertion, and Refinement of Symbolic Rules in Dynamically Driven Recurrent Networks. Connection Science 5(3-4), 307–328 (1993)

    Article  Google Scholar 

17. Giles, C.L., Omlin, C.W.: Rule Revision with Recurrent Networks. IEEE Transactions on Knowledge and Data Engineering 8(1), 183 (1996)

    Article  Google Scholar 

18. Giles, C.L., Lawrence, S., Tsoi, A.C.: Rule Inference for Financial Prediction using Recurrent Neural Networks. In: Proceedings of the IEEE/IAFE Conference on Computational Intelligence for Financial Engineering (CIFEr), pp. 253–259. IEEE, Piscataway NJ (1997)

    Chapter  Google Scholar 

19. Golea, M.: On the Complexity of Rule Extraction from Neural Networks and Network Querying. In: Proceedings of the Rule Extraction From Trained Artificial Neural Networks Workshop, Society For the Study of Artificial Intelligence and Simulation of Behavior Workshop Series (AISB 1996), University of Sussex, Brighton, UK, pp. 51–59 (1996)

    Google Scholar 

20. Healy, M.J.: A Topological Semantics for Rule Extraction with Neural Networks. Connection Science 11(1), 91–113 (1999)

    Article  Google Scholar 

21. Humphrey, M., Cunningham, S.J., Witten, I.H.: Knowledge Visualization Techniques for Machine Learning. Intelligent Data Analysis 2(4) (1998)

    Google Scholar 

22. Ivanova, I., Kubat, M.: Initialisation of Neural Networks by Means of Decision Trees. Knowledge Based Systems 8(6), 333–344 (1995)

    Article  Google Scholar 

23. Krishnan, R.: A Systematic Method for Decompositional Rule Extraction From Neural Networks. In: Proceedings of the NIPS 1996 Rule Extraction From Trained Artificial Neural Networks Workshop, Queensland University of Technology, pp. 38–45 (1996)

    Google Scholar 

24. Krishnan, R., Sivakumar, G., Battacharya, P.: A Search Technique for Rule Extraction from Trained Neural Networks. Pattern Recognition Letters 20, 273–280 (1999)

    Article  MATH  Google Scholar 

25. Kubat, M.: Decision Trees Can Initialize Radial-Basis Function Networks. IEEE Transactions on Neural Networks 9(5), 813–821 (1998)

    Article  MathSciNet  Google Scholar 

26. Lin, C.-T., Lee, C.S.G.: Neural Fuzzy Systems: a Neuro-Fuzzy Synergism to Intelligent Systems. Prentice-Hall, Upper Saddle River NJ (1996)

    Google Scholar 

27. Maire, F.: A Partial Order for the M-of-N Rule-Extraction Algorithm. IEEE Transactions on Neural Networks 8(6), 1542–1544 (1997)

    Article  Google Scholar 

28. Maire, F.: Rule-Extraction by Backpropagation of Polyhedra. Journées Francophones sur l’Apprentissage Automatique (JFA 1998), Arras France (May1998).

    Google Scholar 

29. McGarry, K., Wermter, S., MacIntyre, J.: Knowledge Extraction from Radial Basis Function Networks. In: Proceedings of the International Joint Conference on Neural Networks, IJCNN 1999 (1999) (to appear)

    Google Scholar 

30. Meneganti, M., Saviello, S., Tagliaferri, R.: Fuzzy Neural Networks for Classification and Detection of Anomalies. IEEE Transactions on Neural Networks 9(5), 848–861 (1998)

    Article  Google Scholar 

31. Michie, D., Spiegelhalter, D.L., Taylor, C.C.: Machine Learning. Neural and Statistical Classification. Hertfordshire Ellis, Horwood (1994)

    Google Scholar 

32. Neumann, J.: Classification and Evaluation of Algorithms for Rule Extraction From Artificial Neural Networks A Review. Centre for Cognitive Science, University of Edinburgh (1998) (unpublished)

    Google Scholar 

33. Omlin, C.W., Giles, C.L.: Extraction of Rules from Discrete-Time Recurrent Neural Networks. Connection Science 5(3-4), 307–336 (1993)

    Google Scholar 

34. Omlin, C.W., Thornber, K., Giles, C.L.: Fuzzy Finite State Automata Can be Deterministically Encoded Into Recurrent Neural Networks. IEEE Transactions on Fuzzy Systems (to appear)

    Google Scholar 

35. Optiz, D.W., Shavlik, J.W.: Dynamically Adding Symbolically Meaningful Nodes to Knowledge-Based Neural Networks. Knowledge-Based Systems 8(6), 301–311 (1995)

    Article  Google Scholar 

36. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, San Mateo (1993)

    Google Scholar 

37. Quinlan, J.R.: Comparing Connectionist and Symbolic Learning Methods. In: Rivest, R. (ed.) Computational Learning Theory and Natural Learning Systems: Constraints and Prospects, pp. 445–456. MIT Press, Cambridge (1994)

    Google Scholar 

38. Saito, K., Nakano, R.: Law Discovery Using Neural Networks. In: Proceedings of the NIPS 1996 Rule Extraction From Trained Artificial Neural Networks Workshop, pp. 62–69. Queensland University of Technology (1996)

    Google Scholar 

39. Schellhammer, I., Diederich, J., Towsey, M., Brugman, C.: Knowledge Extraction and Recurrent Neural Networks: an analysis of an Elman network trained on a natural language learning task. Queensland University of Technology, NRC Technical Report, 97–151 (1997)

    Google Scholar 

40. Setiono, R., Huan, L.: NeuroLinear: From Neural Networks to Oblique Decision Rules. Neurocomputing 17, 1–24 (1997)

    Article  Google Scholar 

41. Setiono, R.: Extracting Rules from Neural Networks by Pruning and Hidden Unit Splitting. Neural Computation 9, 205–225 (1997)

    Article  MATH  Google Scholar 

42. Setiono, R., Thong, J.Y.L., Yap, C.S.: Symbolic Rule Extraction from Neural Networks: An Application to Identifying Organisations Adopting IT. Information and Management 34(2), 91–101 (1998)

    Article  Google Scholar 

43. Sun, R., Sessions, C.: Extracting Plans from Reinforcement Learners. In: Proceedings of the International Symposium on Intelligent Data Engineering and Learning (IDEAL 1998), Springer, Heidelberg (1998)

    Google Scholar 

44. Sun, R., Peterson, T.: Autonomous Learning of Sequential Tasks: Experiments and Analyses. IEEE Transactions on Neural Networks 9(6), 1217–1234 (1998)

    Article  Google Scholar 

45. Sun, R.: Knowledge Extraction from Reinforcement Learning. In: Proceedings of the International Joint Conference on Neural Networks, IJCNN 1999 (1999) (to appear)

    Google Scholar 

46. Thrun, S.B., Bala, J., Bloedorn, E., Bratko, I., Cestnik, B., Cheng, J., De Jong, K., Dzeroski, S., Fahlman, S.E., Fisher, D., Hamann, R., Kaufman, K., Keller, S., Kononenko, I., Kreuziger, J., Michalski, R.S., Mitchell, T., Pachowicz, P., Reich, Y., Vafaie, H., Van de Welde, K., Wenzel, W., Wnek, J., Zhang, J.: The MONK’s Problems: a Performance Comparison of Different Learning Algorithms. Carnegie Mellon University, Technical report CMU-CS-91-197 (1991)

    Google Scholar 

47. Thrun, S.B.: Extracting Provably Correct Rules From Artificial Neural Networks. Technical Report IAI-TR-93-5 Institut fur Informatik III Universitat Bonn (1994)

    Google Scholar 

48. Tickle, A.B., Hayward, R., Diederich, J.: Recent Developments in Techniques for Extracting Rules from Trained Artificial Neural Networks. Herbstschule Konnektionismus (HeKonn 1996), Munster (October 1996)

    Google Scholar 

49. Tickle, A.B., Andrews, R., Golea, M., Diederich, J.: Rule Extraction from Trained Artificial Neural Networks. In: Browne, A. (ed.) Neural Network Analysis, pp. 61–99. Architectures and Applications Institute of Physics Publishing, Bristol (1997)

    Google Scholar 

50. Tickle, A.B., Golea, M., Hayward, R., Diederich, J.: The Truth is in There: Current Issues in Extracting Rules from Trained Feed-Forward Artificial Neural Networks. In: Proceedings of the 1997 IEEE International Conference on Neural Networks (ICNN 1997), vol. 4, pp. 2530–2534 (1997)

    Google Scholar 

51. Tickle, A.B.: Machine Learning, Neural Networks and Information Security: Techniques for Extracting Rules from Trained Feed-Forward Artificial Neural Networks and their Application in an Information Security Problem Domain. PhD Dissertation, Queensland University of Technology (1997)

    Google Scholar 

52. Tickle, A.B., Andrews, R., Golea, M., Diederich, J.: The Truth will come to Light: Directions and Challenges in Extracting the Knowledge Embedded within Trained Artificial Neural Networks. IEEE Transactions on Neural Networks 9(6), 1057–1068 (1998)

    Article  Google Scholar 

53. Tickle, A.B., Maire, F., Bologna, G., Diederich, J.: Extracting the Knowledge Embedded within Trained Artificial Neural Networks: Defining the Agenda. In: Proceedings of the Third International ICSC Symposia on Intelligent Industrial Automation (IIA 1999), and Soft Computing (SOCO 1999), pp. 732–738 (1999)

    Google Scholar 

54. Towell, G., Shavlik, J.: The Extraction of Refined Rules From Knowledge Based Neural Networks. Machine Learning 131, 71–101 (1993)

    Google Scholar 

55. Valiant, L.G.: A theory of the learnable. Communications of the ACM 27(11), 1134–1142 (1984)

    Article  MATH  Google Scholar 

56. Williams, R.J., Zipser, D.: A Learning Algorithm for Continually Running Fully Recurrent Neural Networks. Neural Computation 1(2), 270–280 (1989)

    Article  Google Scholar 

Download references