Nothing Special   »   [go: up one dir, main page]

skip to main content
Skip header Section
Strength or Accuracy: Credit Assignment in Learning Classifier SystemsOctober 2003
Publisher:
  • SpringerVerlag
ISBN:978-1-85233-770-4
Published:01 October 2003
Skip Bibliometrics Section
Reflects downloads up to 20 Feb 2025Bibliometrics
Abstract

No abstract available.

Cited By

  1. ACM
    Uwano F, Dobashi K, Takadama K and Kovacs T Generalizing rules by random forest-based learning classifier systems for high-dimensional data mining Proceedings of the Genetic and Evolutionary Computation Conference Companion, (1465-1472)
  2. ACM
    Lanzi P Learning classifier systems Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation, (407-430)
  3. ACM
    Kovacs T and Tindale R Analysis of the niche genetic algorithm in learning classifier systems Proceedings of the 15th annual conference on Genetic and evolutionary computation, (1069-1076)
  4. ACM
    Marzukhi S, Browne W and Zhang M Adaptive artificial datasets through learning classifier systems for classification tasks Proceedings of the 15th annual conference companion on Genetic and evolutionary computation, (1243-1250)
  5. ACM
    Marzukhi S, Browne W and Zhang M Two-cornered learning classifier systems for pattern generation and classification Proceedings of the 14th annual conference on Genetic and evolutionary computation, (895-902)
  6. ACM
    Kovacs T, Edakunni N and Brown G Accuracy exponentiation in UCS and its effect on voting margins Proceedings of the 13th annual conference on Genetic and evolutionary computation, (1251-1258)
  7. ACM
    Butz M Learning classifier systems Proceedings of the 13th annual conference companion on Genetic and evolutionary computation, (941-962)
  8. ACM
    Butz M Learning classifier systems Proceedings of the 12th annual conference companion on Genetic and evolutionary computation, (2331-2352)
  9. Alcala-Fdez J, Flugy-Pape N, Bonarini A and Herrera F (2010). Analysis of the Effectiveness of the Genetic Algorithms based on Extraction of Association Rules, Fundamenta Informaticae, 98:1, (1-14), Online publication date: 1-Jan-2010.
  10. Mucientes M and Bugarín A (2010). People detection through quantified fuzzy temporal rules, Pattern Recognition, 43:4, (1441-1453), Online publication date: 1-Apr-2010.
  11. Santos M, Mathew W and Santos H GridclassTK Proceedings of the European conference of systems, and European conference of circuits technology and devices, and European conference of communications, and European conference on Computer science, (43-47)
  12. Romero C, González P, Ventura S, del Jesus M and Herrera F (2009). Evolutionary algorithms for subgroup discovery in e-learning, Expert Systems with Applications: An International Journal, 36:2, (1632-1644), Online publication date: 1-Mar-2009.
  13. Li M and Wang Z (2009). A hybrid coevolutionary algorithm for designing fuzzy classifiers, Information Sciences: an International Journal, 179:12, (1970-1983), Online publication date: 1-May-2009.
  14. Santos M, Mathew W, Kovacs T and Santos H (2009). A grid data mining architecture for learning classifier systems, WSEAS Transactions on Computers, 8:5, (820-830), Online publication date: 1-May-2009.
  15. ACM
    Butz M Learning classifier systems Proceedings of the 10th annual conference companion on Genetic and evolutionary computation, (2367-2388)
  16. ACM
    Kovacs T and Bull L Toward a better understanding of rule initialisation and deletion Proceedings of the 9th annual conference companion on Genetic and evolutionary computation, (2777-2780)
  17. ACM
    Marshall J, Brown G and Kovacs T Bayesian estimation of rule accuracy in UCS Proceedings of the 9th annual conference companion on Genetic and evolutionary computation, (2831-2834)
  18. ACM
    Butz M Learning classifier systems Proceedings of the 9th annual conference companion on Genetic and evolutionary computation, (3035-3056)
  19. ACM
    Brown G, Kovacs T and Marshall J UCSpv Proceedings of the 9th annual conference on Genetic and evolutionary computation, (1774-1781)
  20. Sigaud O and Wilson S (2007). Learning classifier systems: a survey, Soft Computing - A Fusion of Foundations, Methodologies and Applications, 11:11, (1065-1078), Online publication date: 1-Sep-2007.
  21. Berlanga F, del Jesus M, Gacto M and Herrera F A genetic-programming-based approach for the learning of compact fuzzy rule-based classification systems Proceedings of the 8th international conference on Artificial Intelligence and Soft Computing, (182-191)
  22. Kovacs T and Kerber M (2006). A Study of Structural and Parametric Learning in XCS, Evolutionary Computation, 14:1, (1-19), Online publication date: 1-Mar-2006.
  23. ACM
    Marshall J and Kovacs T A representational ecology for learning classifier systems Proceedings of the 8th annual conference on Genetic and evolutionary computation, (1529-1536)
  24. ACM
    Lanzi P and Loiacono D Standard and averaging reinforcement learning in XCS Proceedings of the 8th annual conference on Genetic and evolutionary computation, (1489-1496)
Contributors
  • University of Bristol

Reviews

Jonathan P. E. Hodgson

A typical reinforcement learning algorithm is made up of an alternation of two phases. In one phase, the value of a given action is updated, based on the reward received by the system. Because this is an exploration phase, the system may not always choose the currently most-favored action, but in this phase rules are generally evaluated based on their strength, defined as the predicted expected value of their actions. Thus, a rule that predicts higher rewards will be preferred to one that predicts lower rewards. In the other phase, a genetic algorithm based on some measure of rule fitness is used to improve the rules. This book is an important step in the study of reinforcement learning. It seeks to understand the way in which rules in a learning program of this type can be evaluated, and how they evolve. The book undertakes a detailed comparison of two measures for evaluating rule fitness in reinforcement learning. In Stewart Wilson's XCS system, the genetic algorithm uses rule accuracy as the measure of fitness (other systems have used rule strength). In order to make the comparison as valid as possible, Kovacs has implemented a system, SB-XCS, which is designed to differ from XCS only in that it uses rule strength rather than accuracy to measure rule fitness. Kovacs' empirical results show that both XCS and SB-XCS can learn the optimal rules (as well as others) for the six-multiplexer problem. On the other hand, SB-XCS fares worse than XCS on the Woods2 problem. Kovacs advances two possible reasons for this. One is that XCS retains rules of high accuracy, which record that some actions are bad (and have low rewards), whereas SB-XCS does not retain this kind of rule. The second possibility is that unlike SB-XCS, XCS is biased toward producing rules that evaluate all possible state-action pairs. This empirical comparison of XCS and SB-XCS is the basis for much of the book's content. The comparison leads naturally to a consideration of what exactly a system should learn, and what metrics can be used to measure learning. Kovacs also considers the question of what kind of problems are appropriate for each system. Here, the distinction is between sequential tasks, in which the total reward can depend on the sequence of steps taken to reach the goal, and nonsequential ones, where the complete reward is given after a single step plays a significant role. The reward structure for a sequential task usually requires some discounting of intermediate results. One conclusion the author reaches in this context is that the weighting of rewards to teach desirable behavior must be carefully managed, if it is to be effective. A biased reward system can inhibit learning; Kovacs gives a detailed example of a sequential task to establish this. The concluding chapter summarizes the lessons learned and points out directions for future research. Kovacs emphasizes the need for a design methodology for reinforcement learning systems. Genetic algorithms do not do all the work for us, and complex systems are not going to work the way we expect them to, at least not in the present state of knowledge. For a work that covers as much territory as this one, the book is remarkably self-contained. There are appendices that summarize the needed background on reinforcement learning, rule generalization, and evolutionary computing. A very useful appendix provides an example execution of one cycle (action and credit assignment) of XCS. The thoroughness and clarity of the writing makes this a book that can (and should) be read by anyone with an interest in reinforcement learning. In addition to providing valuable insights into the workings of the two algorithms, it suggests numerous directions for future research. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Please enable JavaScript to view thecomments powered by Disqus.

Recommendations