Automated learning of decision rules for text categorization

Reviewer: Ian Hugh Witten

Can rules for document classification be induced from a training set of manually classified documents, enabling new documents to be classified automatically__?__ This question is important because of the time and skill required for manual classification and the huge volumes of textual information to be processed. The problem splits into two parts: creating a feature set to represent each document, and inferring classification rules based on these features. For the first part, the authors advocate the use of topic-specific dictionaries, one for each classification topic, prepared manually in advance. A document is represented by the most frequently occurring dictionary words it contains, for each dictionary. For the second part, a new rule-learning scheme called Swap-1 is described that uses a dynamic optimization technique to overcome the possible shortcomings of the usual greedy rule-selection procedure. The scheme is tested on a collection of 15,000 Reuters news stories and 90 topics. Three-quarters of the stories are used for training, and the resulting rules are evaluated on the remaining stories in terms of recall and precision. Significant improvement is claimed over previously reported results on the same data, although the experimental conditions are slightly different. I found this paper difficult to read and understand, principally because it introduces several new ideas in a sketchy manner and does not evaluate them properly. For <__?__Pub Fmt nolinebreak>example,<__?__Pub Fmt /nolinebreak> it would have been interesting to compare results using <__?__Pub Fmt nolinebreak>Swap-1<__?__Pub Fmt /nolinebreak><__?__Pub Caret> with those of standard rule-learning schemes such as C4.5 [1] and to evaluate what improvement the local-dictionary feature selection scheme gives over traditional methods. Focusing the experiments on a comparison with a single study, details of which are not included, seems to be of lesser value, particularly in a high-profile journal of fairly general coverage.