Abstract:
A new rule extraction algorithm, called rule extraction from artificial neural networks
(REANN) is proposed and implemented to extract symbolic rules from ANNs. A
standard three-layer feedforward ANN is the basis of the algorithm. A four-phase
training algorithm is proposed for backpropagation learning. In the first phase, the
number of hidden nodes of the network is determined automatically in a constructive
fashion by adding nodes one after another based on the performance of the network on
training data. In the second phase, the ANN is pruned such that irrelevant connections
and input nodes are removed while its predictive accuracy is still maintained. In the third
phase, the continuous activation values of the hidden nodes are discretized by using an
efficient heuristic clustering algorithm. And finally in the fourth phase, rules are
extracted by examining the discretized activation values of the hidden nodes using a rule
extraction algorithm, REx. Extensive experimental studies on several benchmarks
classification problems, such as breast cancer, iris, diabetes, wine, season, golf-playing,
and lenses classification problems, demonstrate the effectiveness of the proposed
approach with good generalization ability.
