Algorithm to extract rules from artificial neural networks

Abstract

A new rule extraction algorithm, called rule extraction from artificial neural networks (REANN) is proposed and implemented to extract symbolic rules from ANNs. A standard three-layer feedforward ANN is the basis of the algorithm. A four-phase training algorithm is proposed for backpropagation learning. In the first phase, the number of hidden nodes of the network is determined automatically in a constructive fashion by adding nodes one after another based on the performance of the network on training data. In the second phase, the ANN is pruned such that irrelevant connections and input nodes are removed while its predictive accuracy is still maintained. In the third phase, the continuous activation values of the hidden nodes are discretized by using an efficient heuristic clustering algorithm. And finally in the fourth phase, rules are extracted by examining the discretized activation values of the hidden nodes using a rule extraction algorithm, REx. Extensive experimental studies on several benchmarks classification problems, such as breast cancer, iris, diabetes, wine, season, golf-playing, and lenses classification problems, demonstrate the effectiveness of the proposed approach with good generalization ability.