New correlation based pruning algorithm for designing artificial neural networks

Abstract

Artificial neural networks (ANNs) are complex and useful problem solvers. Architecture determination of ANN is an important issue for the successful application of ANNs in many practical problems. It is well known that a three layered ANN, consists of an input, a hidden, and an output layer, can solve any kind of linear and nonlinear problems. This thesis proposes a new pruning algorithm, architecture designing by correlation and sensitivity pruning (ADCSP), to determine the three layered near optimal ANN architectures automatically. The salient feature of ADCSP is that it uses correlations among the hidden neurons to design the ANN architecture. It uses merge approach to prune an ANN. It uses computationally inexpensive formula to determine redundant hidden neurons for pruning. As a result, the convergence of it becomes faster. ADCSP always try to maintain its generalization ability and avoid overfitting. It has been tested extensively on a number of benchmark problems in machine learning and ANNs. These problems are Australian credit card assessment problem, iris problem, soybean problem, and four medical problems (breast cancer, diabetes, heart disease, and thyroid). The experimental results show that ADCSP can determine smaller architectures with good generalization ability compared to many other works.