A new sensitivity-based pruning technique for feed-forward neural networks that improves generalization
Abstract
Multi-layer neural networks of the back-propagation type (MLP-networks) became a well-established tool used in various application areas. Reliable solutions require, however, also sufficient generalization capabilities of the formed networks and an easy interpretation of their function.
These characteristics are strongly related to less sensitive networks with an optimized network structure. In this paper, we will introduce a new pruning technique called SCGSIR that is inspired by the fast method of scaled conjugate gradients (SCG) and sensitivity analysis.
Network sensitivity inhibited during training impacts efficient optimization of network structure. Experiments performed so far yield promising results outperforming the reference techniques when considering both their ability to find networks with optimum architecture and improved generalization.