Download PDFOpen PDF in browserForced Selective Information Reduction for Interpreting Multi-Layered Neural Networks17 pages•Published: September 20, 2022AbstractThe present paper aims to reduce unnecessary information obtained through inputs, supposed to be inappropriately encoded, for producing easily interpretable networks with better generalization. The proposed method lies mainly in forced reduction of selective information even at the expense of a larger cost to eliminate unnecessary information coming from the inputs in the initial stage of learning. Then, in the later stage of learning, selective information is increased to produce a small number of really important connection weights for learning. The method was preliminarily applied to two business data sets: the bankruptcy and the mission statement data sets, in which the interpretation is considered as important as generalization performance. The results show that selective information could be decreased, though the cost to realize this reduction became larger. However, the accompa- nying selective information increase could be used to compensate for the expensive cost to produce simpler and interpretable internal representations with better generalization performance.Keyphrases: cost, generalization, interpretation, multi layered neural networks, selective information In: Tokuro Matsuo (editor). Proceedings of 11th International Congress on Advanced Applied Informatics, vol 81, pages 24-40.
|
