Deterministic dropout for deep neural networks using composite random forest
Article Type
Research Article
Publication Title
Pattern Recognition Letters
Abstract
Dropout prevents overfitting in deep neural networks. Typical strategy of dropout involves random termination of connections irrespective of their importance. Termination blocks the propagation of class discriminative information across the network. As a result, dropout may lead to inferior performance. We propose a deterministic dropout where only unimportant connections are dropped ensuring propagation of class discriminative information. We identify the unimportant connections using a novel composite random forest, integrated into the network. We prove that better generalization is achieved by terminating these unimportant connections. The proposed algorithm is useful in preventing overfitting in noisy datasets. The proposal is equally good for datasets with smaller number of training examples. Experiments on several benchmark datasets show up to 8% improvement in classification accuracy.
First Page
205
Last Page
212
DOI
10.1016/j.patrec.2019.12.023
Publication Date
3-1-2020
Recommended Citation
Santra, Bikash; Paul, Angshuman; and Mukherjee, Dipti Prasad, "Deterministic dropout for deep neural networks using composite random forest" (2020). Journal Articles. 379.
https://digitalcommons.isical.ac.in/journal-articles/379