Feature Selection for Neural Networks Using Group Lasso Regularization
Article Type
Research Article
Publication Title
IEEE Transactions on Knowledge and Data Engineering
Abstract
We propose an embedded/integrated feature selection method based on neural networks with Group Lasso penalty. Group Lasso regularization is considered to produce sparsity on the inputs to the network, i.e., for selection of useful features. Lasso based feature selection using a multi-layer perceptron usually requires an additional set of weights, while our Group Lasso formulation does not require that. However, Group Lasso penalty is non-differentiable at the origin. This may lead to oscillations in numerical simulations and make it difficult to analyze theoretically. To address this issue, four smoothing Group Lasso penalties are introduced. A rigorous proof for the convergence of the proposed algorithm is presented under suitable assumptions. To verify the effectiveness, a three-step algorithmic architecture is adopted in implementation. Experimental results on several datasets validate the theoretical results and demonstrate the competitive performance of the proposed method.
First Page
659
Last Page
673
DOI
10.1109/TKDE.2019.2893266
Publication Date
4-1-2020
Recommended Citation
Zhang, Huaqing; Wang, Jian; Sun, Zhanquan; Zurada, Jacek M.; and Pal, Nikhil R., "Feature Selection for Neural Networks Using Group Lasso Regularization" (2020). Journal Articles. 331.
https://digitalcommons.isical.ac.in/journal-articles/331