Text classification based on deep belief network and softmax regression

Mingyang Jiang, Yanchun Liang, Xiaoyue Feng, Xiaojing Fan, Zhili Pei, Yu Xue, Renchu Guan

Abstract

In this paper, we propose a novel hybrid text classification model based on deep belief network and softmax regression. To solve the sparse high-dimensional matrix computation problem of texts data, a deep belief network is introduced. After the feature extraction with DBN, softmax regression is employed to classify the text in the learned feature space. In pre-training procedures, the deep belief network and softmax regression are first trained, respectively. Then, in the fine-tuning stage, they are transformed into a coherent whole and the system parameters are optimized with Limited-memory Broyden–Fletcher–Goldfarb–Shanno algorithm. The experimental results on Reuters-21,578 and 20-Newsgroup corpus show that the proposed model can converge at fine-tuning stage and perform significantly better than the classical algorithms, such as SVM and KNN.
BibTeX
@article{jiang2018text,
  author = {Mingyang Jiang and Yanchun Liang and Xiaoyue Feng and Xiaojing Fan and Zhili Pei and Yu Xue and Renchu Guan},
  title = {{Text classification based on deep belief network and softmax regression}},
  journal = {Neural Computing and Applications},
  year = {2018},
  url = {https://openreview.net/forum?id=4RFw44W0OK},
}

Loading related content...

Figure