Gradient-Norm Based Attentive Loss for Molecular Property Prediction
Lists
Only you see this
Abstract
Molecular property prediction is one fundamental yet challenging task for drug discovery. Many studies have addressed this problem by designing deep learning algorithms, e.g., sequence-based models and graph-based models. However, the underlying data distribution is rarely explored. We discover that there exist easy samples and hard samples in the molecule datasets, and the overall distribution is usually imbalanced. Current research mainly treats them equally during the model training, while we believe that they shall not share the same weights since neural networks training is dominated by the majority class. Therefore, we propose to utilize a self-attention mechanism to generate a learnable weight for each data sample according to the associated gradient norm. The learned attention value is then embedded into the prediction models to construct an attentive loss for the network updating and back-propagation. It is empirically demonstrated that our proposed method can consistently boost the prediction performance for both classification and regression tasks.
BibTeX
@inproceedings{ma2021gradient,
author = {Hehuan Ma and Yu Rong and Boyang Liu and Yuzhi Guo and Chaochao Yan and Junzhou Huang},
title = {{Gradient-Norm Based Attentive Loss for Molecular Property Prediction}},
booktitle = {BIBM},
year = {2021},
doi = {10.1109/bibm52615.2021.9669671},
}Loading related content...