Data pseudo-labeling while adapting BERT for multitask approaches
Lists
Only you see this
Abstract
Nowadays, BERT models have found wide use in the NLP field. However, standard BERT architecture training can be stifled by the lack of labels for different tasks while treating multitask settings as a one-task multilabel setting. For every example, we have labels from this example's source task but not from other tasks. This article addressed this issue, exploring eight different data pseudo-labeling approaches in the GLUE 4-task setting. These approaches do not require changes in samples or model architecture. One of the presented techniques excels results on RTE from the original article, by 6.2 %, and falls behind the original article on QQP, MNLI, and SST only by 0.5-1.2 %. This way also excels other pseudo-labeling approaches explored in the article by 0.5-2% on average if we consider similar tasks. However, for tasks that are dissimilar to each other, different proposed approach yields the best results.
BibTeX
@inproceedings{karpov2021data,
author = {Dmitry Karpov and Mikhail Burtsev},
title = {{Data pseudo-labeling while adapting BERT for multitask approaches}},
booktitle = {Kompʹûternaâ},
year = {2021},
doi = {10.28995/2075-7182-2021-20-358-366},
}Loading related content...