On Task Description of In-context Learning: A Study from Information Perspective
Lists
Only you see this
Abstract
Large language models (LLMs) have demonstrated remarkable performance in a wide range of applications, making in-context learning an essential technique. Although the in-context learning has been widely applied, our understanding of its underlying processes still remains limited. In-context learning in LLMs primarily relies on two types of information: in-context samples and task descriptions. While previous research has extensively investigated the influence of in-context samples on learning behavior, the role of task descriptions has not been adequately explored, despite their practical significance. In this paper, we present a study examining the impact of task descriptions on the in-context learning performance of LLMs. We devise a synthetic experiment setting, making the information of task description controllable. Through a series of well-designed experiments, we systematically vary task description information and assess the resulting effects on model performance across multiple tasks. Our findings reveal complex roles of task descriptions: task description will suppress the model to learn from in-context examples; task description will increase the lower bound of the in-context learning performance. This study contributes to a deeper understanding of the in-context learning mechanism in LLMs, paving the way for more effective real-world applications of these powerful models
BibTeX
@misc{xuanyuan2024task,
author = {Meidai Xuanyuan and Tao Yang and Jingwen Fu and Yuwang Wang},
title = {{On Task Description of In-context Learning: A Study from Information Perspective}},
year = {2024},
url = {https://openreview.net/forum?id=TFR0GrzERG},
howpublished = {OpenReview},
note = {Submitted to ICLR 2024},
}