Retrospective Learning from Interactions

Zizhao Chen, Mustafa Omer Gul, Yiwei Chen, Gloria Geng, Anne Wu, Yoav ArtziSubmission to ICLR 2025 · Rejected

Abstract

Multi-turn language interactions naturally include implicit feedback signals. For example, if a listener responds in an unexpected way to an instruction, the instructor may rephrase it, express frustration, or pivot to an alternative task. These signals are task-independent and occupy a relatively constrained subspace of language, allowing a language model to identify them even if it fails on the actual task. This holds the promise of continually learning and improving from interactions without additional annotations. We introduce *ReSpect*, a method to learn from signals in past interactions via retrospection. We deploy *ReSpect* in a new multimodal interaction scenario, where humans instruct a multimodal LLM to solve an abstract reasoning task with a combinatorial solution space. Through thousands of interactions with humans, we show how *ReSpect* gradually improves task completion rate from 31\% to 82\%, all without any external annotation.
BibTeX
@inproceedings{chen2025retrospective,
  author = {Zizhao Chen and Mustafa Omer Gul and Yiwei Chen and Gloria Geng and Anne Wu and Yoav Artzi},
  title = {{Retrospective Learning from Interactions}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2025},
}
Figure