Find the Intention of Instruction: Comprehensive Evaluation of Instruction Understanding for Large Language Models

Hyeonseok Moon, Jaehyung Seo, Seungyoon Lee, Chanjun Park, Heuiseok Lim

Introduction

While Large Language Models (LLMs) have demonstrated remarkable proficiency in following explicit instructions, their ability to truly comprehend the underlying intent remains a significant area of concern. In real-world scenarios, prompts often contain complex layers of text where some segments resemble instructions but are not intended for execution. This paper introduces IOINST, a novel benchmark designed to evaluate whether models can distinguish between the actual task and distracting, instruction-like content.

The IOINST Benchmark

The core of the IOINST task is a reverse-engineering challenge: given a specific response, the model must identify which instruction from a set of candidates originally produced it. This setup tests whether a model understands the intention behind an instruction rather than just reacting to its format.

Example of the IOINST task structure.

The task structure involves three components:

  1. Context: A response generated by an LLM (e.g., "Dolphin and shark").
  2. Meta-Instruction: The primary task (e.g., "Identify the instruction that generated the response above").
  3. Candidate Instructions: A set of four options, including the correct "Label Instruction" and three distractors.

Methodology and Error Analysis

To rigorously evaluate model performance, the researchers categorized failures into two distinct error types:

  • Case 1: The model attempts the meta-instruction but selects the incorrect candidate.
  • Case 2: The model fails to follow the meta-instruction entirely, instead executing one of the candidate instructions (e.g., naming animals instead of selecting the text of the instruction).

Distractors were crafted with varying levels of difficulty using a pipeline involving semantic search and human verification to ensure the "Context" response maps uniquely to the label instruction.

Pipeline for generating contrastive instructions.

Distractor types include:

  • Random: Unrelated instructions.
  • Semantic: Instructions semantically similar to the label but leading to different outputs.
  • Anti-Attribute: Instructions sharing the same task but differing in fine-grained constraints (e.g., word count).

Key Insights

The Instruction Distraction Effect

LLMs exhibit a "reflexive" tendency to execute any command-like text they encounter. This is evidenced by the high frequency of Case 2 errors, where models ignore the meta-instruction to engage with distractors. This suggests that instruction-tuning may have over-sensitized models to specific formats at the expense of deep comprehension.

The Few-Shot Paradox

Counter-intuitively, providing few-shot examples often degraded performance on the IOINST benchmark. The authors suggest that these examples introduce additional instruction-formatted noise, further confusing the model's perception of the primary intent.

Structural Sensitivity

Model performance improved when meta-instructions were detailed and when the "Context" (the response) was presented before the candidate instructions, indicating that prompt engineering remains a critical factor in intent recognition.

Results

Experimental results highlight a stark contrast between closed-source and open-source models. While GPT-4o achieved approximately 95% accuracy in simpler settings, open-source models like Llama-3-8B struggled significantly, particularly in the Anti-Attribute category where accuracy plummeted to 23.58%. Furthermore, increasing the model's "temperature" (randomness) led to a sharp decline in the ability to follow meta-instructions, making models more susceptible to distraction.

Conclusion

The IOINST benchmark reveals that current LLMs, while capable of following isolated commands, lack robust instruction comprehension in complex contexts. Future work should focus on data-centric training strategies and architectural improvements to help models prioritize user intent over superficial formatting.

Summary updated

BibTeX
@inproceedings{moon2025find,
  author = {Hyeonseok Moon and Jaehyung Seo and Seungyoon Lee and Chanjun Park and Heuiseok Lim},
  title = {{Find the Intention of Instruction: Comprehensive Evaluation of Instruction Understanding for Large Language Models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.18653/v1/2025.findings-naacl.330},
}
Figure