SituatedThinker: Grounding LLM Reasoning with Real-World through Situated Thinking

Junnan Liu, Linhao Luo, Thuy-Trang Vu, Gholamreza Haffari

Abstract

Recent advances in large language models (LLMs) demonstrate their impressive reasoning capabilities. However, the reasoning confined to internal parametric space limits LLMs' access to real-time information and understanding of the physical world. To overcome this constraint, we introduce SituatedThinker, a novel framework that enables LLMs to ground their reasoning in real-world contexts through situated thinking, which adaptively combines both internal knowledge and external information with predefined interfaces. By utilizing reinforcement learning, SituatedThinker incentivizes deliberate reasoning with the real world to acquire information and feedback, allowing LLMs to surpass their knowledge boundaries and enhance reasoning. Experimental results demonstrate significant performance improvements on multi-hop question-answering and mathematical reasoning benchmarks. Furthermore, SituatedThinker demonstrates strong performance on unseen tasks, such as KBQA, TableQA, and text-based games, showcasing the generalizable real-world grounded reasoning capability.

Introduction

Despite the impressive reasoning of "long-Chain-of-Thought" (long-CoT) models like OpenAI’s o1 or DeepSeek-R1, they remain largely confined to their internal parametric memory. This "closed-world" nature leads to three major issues: hallucinations, static knowledge that cannot access real-time data, and a lack of physical grounding for tasks involving dynamic environments.

While existing solutions like Retrieval-Augmented Generation (RAG) or tool-calling help, they often rely on rigid, predefined workflows. SITUATEDTHINKER addresses this by introducing situated thinking—the ability for a model to decide when it needs help and how to use external feedback to refine its own internal reasoning.

The SITUATEDTHINKER Framework

The core of the framework is the integration of internal reasoning with external "situated actions." The model uses standardized interfaces to interact with the world, pausing its internal monologue to invoke external tools (like search engines or code executors) when it recognizes a knowledge gap.

Overview of the SITUATEDTHINKER framework, illustrating the interaction between the Internal World (LLM), Situated Thinking, and the External World (Knowledge Graphs, Databases, etc.).

To train the model, the authors utilized Group Relative Policy Optimization (GRPO). Notably, the model was not given explicit rewards for using tools. Instead, a simple rule-based reward was applied: Is the final answer correct? Through thousands of trials, the model "discovered" that invoking interfaces to get external data or verify logic was the most reliable path to success.

Experimental Results

SITUATEDTHINKER was trained on a generalist dataset (MuSiQue and Big-Math) and evaluated across diverse benchmarks. It consistently outperformed both base models and standard RAG/tool-calling baselines.

Performance comparison of SITUATEDTHINKER against various RAG baselines on multi-hop QA benchmarks.

Key performance highlights include:

  • Multi-Hop QA: On the Bamboogle benchmark, SITUATEDTHINKER (14B) achieved an Exact Match (EM) score of 0.512, nearly tripling the performance of Naive RAG (0.185).
  • Mathematics: On the challenging MATH500 set, it reached 87.1% accuracy, outperforming several math-specialized models despite being a generalist framework.
  • Zero-Shot Generalization: The model showed massive performance jumps on tasks it was never trained for, such as the TextWorld environment, where it reached 94% accuracy by effectively using the game's feedback loop.

Key Insights and Discussion

One of the most striking findings is the emergence of knowledge boundaries. The model learns to recognize its own limits, invoking tools specifically when internal parameters are insufficient. Scaling behaviors also emerged: the 14B parameter model used retrieval less often than the 8B model but used code execution more, suggesting that larger models rely more on internal facts but recognize the need for external verification for complex logic.

Limitations and Future Work

Currently, the framework is text-only. Future research will focus on incorporating vision and audio interfaces to enhance real-world grounding. Additionally, extending the framework to handle non-deterministic tasks and open-ended creative reasoning remains a major challenge for moving from text-based tools to actual physical interaction in robotics.

Summary updated

Abstract

Recent advances in large language models (LLMs) demonstrate their impressive reasoning capabilities. However, the reasoning confined to internal parametric space limits LLMs' access to real-time information and understanding of the physical world. To overcome this constraint, we introduce SituatedThinker, a novel framework that enables LLMs to ground their reasoning in real-world contexts through situated thinking, which adaptively combines both internal knowledge and external information with predefined interfaces. By utilizing reinforcement learning, SituatedThinker incentivizes deliberate reasoning with the real world to acquire information and feedback, allowing LLMs to surpass their knowledge boundaries and enhance reasoning. Experimental results demonstrate significant performance improvements on multi-hop question-answering and mathematical reasoning benchmarks. Furthermore, SituatedThinker demonstrates strong performance on unseen tasks, such as KBQA, TableQA, and text-based games, showcasing the generalizable real-world grounded reasoning capability. Our codes are available at https://github.com/jnanliu/SituatedThinker.
BibTeX
@misc{liu2025situatedthinker,
  author = {Junnan Liu and Linhao Luo and Thuy-Trang Vu and Gholamreza Haffari},
  title = {{SituatedThinker: Grounding LLM Reasoning with Real-World through Situated Thinking}},
  year = {2025},
  eprint = {2505.19300},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.19300},
}
Figure