Abstract
While Large Language Models (LLMs) possess significant capabilities in open-world agent tasks, they also face challenges in rapidly adapting to new, specialized tasks due to their reliance on static pre-trained knowledge. Traditional methods such as fine-tuning are often costly, data-intensive, and may lead to "catastrophic forgetting." Therefore, we present KnowMap, a novel approach that dynamically constructs a knowledge base from environmental and experiential data. KnowMap fine-tunes a small knowledge-embedding model to equip a larger LLM with valuable task-specific knowledge. Our experiments on the ScienceWorld benchmark demonstrate 17.71% improvement for the performance of gpt-4-turbo model. KnowMap not only provides an efficient and effective means for LLM task-adapting, but also highlights how integrating environmental and experiential knowledge can enhance LLMs' reasoning capabilities.
Introduction
Large Language Models (LLMs) are impressive generalists, yet they struggle with "embodied AI" (EAI) tasks—scenarios where an agent must plan and act within a dynamic environment. Their pre-trained knowledge is static; they do not inherently know that a specific object in a simulation is broken or remember which action sequences failed previously. Traditional solutions like Reinforcement Learning (RL) or Supervised Fine-Tuning (SFT) are computationally expensive or risk "catastrophic forgetting." KnowMap addresses this by providing a way to give LLMs "on-the-fly" expertise without modifying their core weights.
Methodology
KnowMap introduces a collaborative "strong-weak" architecture, keeping the heavy-duty LLM frozen while optimizing how it accesses information through a specialized retrieval system.
The Agent Scaffold
The system operates in a three-part loop: a Planner formulates goals, an Actuator executes actions, and an Evaluator analyzes feedback to refine future planning.
Dual Knowledge Base
- Environmental Knowledge: Stores structured triples—
(entity, relation, attribute)—updated in real-time as the agent explores the environment. - Experiential Knowledge: Stores "reflections" from past successful or failed trajectories, broken down into manageable sub-goals.
Knowledge-Embedding Fine-Tuning
The core innovation involves fine-tuning a small 0.56 billion parameter embedding model (BGE-M3) using a contrastive loss function (InfoNCE). This trains the model to match task queries with the most relevant environmental and experiential data, ensuring the LLM receives high-quality context without the overhead of tuning the base model.
Experimental Setup
The framework was evaluated using ScienceWorld, a complex text-based simulation environment. ScienceWorld features 30 task types with over 200,000 possible action-object combinations and simulates real-world physics, including thermodynamics, electricity, and biological life cycles. Unlike simpler benchmarks, it requires long-term planning and deep domain knowledge to navigate successfully.
Results and Analysis
KnowMap was tested across several LLM backbones, including GPT-4-Turbo and DeepSeek-V3, showing consistent performance gains.
The results demonstrate that KnowMap significantly boosts performance. For GPT-4-Turbo, performance increased from a 64.78% baseline to 76.25%, representing a 17.71% relative improvement. KnowMap outperformed popular frameworks like ReasonPlanner (65.06%) and Reflexion (45.34%). It also approached the performance of SwiftSage (84.68%) without the need to fine-tune a 3-billion parameter action-generation model, highlighting its engineering efficiency.
Key insights from the study include:
- The Synergy Effect: Environmental knowledge (the "what") and experiential knowledge (the "how") are far more effective when used together.
- Retrieval as a Bottleneck: Even powerful models can see performance drop if fed irrelevant context. Fine-tuning the retriever ensures the LLM only sees pertinent information.
- Efficiency over Scale: KnowMap achieves competitive results by tuning a model roughly 1/100th the size of the LLMs it supports.
Conclusion
KnowMap provides an efficient strategy for adapting LLMs to specialized tasks through dynamic knowledge retrieval. Future work will address current limitations, such as manually defined relations, and explore extending the framework to multi-modal settings and real-world robotics.
Summary updated