HLMTrans: A Sim-to-Real Transfer Framework for Spatial Crowdsourcing with Human-Guided Language Models

Qingshun Wu, Yafei Li, Lulu Li, Yuanyuan Jin, Shuo He, Mingliang XuIJCAI 2025 · DOI

Introduction

Spatial Crowdsourcing (SC) platforms, such as Uber, Didi, and Gigwalk, rely on efficient task assignment to match workers with tasks in real-time. While Reinforcement Learning (RL) models have shown great promise in learning optimal assignment policies within simulators, they frequently fail when deployed in real-world environments. This performance drop is primarily due to the sim-to-real gap, specifically regarding travel time misalignment.

Simulators often rely on simplified distance-time formulas that fail to account for the non-linear delays caused by traffic congestion, weather conditions, and traffic lights. Furthermore, simulators typically ignore individual worker preferences, such as a desire to avoid specific high-traffic corridors during peak hours. Existing data-driven travel time estimators require massive amounts of real-world data, which is often unavailable or prohibitively expensive to collect for every possible state.

The HLMTrans Framework

To address these challenges, the authors propose HLMTrans, a novel framework that leverages the zero-shot reasoning capabilities of Large Language Models (LLMs) to bridge the system dynamics gap between simulation and reality.

The HLMTrans framework architecture, illustrating the interaction between the RL decision module, the LLM-based sim-to-real transfer, and human feedback.

The framework consists of four integrated modules:

  1. RLMs for Task Assignment: A Deep Q-Network (DQN) handles the core logic of matching workers to tasks based on the platform's current state.
  2. LLM-based Sim-to-Real Transfer: Instead of relying on a simulator's mathematical formulas, the framework uses an LLM (e.g., LLaMA-3) to estimate realistic travel times. To ensure efficiency, the authors developed the Monte Carlo Graph Search-based Congestion Influence Subgraph (MCGS-CI) algorithm. This identifies the minimal road network segment affecting a route and provides it to the LLM as a contextual prompt.
  3. Grounded Action Transformation (GAT): The LLM’s travel time estimates are treated as domain knowledge and fed into a GAT forward model. This model adjusts the simulator’s dynamics, allowing the RL agent to learn a policy that is robust to real-world conditions.
  4. Human Preference Learning (HPLM): Using a mechanism inspired by Direct Preference Optimization (DPO), the LLM is fine-tuned based on human feedback. This allows the system to prioritize factors like time efficiency versus distance based on actual human rankings.

Experimental Evaluation

HLMTrans was evaluated using two massive real-world taxi trajectory datasets from Didi Chuxing in Chengdu (CD) and Xian (XA). The road networks were sourced from OpenStreetMap, involving tens of thousands of nodes and hundreds of thousands of orders.

Performance comparison of HLMTrans against state-of-the-art baselines on Chengdu (CD) and Xian (XA) datasets, showing Average Execution Time (AET) across varying worker scales.

Key Results

  • Revenue Superiority: HLMTrans significantly outperformed the state-of-the-art baseline (dTrans), achieving a 21.9% increase in Average Total Revenue (ATR) in Chengdu and a 23.7% increase in Xian.
  • LLM Reasoning: The study found that LLMs, when provided with a "Congestion Influence Subgraph," can reason about traffic dynamics (weather, time of day, road types) more accurately than specialized learning-based models like MulT-TTE.
  • Robustness: The framework showed the most significant gains when task deadlines were longer, as it could more effectively plan complex assignments that account for real-world delays.

Conclusion

HLMTrans demonstrates that LLMs can serve as powerful "world models" for reasoning about complex system dynamics in logistics and spatial crowdsourcing. By focusing on the dynamics gap rather than just the perception gap, and by incorporating human-in-the-loop feedback, the framework provides a robust path for transitioning RL policies from simulation to the messy reality of urban traffic.

Summary updated

BibTeX
@inproceedings{wu2025hlmtrans,
  author = {Qingshun Wu and Yafei Li and Lulu Li and Yuanyuan Jin and Shuo He and Mingliang Xu},
  title = {{HLMTrans: A Sim-to-Real Transfer Framework for Spatial Crowdsourcing with Human-Guided Language Models}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/471},
}
Figure