Enhancing Lightweight Vision Language Models through Group Competitive Learning for Socially Compliant Navigation

Xinyu Zhang, Atsushi Konno, Toshihiko Yamasaki, Ling XiaoarXiv 2026 · arXiv

TL;DR

Researchers have developed Group Competitive Learning (GCL), a training strategy that allows lightweight Vision Language Models (VLMs) to punch far above their weight class in social robot navigation. By pitting two models against each other in a structured "benign competition," a 3B-parameter model was able to outperform an 8B-parameter baseline by 28%, offering a path toward robots that are both socially intelligent and fast enough for real-time deployment.

What Problem Does This Paper Address?

Social robot navigation is a high-stakes balancing act. A robot in a crowded mall or hospital shouldn't just avoid hitting people; it needs to respect "unwritten" social norms, like maintaining personal space or yielding the right-of-way.

Large Vision Language Models (VLMs) are great at this kind of complex reasoning, but they are computationally "heavy," leading to laggy performance that is dangerous for a moving robot. Conversely, lightweight VLMs are fast but often fail to grasp the nuances of social interaction. Traditional Knowledge Distillation (KD)—where a small "student" model mimics a large "teacher"—often fails here because it focuses on low-level visual features rather than the high-level reasoning required for social compliance.

Methods

The authors introduce Group Competitive Learning (GCL), a dual-stream strategy where two models (a Learner and a Guide) mutually enhance each other. Unlike standard distillation, GCL treats the relationship as a competition to unlock the group's collective potential.

Fig. 1: The Overview of GCL. The left of the figure illustrates the GCL strategy’s inputs examples: texts and images across social navigation dataset. The middle shows the GCL strategy’s components: Group Competitive Objective, to align VLMs at both semantic and distributional token levels, and Asymmetric Group Optimization, to ex- plore the optimization dynamics and performance boundaries of GCL. The right shows the socially compliant instruction of GCL outputs.

The framework consists of two core pillars:

  1. Group Competitive Objective (GCO): A multi-part loss function that aligns the models at two levels:
  2. Global Semantic Loss (GSL): Uses contrastive learning to ensure both models have a consistent high-level understanding of the scene.
  3. Distributional Regularization Loss (DRL): Uses Jensen-Shannon divergence to align the models' fine-grained token-level predictions, helping the learner capture "dark knowledge" (implicit relationships between words) from the guide.
  4. Asymmetric Group Optimization (AGO): This is the "secret sauce" that manages the competition. It assigns different Learning Rates (the better model gets a lower rate to stay stable, while the learner gets a higher rate to adapt quickly) and different Temperatures (larger models stay "rigid" to act as anchors, while smaller models use higher temperatures to smooth their outputs and enhance plasticity).

Datasets

The researchers evaluated GCL on two key social navigation benchmarks:

  • SNEI (Social Navigation Environment Interaction): Contains 1,625 image-text pairs across various social scenarios.
  • MUSON (Multimodal Social Navigation): A larger reasoning-oriented dataset with 4,000 pairs focused on urban environments.

Key Insights and Ideas

  • Capacity Inversion: The most striking finding is that a smaller model can actually surpass its larger guide. For example, a 3B model trained alongside a 4B guide didn't just catch up; it ended up performing better than the guide itself.
  • Asymmetry is Essential: The paper proves mathematically that you cannot treat the two models the same. By using Asymmetric Shift Force (via different temperatures), the larger model acts as a "rigid semantic anchor," preventing the training process from collapsing into a "blind leading the blind" scenario.
  • Beyond Visuals: While many VLM distillation methods focus on aligning visual encoders, GCL shows that aligning the reasoning (the language model's output distribution) is what actually drives social compliance.

Key Results

The empirical results demonstrate a significant leap in performance over vanilla Supervised Fine-Tuning (SFT):

  • Massive Gains: The Qwen2.5-VL-3B model saw a 40% improvement in F1 score (reaching 0.968) when trained via GCL.
  • Efficiency vs. Accuracy: Under standard training, the 3B model trailed the 8B model (0.692 vs 0.755). After GCL, the 3B model outperformed the 8B baseline by 28%.
  • Robustness: The strategy worked across 12 different model combinations, including both homogeneous (same architecture) and heterogeneous (different architectures) pairings.
Table 1: Performance of GCL across 12 competitive groups, showing significant F1 score improvements.

Limitations and Future Work

Despite the success, the authors note that GCL still faces hurdles in fine-grained spatiotemporal reasoning. In very complex scenes—such as those involving highly reflective surfaces or very subtle human-object interactions (like someone just starting to reach for a door handle)—the models can still misinterpret the situation.

Future work aims to extend GCL to a wider variety of VLM families and explore even smaller, more efficient models to push the boundaries of what can be deployed on low-power robotic hardware.

Summary updated

Abstract

Social robot navigation requires a sophisticated integration of scene semantics and human social norms. Scaling up Vision Language Models (VLMs) generally improves reasoning and decision-making capabilities for socially compliant navigation. However, increased model size incurs substantial computational overhead, limiting suitability for real-time robotic deployment. Conversely, lightweight VLMs enable efficient inference but often exhibit weaker reasoning and decision-making performance in socially complex environments. Achieving both strong reasoning ability and efficiency remains an open challenge. To bridge this gap, we propose Group Competitive Learning (GCL), a strategy designed to amplify the capabilities of lightweight VLMs. Our strategy introduces the Group Competitive Objective (GCO) to harmonize global semantics with distributional regularization, alongside Asymmetric Group Optimization (AGO) to explore the upper limits of model performance. Empirical evaluations on social navigation benchmarks demonstrate that GCL significantly elevates VLM performance. Specifically, GCL enables the Qwen2.5-VL-3B learner model and guide Qwen3-VL-4B to achieve an F1 score of 0.968 and 0.914, representing 40\% and 12\% improvement over vanilla supervised fine-tuning (SFT). Notably, under vanilla SFT, the 3B model initially trails the 8B model (F1: 0.692 vs. 0.755). However, through the GCL, the 3B model outperforms (28\%) the 8B baseline model. These results suggest that GCL provides an effective solution for achieving both high accuracy and computational efficiency in real-world deployment.
BibTeX
@misc{zhang2026enhancing,
  author = {Xinyu Zhang and Atsushi Konno and Toshihiko Yamasaki and Ling Xiao},
  title = {{Enhancing Lightweight Vision Language Models through Group Competitive Learning for Socially Compliant Navigation}},
  year = {2026},
  eprint = {2603.11447},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.11447},
}
Figure