Introduction
Grounded language learning requires agents to interpret natural language instructions within a physical or virtual context. While generative models like Probabilistic Context-Free Grammars (PCFGs) are commonly used to map instructions to formal representations, they often struggle with global dependencies and complex structures. In traditional NLP, discriminative reranking addresses this by selecting the best candidate from a generative model's top $N$ outputs using a richer feature set. However, standard reranking relies on expert-annotated "gold-standard" parse trees, which are rarely available in grounded settings. This paper explores an alternative: using task success (response feedback) as a weak supervisory signal for discriminative reranking.
Methodology
The authors propose a framework that replaces human-labeled parses with Response-Based Updates. Instead of comparing candidates to a ground-truth tree, the system executes the plans in a virtual environment.
- Pseudo-Gold Labels: The parse that achieves the highest success rate (e.g., reaching a destination) is treated as the reference for training.
- Modified Perceptron Training: An adapted Averaged Perceptron algorithm updates model weights based on the difference between the current prediction and all candidate parses that performed better, weighted by their relative success rates.
- Feature Engineering: To combat sparsity in large grounded grammars, the authors introduced predicate-only features (e.g., stripping arguments to turn
Turn(LEFT)intoTurn()) and descended action features, which track the relationship between high-level instructions and low-level plan steps.
Experimental Setup
The approach was evaluated using a navigation dataset consisting of 706 route instructions across three virtual hallway environments. Supervision was provided via human follower traces rather than parse trees. Evaluation followed a "leave-one-environment-out" cross-validation strategy, training on two virtual maps and testing on the third.
Results and Discussion
The results demonstrate that optimizing for task success is more effective for grounded tasks than optimizing for linguistic structure. The "Multi" approach, which utilizes the entire distribution of successful versus unsuccessful parses, provided a much richer learning signal than picking a single best parse.
As shown in Figure 6, the response-based reranker ("Multi") significantly outperformed the baseline, increasing single-sentence execution success from 57.22% to 62.81%. For complex, multi-sentence paragraphs, success rose from 20.17% to 26.57%. Notably, the model trained on task feedback outperformed a model trained on human-annotated "gold" parses by over 10% in single-sentence tasks (62.81% vs 52.57%). This suggests that for grounded tasks, optimizing for the end goal is more effective than trying to mimic a human's internal linguistic structure.
Conclusion
While the proposed method effectively bridges the gap between weak supervision and discriminative reranking, challenges remain regarding feature sparsity and the computational cost of simulating non-deterministic environments. Future work may explore forest reranking or kernel methods to further improve generalization across diverse grounded tasks like robotic manipulation or interactive dialogue.
Summary updated