LatRec: Recognizing Goals in Latent Space (Student Abstract)

Leonardo Amado, Felipe MeneguzziAAAI 2020 · DOI

Abstract

Goal recognition is a critical component of human-AI interaction, yet it traditionally relies on manually authored domain models that are difficult to scale to complex environments. We present LatRec, a framework that automates goal recognition by learning domain models directly from raw image streams. By utilizing deep autoencoders to map visual data into a discrete latent space, LatRec enables the application of symbolic planning techniques to subsymbolic data, effectively bridging the gap between pixel-level observations and high-level plan recognition.

Introduction

Identifying an agent's intentions by observing their actions—known as goal recognition—is fundamental for collaborative AI. Standard approaches require a domain expert to hand-code a complete model of the environment, typically using the Planning Domain Definition Language (PDDL). This model must define every possible state and the underlying "physics" of every action.

In real-world scenarios, creating these models is a significant bottleneck: it is time-consuming, error-prone, and often unfeasible for dynamic environments. Furthermore, real-world observations are frequently noisy or incomplete. LatRec addresses this by asking: Can we perform goal recognition without a human-provided model, using only raw visual data?

The LatRec Framework

The authors propose a pipeline that integrates deep learning with classical symbolic planning. The process is divided into three primary stages: latent space encoding, unsupervised action learning, and goal recognition.

The LatRec pipeline: (a) Autoencoder training, (b) Unsupervised action learning to generate PDDL, and (c) The plan recognition process using latent representations.

1. Latent Space Encoding

A deep autoencoder processes 42x42 grayscale images. The encoder compresses these into a 6x6 latent representation (a bit vector). To ensure this representation remains discrete—a requirement for symbolic planning—the system employs a Gumbel-Softmax activation function. This allows the system to treat visual states as sets of binary predicates.

2. Unsupervised Action Learning

By analyzing sequences of these latent bit vectors, the system identifies transitions. If state A leads to state B, an "Action Learner" compares the vectors to infer the preconditions and effects of the transition. This process effectively "discovers" the rules of the domain and automatically generates a PDDL model without human intervention.

3. Goal Recognition

With the learned model in place, the system can take a new sequence of images and apply off-the-shelf goal recognition algorithms, such as landmark-based heuristics, to predict the most likely goal. Additionally, the framework supports a purely data-driven alternative using a Long Short-Term Memory (LSTM) network that learns to predict goal states directly from the latent sequence, bypassing the symbolic PDDL step.

Experimental Evaluation

The framework was evaluated using datasets derived from the LatPlan framework, featuring simple games and puzzles. Training data consisted of plan traces generated by automated planners, while evaluation scenarios involved partially observable traces (with 10% to 70% of steps missing) to test robustness.

Key results include:

  • Feasibility: The system successfully derived functional PDDL domains from image evidence alone.
  • Performance: LatRec proved capable of identifying goals even under conditions of low observability, comparing favorably against three state-of-the-art goal recognition approaches.
  • Interpretability: Because the system uses an autoencoder, recognized goals in the latent space can be passed through the decoder to generate a visual representation of the predicted goal, allowing for human verification.

Discussion and Future Work

LatRec demonstrates that autoencoders can serve as a bridge between subsymbolic pixels and symbolic predicates. By learning domain theory from data streams, the system is naturally more resilient to sensor noise than rigid, hand-coded models.

However, several limitations remain. The current system operates under a closed-world assumption, treating goal recognition as a classification task limited to goals seen during training. Additionally, the Action Learner can generate redundant PDDL actions, which may impact efficiency. Future research will focus on moving toward probabilistic predicates to handle uncertainty more gracefully and scaling the framework from simple 2D grids to high-fidelity 3D environments.

Summary updated

BibTeX
@inproceedings{amado2020latrec,
  author = {Leonardo Amado and Felipe Meneguzzi},
  title = {{LatRec: Recognizing Goals in Latent Space (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i10.7145},
}
Figure