Abstract
Traditional computational approaches to narrative understanding often prioritize verbs and explicit event nominals. This paper proposes a shift in focus toward the "latent narratives" carried by physical objects. By formalizing the concepts of "habitats" and "affordances," the authors demonstrate how objects encode conventionalized events and environmental constraints. This framework allows AI systems to infer complex stories and identify narrative subversions—such as the transition from a functional artifact to resultative debris—without relying solely on explicit action descriptions.
Introduction: Beyond the Verb-Centric Model
In the fields of computational linguistics and computer vision, the standard method for event recognition involves identifying verbs (actions) or event nominals (e.g., "the explosion"). However, this approach frequently overlooks the rich, implied context provided by the objects within a scene. A "chainsaw" is never just a static object; it carries an inherent narrative of "cutting."
The authors argue that objects are not passive participants but are instead "narrative anchors" that encode specific purposes and physical requirements. The challenge lies in computationally representing this structure to help machines understand not just what is happening, but the deeper story of why and how objects are situated in a specific environment.
Theoretical Framework: GL and VoxML
To model how objects reference events, the authors integrate two primary theoretical frameworks:
- Generative Lexicon (GL): This linguistic theory utilizes "qualia structure" to define objects. The authors specifically highlight the TELIC role, which defines the purpose or function of an object (e.g., the telic role of a plane is to fly; a chainsaw is to cut).
- Visualization Modeling Language (VoxML): To ground these definitions in physical space, the authors use VoxML to encode Habitats. A habitat represents the set of environmental and situational constraints required for an object to fulfill its telic role.
As illustrated in the figure above, the same action—"spraying"—takes on entirely different narrative meanings based on the objects involved. By mapping the Agent, Source, and Substance, the system can distinguish between a rescue operation (firefighting) and a creative or transgressive act (graffiti).
Habitats and Narrative Anchors
A key insight of this research is that functionality is a state, not just a label. By modeling the spatial requirements of an object, AI can predict future actions or identify when a narrative has been violated. For instance, a plane in a hangar follows a latent narrative of readiness. Conversely, plane debris in a field violates that narrative, forcing an observer to infer a "resultative" event: a crash.
This distinction is critical for machine reasoning. A chainsaw resting on a stand (as seen in the left of the figure above) exists in a state of potentiality. It is only when the object is brought into its proper habitat—contact with a tree—that the latent narrative of "cutting" is realized. By composing the latent event structures of multiple objects (e.g., a dryer and folded laundry), a machine can construct a coherent narrative of a "cleaning process" even in the absence of explicit verbs.
Conclusion and Future Directions
This position paper establishes a theoretical foundation for treating objects as complex narrative entities. While the authors are currently annotating major datasets like Flickr30k and VisualGenome to test these theories, they acknowledge significant hurdles in scalability. Future work will focus on automating the extraction of "qualia relations" from large-scale text and image corpora. Ultimately, understanding the "canonical" story of an object may allow AI to detect subtle biases in reporting or identify nuanced differences in how various perspectives describe the same physical results.
Summary updated