TL;DR
This paper introduces a novel annotation scheme and methodology for identifying "animate entities" (characters, talking objects, etc.) within comic book panels. By analyzing annotator agreement on visual segmentation and animacy classification, the study reveals that animacy is best understood as a graded concept and that ambiguity can be a deliberate narrative tool.
What Problem Does This Paper Address?
Comics are a rich, multi-modal medium where narrative is conveyed through a complex interplay of text and images. While computational approaches to understanding narrative structures in comics are emerging, a significant gap exists in systematically identifying and annotating key narrative elements like animate entities. These entities are crucial for understanding character development, inferring actions and intentions from static images, and ultimately, for building comprehensive computational models of comic narratives. Existing annotation schemes often focus on text or broad character identification, leaving the nuanced visual representation of animacy under-addressed. This paper aims to fill this gap by developing and evaluating an annotation process specifically for visually identifying animate entities in comic panels.
Methods
The research employs a two-phase annotation experiment:
- Experiment 1: Visual Segmentation: Annotators were tasked with outlining animate entities within comic panels using a polygon segmentation tool. This approach aimed to capture annotator judgments without pre-defined assumptions about what constitutes an animate entity. The primary metric for agreement was the Intersection over Union (IOU) score, a standard measure of overlap between segmented regions.
- Experiment 2: Animacy Classification: Building on the segmentation results, annotators were then asked to assign a hierarchical animacy type to the pre-outlined entities. This hierarchy ranged from "Human" to "Human-like," "Animal-like," and "Not sure if animate," aiming to move beyond a binary classification and capture more nuanced judgments. Krippendorff's alpha (KA) was used as the primary inter-annotator agreement metric for this classification task.
Datasets
The study utilized four comic stories from the "Alarming Tales" magazine (published 1957-1958). These comics were chosen for their public domain status, consistent artistic style, and genre (sci-fi fantasy), and importantly, for containing entities with potentially ambiguous animacy. The comics were accessed via Comic Book Plus.
Key Insights and Ideas
- Animacy as a Graded Concept: The research strongly suggests that animacy is not a simple binary (animate/inanimate) but rather a spectrum. The hierarchical classification in Experiment 2, and the analysis of disagreements, highlight that entities can possess varying degrees of animacy, often influenced by context and demonstrated behaviors.
- Intentional Movement as a Key Indicator: The study found that evidence of intentional movement is a foundational aspect of perceived animacy. Entities that demonstrated purposeful actions, even if not human-like, were more likely to be agreed upon as animate.
- Ambiguity as a Narrative Device: The presence of significant disagreement among annotators for certain entities (e.g., "plant-men") suggests that ambiguity in animacy can be a deliberate narrative technique. This uncertainty can engage readers, prompting them to infer more and progress through the story to resolve the tension.
- Visual Segmentation is a Viable First Step: The polygon outlining method in Experiment 1 proved effective in qualitatively measuring agreement and identifying areas of potential animacy, serving as a robust foundation for subsequent classification.
Key Results
- Experiment 1: Polygon-based outlines successfully produced a qualitative measure of agreement, with IOU scores generally indicating reasonable overlap. The study found that the threshold for sufficient overlap between outlines was around 0.7.
- Experiment 2: The hierarchical animacy assignment showed that ordinal weighting of the animacy categories yielded higher agreement than nominal weighting. Stories 2 and 4 achieved adequate agreement with ordinal weighting, while stories 1 and 3 showed lower agreement, particularly for entities like "plant-men" and "robot-plants" where the hierarchy struggled to capture their status. The "not sure if animate" category was frequently used for ambiguous cases.
Limitations and Future Work
Limitations:
- Small Comic Corpus: The study used only four comic stories, which is a limited sample size for the vast diversity of animate entities found in comics.
- Crowd-sourced Annotation: Reliance on remote, crowd-sourced annotation meant researchers couldn't directly instruct annotators or clarify misunderstandings in real-time.
- Reader-Centric Perspective: The annotation scheme was designed from the reader's perspective. This missed opportunities to capture how creators might link entities (e.g., a sofa transforming into a character) through co-reference, which would require a creator-centric view.
Future Work:
- Refining the Annotation Scheme: The authors plan to refine the annotation scheme for broader corpus analysis, potentially incorporating authorial intent alongside reader perspective.
- Multi-modal Integration: Integrating visual discourse referent annotation with linguistic reference annotation in comic text is a key next step for multi-modal discourse processing.
- Capturing Ambiguity: Developing methods to explicitly capture animacy ambiguity, perhaps by deriving a measure from annotator disagreements, is crucial for future narrative understanding research.
- Expanding the Hierarchy: The authors suggest adding a category for non-human, non-animal-like entities to better accommodate complex cases like sentient AI or abstract beings.
Summary updated