A Dependency Structure Annotation for Modality

Meagan Vigus, Jens E. L. Van Gysel, William CroftACL 2019

TL;DR

This paper introduces a novel dependency structure for annotating modality in text, moving beyond simple event-level labels to capture complex relationships. By representing modality as a graph of "conceivers" (sources of information) and "events" with weighted edges representing epistemic strength, this approach offers a more nuanced and linguistically grounded way to understand how certainty and perspective are conveyed in language.

What Problem Does This Paper Address?

Understanding the reality status of events in text – whether they are factual, hypothetical, desired, or believed – is crucial for a complete semantic representation. Existing annotation schemes often struggle to capture the intricate ways modality is expressed, especially when multiple sources of information or nested perspectives are involved. This paper addresses the need for a more robust annotation framework that can represent these complex modal relationships, building upon prior work in factuality annotation and temporal dependency structures.

Methods

The core innovation is a dependency structure annotation scheme for modality. This scheme represents:

  • Nodes:
    • Conceivers: Entities (e.g., the author, individuals mentioned in the text) whose perspective or mental content is relevant to an event. These are organized hierarchically, with the author as the root.
  • Events: The actions or states being described.
  • Edges: These represent the epistemic strength and polarity of the relation between a conceiver and an event, or between two events.
  • Epistemic Strength: Based on Boye's (2012) three levels (FULL, PARTIAL, NEUTRAL), which correspond to certainty, probability, and possibility. These are mapped to FactBank's values (CERTAIN, PROBABLE, POSSIBLE).
  • Polarity: A binary distinction (POSITIVE/NEGATIVE) to handle negation.
  • Special Relation: A "Same Space" (SAMESP) relation is introduced to indicate that two events share the same modal strength from the same conceiver.

This structure is inspired by mental spaces theory and adapts temporal dependency parsing techniques. The annotation process involves three passes: identifying events, building the "superstructure" of conceivers, and then annotating the modal relations between nodes.

Datasets

The authors annotated six documents containing a total of 377 events. These documents represent three genres: news stories, narratives, and discussion forums. The paper does not introduce a new benchmark dataset but rather applies its annotation scheme to existing text.

Key Insights and Ideas

  • Modality as a Dependency Graph: Representing modality as a dependency structure, rather than just assigning a label to an event, allows for the explicit modeling of nested modal expressions and scope relations. This is crucial for sentences with multiple modal verbs or complex embedding.
  • Unified Strength Framework: The paper proposes that epistemic strength can be interpreted across different modality types (evidential, deontic, dynamic) by focusing on the likelihood of an event's occurrence or generalization. This provides a more unified approach to annotating diverse modal phenomena.
  • Conceivers as Core Elements: The explicit representation of "conceivers" (sources of perspective) as nodes in the graph is key. This allows for tracking whose belief, desire, or knowledge is being attributed, mirroring insights from mental spaces theory.
  • Handling Ambiguity: The annotation scheme is designed to handle textual ambiguities, particularly in indirect speech, where the source of an event can be unclear.

Key Results

  • High Inter-Annotator Agreement: The annotation scheme achieved relatively high agreement scores (e.g., F-scores around 0.93 for event annotation across genres), comparable to or exceeding those reported in similar work on temporal relations.
  • Feasibility of the Scheme: The pilot annotation demonstrated that the scheme is relatively easy to implement and that annotators can consistently assess epistemic strength relations and relevant conceivers.
  • Identification of Error Types: The error analysis revealed common disagreements related to lexical item interpretation, the necessity of childless conceiver nodes, and scope relations, providing valuable feedback for refining the annotation guidelines.

Limitations and Future Work

  • Scope of Application: The current annotation scheme has only been applied to English texts, though the authors intend for it to be applicable cross-linguistically.
  • SAMESP Relation: The "Same Space" (SAMESP) relation, while useful for visualization, was found to introduce too many non-substantive errors and has been removed from the primary annotation scheme, though it can be reintroduced for visualization purposes.
  • Lexical Item Ambiguity: Disagreements often arose from the inherent ambiguity of certain lexical items in conveying specific modal strengths, suggesting a need for more detailed guidelines or further refinement of the strength categories.
  • Future Work: The authors suggest that the scheme could be extended to other languages and potentially integrated with other annotation layers (e.g., temporal, aspectual) for richer semantic representations. Further work on refining guidelines for ambiguous cases is also implied.

Summary updated

BibTeX
@inproceedings{vigus2019dependency,
  author = {Meagan Vigus and Jens E. L. Van Gysel and William Croft},
  title = {{A Dependency Structure Annotation for Modality}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure