No article text was recorded for this version.
annotator disagreement signal
Version 1 of 1129 papers180 conceptsfrozen 2026-10-04
Papers in this version
- Blog: Learning part-of-speech taggers with inter-annotator agreement loss
- Blog: Proceedings of Context and Meaning: Navigating Disagreements in NLP Annotation
- Blog: DH-FBK at SemEval-2022 Task 4: Leveraging Annotators’ Disagreement and Multiple Data Views for Patronizing Language Detection
- Blog: What to do about non-standard (or non-canonical) language in NLP
- Blog: xiacui at SemEval-2023 Task 11: Learning a Model in Mixed-Annotator Datasets Using Annotator Ranking Scores as Training Weights
- Blog: Predicting word sense annotation agreement
- Blog: Modeling Ambiguity with Many Annotators and Self-Assessments of Annotator Certainty
- Blog: Predicting Median, Disagreement and Noise Label in Ordinal Word-in-Context Data
- Blog: Supersense tagging with inter-annotator disagreement
- Blog: Beyond Averages: Learning with Annotator Disagreement in STS
- Blog: Learning to parse with IAA-weighted loss
- Blog: Difficulty-aware Glaucoma Classification with Multi-Rater Consensus Modeling
- Blog: Ambiguity Helps: Classification with Disagreements in Crowdsourced Annotations
- Blog: Learn2Agree: Fitting with Multiple Annotators Without Objective Ground Truth
- Blog: Learning Ambiguity from Crowd Sequential Annotations
- Blog: Dis2Dis: Explaining Ambiguity in Fact-Checking
- Blog: On the Challenges and Practices of Reinforcement Learning from Real Human Feedback
- Blog: AmbiFC: Fact-Checking Ambiguous Claims with Evidence
- Blog: A Collaborative Content Moderation Framework for Toxicity Detection based on Conformalized Estimates of Annotation Disagreement
- Blog: AmbiFC: Fact-Checking Ambiguous Claims with Evidence
- Blog: Studying Topical Relevance with Evidence-based Crowdsourcing
- Blog: How we do things with words: Analyzing text as social and cultural data
- Blog: VariErr NLI: Separating Annotation Error from Human Label Variation
- Blog: Efficient Annotator Reliability Assessment and Sample Weighting for Knowledge-Based Misinformation Detection on Social Media
- Blog: STABLEVAL: Disagreement-Aware and Stable Evaluation of AI Systems
- Blog: FuocChuVIP123 at CoMeDi Shared Task: Disagreement Ranking with XLM-Roberta Sentence Embeddings and Deep Neural Regression
- Blog: Perspectives on Hate: General vs. Domain-Specific Models
- Blog: Crowdsourcing Semantic Label Propagation in Relation Classification
- Blog: Building a Long Text Privacy Policy Corpus with Multi-Class Labels
- Blog: A Crowdsourced Frame Disambiguation Corpus with Ambiguity
- Blog: Content and Behaviour Based Metrics for Crowd Truth
- Blog: Crowdsourcing StoryLines: Harnessing the Crowd for Causal Relation Annotation
- Blog: Reasoning About Intent for Ambiguous Requests
- Blog: What to do about non-standard (or non-canonical) language in NLP
- Blog: CEI: A Benchmark for Evaluating Pragmatic Reasoning in Language Models
- Blog: Reports of the Workshops Held at the Sixth AAAI Conference on Human Computation and Crowdsourcing
- Blog: No Strong Feelings One Way or Another: Re-operationalizing Neutrality in Natural Language Inference
- Blog: The Treasure Beneath Multiple Annotations: An Uncertainty-Aware Edge Detector
- Blog: AIMS.au: A Dataset for the Analysis of Modern Slavery Countermeasures in Corporate Statements
- Blog: Don't waste a single annotation: improving single-label classifiers through soft labels
- Blog: Separating Value Disagreement from Data Uncertainty in Pluralistic Preference Data
- Blog: Disagreement in Argumentation Annotation
- Blog: Learn2Agree: Fitting with Multiple Annotators without Objective Ground Truth
- Blog: Supervised Uncertainty Quantification for Segmentation with Multiple Annotations
- Blog: Learn2Agree: Fitting with Multiple Annotators without Objective Ground Truth
- Blog: Modeling disagreement in automatic data labeling for semi-supervised learning in Clinical Natural Language Processing
- Blog: Why Don’t You Do It Right? Analysing Annotators’ Disagreement in Subjective Tasks
- Blog: Multi-Perspective Stance Detection
- Blog: An Annotated Corpus of Reference Resolution for Interpreting Common Grounding
- Blog: When Experts Disagree: Characterizing Annotator Variability for Vessel Segmentation in DSA Images
- Blog: CICL_DMS at SemEval-2023 Task 11: Learning With Disagreements (Le-Wi-Di)
- Blog: No Need to Sacrifice Data Quality for Quantity: Crowd-Informed Machine Annotation for Cost-Effective Understanding of Visual Data
- Blog: A Case for a Range of Acceptable Annotations
- Blog: What Can We Learn from Collective Human Opinions on Natural Language Inference Data?
- Blog: AmbiFC: Fact-Checking Ambiguous Claims with Evidence
- Blog: On Reference (In-)Determinacy in Natural Language Inference
- Blog: W-RST: Towards a Weighted RST-style Discourse Framework
- Blog: Learning from Uneven Training Data: Unlabeled, Single Label, and Multiple Labels
- Blog: Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators' Disagreement
- Blog: Mining WordNet for a Fuzzy Sentiment: Sentiment Tag Extraction from WordNet Glosses
Showing 60 of 129 papers.
History & citation
Current · Permanent link
Cite this direction
Latest version (a living page):
@misc{lacuna_annotator_disagreement_signal_4632f05b,
title = {annotator disagreement signal},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/annotator-disagreement-signal/rdr_4632f05bcb8e44308bb1ac783ed60ae7},
note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}This exact version (version 1, frozen text):
@misc{lacuna_annotator_disagreement_signal_4632f05b_v1,
title = {annotator disagreement signal},
author = {{Lacuna}},
howpublished = {Lacuna research direction},
year = {2026},
url = {https://lacuna.tiptreesystems.com/direction/annotator-disagreement-signal/rdr_4632f05bcb8e44308bb1ac783ed60ae7/v/1},
note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}