TL;DR
This paper argues that current opinion mining (sentiment analysis) techniques are too focused on inferring a speaker's internal "private state" and fail to capture the crucial role of the reader's perspective in determining opinion. They propose the concept of "metasubjectivity" and suggest that future progress requires developing models that account for this reader-centric view, even if it means sacrificing some short-term performance gains.
What Problem Does This Paper Address?
Opinion mining, or sentiment analysis, aims to automatically detect and extract subjective information from text. While it has seen rapid growth, the authors argue that existing approaches are fundamentally limited. They are primarily designed to infer an author's internal "private state" or opinion, treating it as an inherent characteristic of the text. This works reasonably well for straightforward cases like product reviews, but it struggles with more complex scenarios. The paper highlights that in many real-world applications, such as market prediction or social science research, understanding how a reader interprets an opinion is as important, if not more so, than understanding the original speaker's intent. This gap, the authors call "metasubjectivity," is the core problem they address: the subjectivity of subjectivity itself, where the meaning and polarity of an opinion are heavily influenced by the perspective of the person receiving or interpreting it.
Methods
The paper doesn't propose a new algorithm or model in the traditional sense. Instead, it offers a conceptual shift and a framework for future research. The key methodological contribution is the introduction and exploration of "metasubjectivity." This concept emphasizes that the interpretation of an opinion is not solely determined by the speaker but is heavily mediated by the "aggregate reader" – a hypothetical reader with a presumed interest and context relevant to the domain of the text.
They illustrate this with examples:
- A sentence about the price difference between a Blackberry and an iPhone might be a factual statement, but it becomes a negative opinion about the iPhone when viewed through the lens of a consumer comparing prices.
- A statement about "virtualization" in an IT business journal could be neutral to a casual reader but negative to an investor concerned about network access control issues, or even positive to an IT solutions provider.
The paper argues that current evaluation methods, which rely on "gold standard" data produced by expert judges or crowdsourced annotators, are ill-equipped to handle metasubjectivity. This is because these methods often assume an underlying objective truth or a shared understanding of what constitutes an opinion, which breaks down when the reader's perspective is paramount. They suggest that future work needs to develop annotation processes and models that explicitly account for this aggregate reader perspective, potentially through more complex user interfaces or by collecting data from a larger, more diverse set of annotators.
Datasets
The paper discusses the limitations of existing datasets and evaluation procedures rather than introducing new ones. It references:
- Somasundaran and Wiebe (2009) for an example of "pragmatic opinion" related to cell phone preferences.
- de Marneffe et al. (2012) for their work on veridicality assessment.
- IT business press articles as a domain for their analysis and proposed annotation challenges.
The authors note that existing "gold standard" datasets are often produced using methods standard in information retrieval, which assume objectivity, and that these are not ideal for capturing metasubjectivity. They also mention their own attempts to annotate IT business press articles, which resulted in very low inter-annotator agreement, highlighting the difficulty of defining opinion consistently when metasubjectivity is involved.
Key Insights and Ideas
- The reader is king (or queen): The most significant insight is that the interpretation and polarity of an opinion are not solely determined by the speaker's intent but are profoundly shaped by the reader's perspective, context, and presumed goals.
- "Metasubjectivity" is the missing piece: Current opinion mining models are too focused on the "private state" of the speaker and neglect the "subjectivity of subjectivity" – how different readers will interpret the same statement differently.
- Current evaluation methods are flawed: Traditional "gold standard" annotation, relying on expert consensus, struggles with metasubjectivity because there's no single, objective truth about what constitutes an opinion in many contexts.
- The "aggregate reader" is a useful abstraction: To move forward, we need to model the perspective of a hypothetical "aggregate reader" who represents the typical or relevant audience for a given text domain.
- Pragmatic knowledge is key: Understanding opinions requires more than just linguistic cues; it necessitates world knowledge and an understanding of how language is used in context to influence others.
Key Results
The paper is primarily a position piece and a call for a paradigm shift, rather than a report of empirical results. Therefore, there are no quantitative performance metrics or specific "results" in the traditional sense. The key "finding" is the conceptualization of metasubjectivity and the demonstration of its impact through illustrative examples. The authors' experience with annotating IT business press articles serves as a qualitative result, showing very low inter-annotator agreement, which they attribute to the inherent metasubjectivity of the task.
Limitations and Future Work
Limitations: * Conceptual focus: The paper is more theoretical and doesn't present a concrete, implemented solution or a new algorithm. * Performance trade-offs: The authors acknowledge that adopting techniques to handle metasubjectivity will likely lead to a temporary loss or damage of incremental performance gains achieved by current methods. * Annotation challenges: Developing new annotation procedures and corpora that effectively capture metasubjectivity is a significant undertaking.
Future Work: * Developing "aggregate reader" models: Research needs to focus on how to define and model this aggregate reader perspective. * Rethinking annotation processes: New methods are needed to collect data that accounts for reader variability, potentially using crowdsourcing in novel ways or designing more sophisticated annotation interfaces. * Exploring syntactic analysis: The paper suggests that more advanced syntactic analysis techniques might be necessary to capture the nuanced ways opinions are expressed and interpreted, especially when dealing with arbitrary non-compositional structures. * Building new corpora: The creation of new training corpora and related resources specifically designed to address metasubjectivity is essential.
Summary updated