Abstract
Conceptual analysis in philosophy often relies on manual examination of texts, a process that is difficult to scale. Traditional computational approaches typically use keyword heuristics, which suffer from low recall due to implicit mentions and polysemy. This paper proposes using Latent Dirichlet Allocation (LDA) to detect concept presence through thematic context. Evaluating on a corpus of French legal decisions, the authors demonstrate that topic-based heuristics significantly outperform keyword baselines, particularly when the specific concept keyword is absent from the text.
Introduction
In philosophy, "conceptual analysis" involves examining how a concept (such as JUSTICE or BROTHER) is used across a body of work to clarify its meaning. Traditionally an "armchair" process, scaling this analysis requires computational tools. However, standard Information Retrieval (IR) techniques are often insufficient. While IR seeks relevance (the best few documents), conceptual analysis seeks presence—every instance where a concept is invoked, however subtly. Keyword-based searches fail to capture implicit mentions, polysemous uses, and latent presence where a concept is part of the background knowledge but not explicitly named.
Methodology
The authors propose using Topic Models, specifically Latent Dirichlet Allocation (LDA), as a bridge to detect these "large concept extensions." The core hypothesis is that while a concept can be expressed in many ways, those expressions are conditioned by latent variables (topics). If a topic like "Family Law" is present, the concept BROTHER is likely nearby, even if the word itself is missing.
Two versions of LDA were implemented: Online Learning (Hoffman’s algorithm) for large datasets and Gibbs Sampling for token-level topic assignment. To link topics back to concepts, the authors tested six heuristics. The most successful was the Top 30 Rank, which considers a concept "present" if its representative keyword is among the top 30 most likely words for any topic associated with that text segment.
Experimental Setup and Annotation
The study utilized a corpus of 5,229 French-language legal decisions from the Quebec Court of Appeal. Legal arguments provide an ideal testbed due to their structured and nuanced nature. To establish a gold standard, the authors compared annotations from legal experts against crowdsourced participants from CrowdFlower.
A significant finding in the annotation process was the "reuse rate" bias. Non-experts were three times more likely than experts to reuse words found verbatim in the text as concept tags. As shown in the contingency table above, experts identified concepts as "present" in many instances (24 cases) where crowdsourced participants did not, suggesting that experts are more comfortable identifying abstract concepts that aren't explicitly written down.
Results and Discussion
The LDA-based methods outperformed the keyword baseline across all metrics, primarily by significantly boosting recall.
- Keyword Baseline: Achieved near-perfect precision (1.00) but abysmal recall (0.01), as it only identifies concepts when the exact word is present.
- Online LDA (Top 30 Rank): Improved the F1-score from 0.04 to 0.29 (experts) and 0.32 (crowd). Recall jumped from 0.01 to a range of 0.15–0.18.
- The "Keyword Absent" Scenario: In segments where the specific keyword was missing, the Online LDA method maintained an F1 of 0.30, whereas the keyword baseline naturally fell to 0.00.
These results demonstrate that "themes" can effectively serve as proxies for concepts that keywords miss. However, the absolute recall remains under 20%, indicating that topic models alone are insufficient to capture all forms of conceptual expression.
Conclusion
By moving beyond simple keyword matching to topic-based heuristics, this research provides a more robust framework for computational conceptual analysis. While the system still misses a majority of conceptual expressions, it represents a significant step forward in detecting latent presence. Future work may involve hybrid models integrating Named Entity Recognition (NER), coreference resolution, and modern word embeddings to further refine semantic representation.
Summary updated