TL;DR
This paper introduces a novel "homothetic clustering" approach to analyze the semantic evolution of Hafez's poetry, creating new unsupervised labels that complement existing chronological classifications. By treating similarity measures as features, they demonstrate a promising method for uncovering deeper poetic insights, especially for smaller, specialized corpora.
What Problem Does This Paper Address?
Literary scholars often rely on chronological classifications of poets' works to understand how their worldview and artistic expression might have evolved over time. For the renowned Persian poet Hafez, the gold standard for this is Mahmoud Houman's 1938 classification, which was largely manual and based on interpreting semantic nuances within his complex poetry. While this classification is invaluable, it's a single perspective. The authors aim to create alternative, unsupervised semantic labels for Hafez's poetry to offer new insights and a complementary lens for literary analysis, without relying on pre-existing chronological labels as direct training data. They want to see if machine learning can capture similar semantic shifts that Houman identified manually.
Methods
The core of their approach is homothetic clustering, a novel technique they developed. Instead of directly using features like TF-IDF or word embeddings, they first represent each poem using features derived from similarities between that poem and a set of "anchor" poems. These anchors are carefully chosen poems from Houman's classification (e.g., representative poems, or poems from the extremes of his timeline). This creates a new vector space where each poem's representation is based on its "similarity of similarities" to these anchors. They then apply standard K-Means clustering to this new feature space.
Other methods employed include:
- Preprocessing: Standard NLP techniques like tokenization, normalization, lemmatization, and filtering of stop-words and rare terms.
- Feature Engineering: They experimented with various standard features like TF-IDF, Latent Semantic Analysis (LSI), Latent Dirichlet Allocation (LDA), and Doc2Vec (both Persian-only and Persian-English).
- Clustering: Primarily K-Means, applied to the different feature sets.
- Evaluation: A suite of metrics including Silhouette score (for cluster cohesion and separation), Homogeneity, Completeness, V-measure, Adjusted Random Index (ARI), and Cohen's Kappa to compare their unsupervised clusters against Houman's gold-standard labels.
Datasets
- Hafez Corpus: The primary dataset is a collection of Hafez's ghazals. For their experiments, they specifically used a subset of 248 poems that had been chronologically labeled by Houman (1938). This subset was used for cross-referencing and evaluation.
- Houman's Labels: This is the "gold-standard" chronological classification used for evaluation.
- Anchor Poems: A subset of Houman's labeled poems were selected as anchors for the homothetic clustering approach.
Key Insights and Ideas
- Similarity of Similarities: The core idea of "homothetic clustering" is to leverage how similar poems are to other poems (anchors) as a new feature space. This allows for a different kind of semantic representation than direct word or topic features.
- Anchors Matter: The choice of anchor poems significantly influences the clustering results. Using representative poems from Houman's classes (HRP) or extreme poems (HEP) yielded better results than random anchors, suggesting the anchors effectively guide the clustering towards semantic groupings related to Houman's original classification.
- Complementary Labels: While their unsupervised clusters didn't achieve statistically significant kappa agreement with Houman's labels (indicating low direct consistency), they did show high Silhouette scores and provided new semantic perspectives. This suggests their labels are not a direct replacement but a valuable complement.
- Poetry Analysis Tool Potential: The authors envision their homothetic clustering framework, combined with visualization tools like LDAvis, as a powerful component for building automated poetry analysis tools, even for small corpora.
Key Results
- Homothetic Clustering Performance: The homothetic clustering approach (Sim²) generally achieved high Silhouette scores (up to 0.95) and good homogeneity/completeness, often outperforming standard word-embedding (Doc2Vec-P) and topic-modeling (LDA) features in these metrics.
- Anchor Influence: The HRP (Houman Representative Picks) and HEP (Houman Extremal Picks) anchor sets performed best, with HRP showing slightly better resemblance to Houman's classes as measured by homogeneity and completeness.
- Kappa vs. Silhouette: While Silhouette scores were high, indicating good internal cluster quality, the agreement with Houman's labels (measured by Kappa) was often low, suggesting the clusters captured different semantic dimensions than Houman's chronological ones.
- Doc2Vec-P as Challenger: Doc2Vec-P (Persian-only word embeddings) was a strong challenger, scoring well in coherence and silhouette, but the homothetic approach with specific anchors ultimately performed better in several key metrics.
Limitations and Future Work
- Kappa Agreement: The authors acknowledge that their homothetic clustering did not achieve statistically significant consistency (low kappa) with Houman's classifications, despite high Silhouette scores. This suggests the generated labels offer new semantic perspectives rather than directly replicating Houman's chronological evolution.
- Quasi-Semi-Supervised: The Sim² approach, while using anchors, is not strictly semi-supervised as it doesn't use Houman's labels for direct training. The low kappa suggests it didn't fully qualify as a "quasi-semi-supervised" algorithm in terms of direct alignment.
- Future Work:
- Broader Application: Replicating and applying this poetry clustering framework to other poetic texts to assist literary research.
- Tool Development: Building interactive poetry analysis tools that combine homothetic clustering with visualizations.
- More Anchors/Refinement: Further exploring anchor selection strategies and refining the similarity measures.
- Deeper Semantic Analysis: Conducting more rigorous literary analysis with the new labels.
Summary updated