Abstract
Weakly supervised text classification aims to categorize documents using only category surface names, bypassing the need for expensive manual annotations. While most existing methods rely solely on textual content, many documents are accompanied by rich metadata—such as authors, venues, and publication years—that can serve as powerful category indicators. We introduce MotifClass, a framework that models these relationships through a Heterogeneous Information Network (HIN) and captures higher-order structures using "motifs." By selecting category-indicative motif instances and generating pseudo-labeled training data, MotifClass significantly outperforms state-of-the-art weakly supervised baselines.
Introduction
In the era of big data, the ability to classify vast document collections without human intervention is invaluable. Traditional supervised learning requires large labeled datasets, which are often unavailable or costly to produce. Weakly supervised methods offer a solution by using minimal input, such as category names (e.g., "Natural Language Processing" or "Data Mining").
However, a significant oversight in current research is the neglect of document metadata. In academic publishing, for instance, the combination of a specific author and a specific conference is often a much stronger signal of a paper's topic than any single word in its title. MotifClass addresses this by treating documents and their metadata as nodes in a Heterogeneous Information Network (HIN).
The Challenge: Specificity and Higher-Order Interactions
Two main challenges arise when using metadata for classification:
- Higher-order interactions: Single metadata fields (e.g., a year) are often weak indicators. However, combinations (e.g., Author + Venue) provide much higher precision.
- Metadata Specificity: Not all metadata is created equal. A term like "language" is too broad to uniquely identify an NLP paper, whereas a venue like "EMNLP" is highly specific.
MotifClass solves these by defining "motifs"—subgraph patterns that represent specific combinations of metadata—and evaluating their specificity.
The MotifClass Framework
The framework consists of three primary stages:
1. Indicative Motif Selection
MotifClass identifies which metadata combinations (motif instances) are most representative of a category. It uses a joint representation learning approach, embedding both motifs and category names into a shared space. To model specificity, the authors employ the von Mises-Fisher (vMF) distribution. In this model, the "concentration parameter" ($\kappa$) acts as a proxy for specificity: a high $\kappa$ indicates a motif instance that is tightly associated with a specific category, while a low $\kappa$ suggests a generic or "broad" motif.
2. Pseudo-labeled Data Collection
Once indicative motifs are identified, the system builds a training set through two methods:
- Retrieval: It identifies unlabeled documents in the corpus that contain these highly indicative motif instances.
- Generation: It synthesizes new document representations that are semantically aligned with the target categories, ensuring a diverse and voluminous training set.
3. Classifier Training
Finally, a standard text classifier (such as a Kim-CNN) is trained on this pseudo-labeled data. This allows the model to learn the linguistic patterns associated with the categories, guided by the high-quality labels derived from metadata motifs.
Experimental Results
The authors evaluated MotifClass on two large-scale datasets: MAG-CS (computer science papers) and Amazon (product reviews).
- Superior Accuracy: On the MAG-CS dataset, MotifClass achieved a Micro-F1 score of 0.571, outperforming the best metadata-aware baseline (0.525) and significantly beating text-only methods like WeSTClass.
- The Power of Higher-Order Data: Ablation studies revealed that nearly 50% of the most useful signals came from higher-order motifs rather than individual metadata fields.
- Efficiency: MotifClass is highly scalable. It processed the MAG-CS dataset 26.6 times faster than previous iterative motif-based approaches.
Conclusion
MotifClass demonstrates that metadata is not just "extra" information but a core component of document identity. By mathematically modeling the specificity of metadata combinations, MotifClass provides a robust, efficient, and highly accurate framework for text classification in the absence of human labels. Future work will explore extending this to multi-label and hierarchical classification tasks.
Summary updated