Structural Patent Classification Using Label Hierarchy Optimization

Mengting Gui, Shufeng Hao, Chongyang Shi, Qi ZhangEMNLP 2025

TL;DR

This paper introduces CSPC-LA, a novel method for patent classification that leverages the intricate structural relationships within patent claims and optimizes the hierarchical structure of patent labels. By modeling claim citations and co-references, and by refining the International Patent Classification (IPC) hierarchy, CSPC-LA significantly improves classification accuracy, particularly for less frequent (long-tail) categories.

What Problem Does This Paper Address?

Patent classification is crucial for efficient patent examination and understanding technological landscapes. Existing methods often rely on superficial text like titles and abstracts, neglecting the rich, detailed information embedded in patent claims. Furthermore, they treat classification labels as independent, failing to exploit the inherent hierarchical structure of patent taxonomies like the IPC. This leads to suboptimal performance, especially for the vast number of "long-tail" categories that have few examples, making it difficult to accurately classify emerging or niche technologies. The paper aims to bridge this gap by developing a model that deeply understands patent content and its structured label system.

Methods

The CSPC-LA model comprises two main modules: Structural Text Representation and Label Aware Representation Learning.

  • Structural Text Representation:
    • Claim Selection: To manage the complexity of patent claims, the model first selects a subset of the most relevant claims by concatenating the title and abstract to form a query and then using a BERT-based model to find claims with high semantic similarity. This focuses on high-information-density text.
  • Claim Citation Reconstruction: To ensure no critical citation links are lost after claim selection, a reconstruction process traces original citation links to find a valid claim within the selected subset.
  • Structural Graph Learning: Two graphs are constructed: a citation graph (representing parent-child relationships in claims) and a co-reference graph (connecting claims that cite the same parent). These graphs are processed using Graph Attention Networks (GATs) to capture the internal logical and semantic dependencies within claims.
  • Information Fusion: Representations from the patent's title/abstract and the structural graph learning module are fused using an adaptive mechanism to create a comprehensive patent representation.
  • Label Aware Representation Learning:
    • Label Hierarchy Learning: The IPC hierarchy is optimized to address class imbalance and structural disparities. This involves reconstructing the hierarchy into a fixed-height "coding tree" using structural entropy to measure and reduce redundancy.
  • Propagation Tree Learning: The optimized label hierarchy is used to guide patent representation. A propagation mechanism allows patent representations to interact with the label hierarchy, enriching the patent embeddings with semantic and structural label information.

Datasets

The paper introduces a new dataset called USPTO-2024, constructed from the USPTO's 2024 raw patent application data. This dataset includes full claims, citation structures, titles, abstracts, and IPC labels, providing a more comprehensive resource than previous datasets that might have excluded claim texts.

Key Insights and Ideas

  • Claims are King (for Classification): The paper strongly advocates for the critical role of patent claims, not just titles and abstracts, in achieving accurate patent classification. The detailed technical information and logical structure within claims are essential.
  • Structure Matters (Both Text and Labels): Both the internal structure of patent claims (via citation and co-reference graphs) and the hierarchical structure of patent labels (IPC) are vital for effective classification. Ignoring either leads to performance degradation.
  • Long-Tail Problem is Real and Solvable: The paper highlights the significant challenge of classifying infrequent (long-tail) patent categories and demonstrates that their proposed method, CSPC-LA, significantly outperforms baselines in this area.
  • Optimizing Label Hierarchy is Crucial: Simply using the raw IPC hierarchy can lead to imbalanced learning. Reconstructing and optimizing this hierarchy (e.g., into a fixed-height tree) is key to mitigating bias towards high-frequency classes.

Key Results

CSPC-LA achieved state-of-the-art performance on the USPTO-2024 dataset.

  • Overall Performance: CSPC-LA achieved the best TOP@1 and TOP@5 precision, recall, and F1-scores among all tested baselines, including traditional methods (TF-IDF+LR, BiLSTM), patent-specific models (DeepPatent, BERT-for-patents, PatentSBerta), and hierarchical multi-label models (HARNN, HiTIN). For instance, it achieved 46.85% precision at TOP@1 and 24.44% F1 at TOP@1.
  • Long-Tail Performance: The model showed superior performance on the bottom 100 rarest classes compared to TF-IDF and HiTIN, indicating its effectiveness in handling imbalanced data.
  • Ablation Studies: Removing key components like claim selection (w/o Method) or graph learning (w/o Graph) led to significant performance drops, validating the contribution of each module. Optimizing the label tree height to H=3 was found to be optimal.

Limitations and Future Work

  • Data Limitations: While using 2024 data, some labels still lacked sufficient samples due to year limitations. Future work could involve processing data from other years to improve diversity.
  • Scope of Relationships: The current work focuses on sentence-level claim relationships. Future work could explore more detailed elaborations of dependent claims around invention objects (entities) and consider macro-level relationships between patents.
  • Computational Constraints: Training speed is limited by equipment and data volume. Future work will focus on parallel operation optimization to accelerate training.
  • Large Language Models (LLMs): The current model does not utilize LLMs due to computational constraints. Future work plans to explore their application for enhanced performance.
  • Entity-Level Relations: The authors aim to explore entity-level structural relations within patents, viewing claim dependencies as semantic elaborations between entities.

Summary updated

BibTeX
@inproceedings{gui2025structural,
  author = {Mengting Gui and Shufeng Hao and Chongyang Shi and Qi Zhang},
  title = {{Structural Patent Classification Using Label Hierarchy Optimization}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure