Hierarchical Neural Network for Extracting Knowledgeable Snippets and Documents

Ganbin Zhou, Rongyu Cao, Xiang Ao, Ping Luo, Fen Lin, Leyu Lin, Qing HearXiv 2018 · arXiv

Introduction

The proliferation of social media and "We-media" platforms has created a vast digital landscape, yet only a small fraction of this content contains high-quality, structured information. While traditional knowledge base construction (e.g., YAGO, DBpedia) relies on semi-structured sources like Wikipedia, tapping into the broader web requires identifying "knowledgeable" content. This paper addresses two primary challenges: identifying knowledgeable documents (articles where over 50% of the content provides structured information) and extracting knowledgeable snippets (sentences that define concepts or describe entity properties).

Previous methods often relied on rigid Hearst patterns or domain-specific models, which are computationally expensive and fail to capture the nuanced semantics of natural language. This research introduces the Sharing-Splitting Neural Network (SSNN), a hierarchical architecture designed to generalize across domains while maintaining high extraction accuracy.

The SSNN Architecture

The SSNN operates on a hierarchical principle, reflecting the natural structure of documents. It consists of two primary levels:

  1. Word-to-Sentence Level ($CNN_1$): This layer transforms word embeddings into dense sentence embeddings.
  2. Sentence-to-Document Level ($CNN_2$): This layer aggregates sentence embeddings into a single document-level vector.
The SSNN architecture featuring low-level sharing and high-level splitting.

The core innovation is the Sharing-Splitting mechanism. The $CNN_1$ and $CNN_2$ layers are shared across all domains (e.g., Finance, Auto, Real Estate), based on the hypothesis that the linguistic "essence" of knowledgeable writing is universal. Conversely, the final softmax layers are split, allowing domain-specific classification to account for unique vocabularies and stylistic nuances. This architecture reduces memory requirements by approximately 66.7% when training on three domains compared to independent models.

Linguistic Signatures of Knowledge

A key finding of this research is that "knowledgeable" text possesses a distinct structural signature that transcends subject matter. By analyzing Part-of-Speech (POS) patterns, the authors identified that exposition and scientific texts—which typically contain definitions and quantifiers—are the primary carriers of knowledge.

Linguistic characteristics of different text types and their classification as knowledgeable.

As shown in the table above, knowledgeable snippets are characterized by general and proper nouns, verbs, and numerals. In contrast, narrative, critical, and lyrical styles rely more heavily on time words, pronouns, and interjections, which the model learns to filter out. To extract specific snippets, the authors employ a saliency-based approach, calculating the derivative of the loss function with respect to each sentence to identify those most critical to the document's "knowledgeable" classification.

Experimental Evaluation

The model was evaluated using a real-world dataset from the WeChat public platform, covering the Automobile, Finance, and Real Estate domains. Each domain consisted of 1,000 manually labeled documents.

Key Results

  • Superior Performance: SSNN consistently outperformed standard CNNs and SVM baselines. On a mixed dataset (AFR), the model achieved an ROC of 0.9015 and an accuracy of 0.8689.
  • Pooling Strategy: The researchers found that average pooling outperformed 1-max pooling, likely because it better preserves the semantic context of the entire document, which is essential for identifying intent.
  • Generalization: By sharing low-level features, the SSNN avoided "domain slip," where a model trained on one topic fails when applied to another.

Conclusion and Future Work

The SSNN provides an efficient and accurate framework for distilling knowledge from the noisy web. While the current model relies on a binary definition of knowledge and requires manual labeling, future work aims to explore continuous "knowledge density" scores and finer-grained extraction of Subject-Predicate-Object triples to directly populate knowledge graphs.

Summary updated

Abstract

In this study, we focus on extracting knowledgeable snippets and annotating knowledgeable documents from Web corpus, consisting of the documents from social media and We-media. Informally, knowledgeable snippets refer to the text describing concepts, properties of entities, or relations among entities, while knowledgeable documents are the ones with enough knowledgeable snippets. These knowledgeable snippets and documents could be helpful in multiple applications, such as knowledge base construction and knowledge-oriented service. Previous studies extracted the knowledgeable snippets using the pattern-based method. Here, we propose the semantic-based method for this task. Specifically, a CNN based model is developed to extract knowledgeable snippets and annotate knowledgeable documents simultaneously. Additionally, a "low-level sharing, high-level splitting" structure of CNN is designed to handle the documents from different content domains. Compared with building multiple domain-specific CNNs, this joint model not only critically saves the training time, but also improves the prediction accuracy visibly. The superiority of the proposed method is demonstrated in a real dataset from Wechat public platform.
BibTeX
@misc{zhou2018hierarchical,
  author = {Ganbin Zhou and Rongyu Cao and Xiang Ao and Ping Luo and Fen Lin and Leyu Lin and Qing He},
  title = {{Hierarchical Neural Network for Extracting Knowledgeable Snippets and Documents}},
  year = {2018},
  eprint = {1808.07228},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1808.07228},
}
Figure