News2vec: News Network Embedding with Subnode Information

Ye Ma, Lu Zong, Yikang Yang, Jionglong Su

Abstract

Traditional NLP models often treat news articles as isolated text sequences, neglecting the rich structural connections between events. We present News2vec, a novel embedding framework that represents news articles as nodes within a heterogeneous network of shared entities, actions, and metadata. By decomposing articles into a "bag of features" via a Subnode model, News2vec captures inter-textual relationships and enables the inductive embedding of unseen articles without retraining the entire graph. Evaluations in stock movement prediction and news recommendation demonstrate that News2vec outperforms standard baselines like BERT and Doc2vec.

Introduction

Most standard NLP models—such as BERT or Word2vec—process news articles as independent sequences of words. While powerful, these models often miss the "big picture": the structural connections between different news events. For example, two articles about different corporate mergers are linked by the concept of "finance" or specific shared entities like "Alibaba."

Existing event embedding models attempt to capture this by extracting (Actor, Predicate, Object) tuples, but they often struggle with flexibility—specifically, they find it difficult to generate vectors for new, "unseen" news stories that weren't part of the original training graph. News2vec bridges this gap by creating a systematic approach that integrates inherited features (like sentiment and topic) with inter-textual knowledge (how stories relate to one another).

Methodology

The News2vec pipeline consists of three main stages designed to transform raw news into a structured vector space.

1. Network Construction

The framework builds a heterogeneous network where "News Nodes" are connected to "Element Nodes" (entities, actions, and nouns extracted via TF-IDF). The edges are weighted based on the importance of the element to the story, creating a web of semantic and structural relationships.

Heterogeneous news network showing connections between articles, entities, and actions.

2. Biased Random Walks

Using a technique inspired by node2vec, the model performs random walks across the network. By tuning hyperparameters ($p$ and $q$), the walk can prioritize "local" context (similar news) or "global" exploration (latent relations across the network), ensuring the resulting embeddings capture both direct and indirect associations.

3. The Subnode Model

This is the paper's core innovation. Instead of learning a single monolithic vector for a news article, the model treats the article as a "bag of features" (semantic elements, word count, sentiment, publication date, etc.). The final news vector is calculated as the sum of its feature vectors.

The News2vec pipeline: from input news to subnode-based feature vectors.

This "sum-of-parts" approach, inspired by the Subword (FastText) model, allows the system to generate an embedding for a brand-new article simply by extracting its features and summing their pre-trained vectors, providing the model with vital inductive capabilities.

Experimental Evaluation

The researchers evaluated News2vec across three distinct domains: Chinese text classification (THUCTC), stock movement prediction (Sohu Financial News), and news recommendation (Browsing Records).

Key Results

  • Stock Movement Prediction: Using an LSTM with an attention mechanism, News2vec+ (the version incorporating extra features like sentiment) achieved an accuracy of 46.24% and a Matthews Correlation Coefficient (MCC) of 0.1936, outperforming BERT (45.83% Acc / 0.1442 MCC).
  • News Recommendation: In predicting user click-through behavior, News2vec reached a success rate of 7.53%, nearly doubling the performance of the standard Paragraph Vector (Doc2vec) baseline (4.31%).

Discussion and Insights

News2vec demonstrates that news is best understood as a composite rather than a monolith. By defining an article as the sum of its subnodes, the model gains the ability to understand articles it has never seen before, provided it has encountered their constituent entities or topics. Furthermore, the network approach captures "latent" connections; if two articles link to the same entity node (e.g., "Suning"), their vectors will be closer in the embedding space even if they share few words.

Limitations

The model's quality relies heavily on the initial extraction of elements via TF-IDF. If key entities are missed, network connections are lost. Additionally, the authors suggest that incorporating more "artificial labels," such as finer-grained topic tags or explicit supervised signals, could further sharpen the embeddings for specific downstream tasks.

Summary updated

Abstract

With the development of NLP technologies, news can be automatically categorized and labeled according to a variety of characteristics, at the same time be represented as low dimensional embeddings. However, it lacks a systematic approach that effectively integrates the inherited features and inter-textual knowledge of news to represent the collective information with a dense vector. With the aim of filling this gap, the News2vec model is proposed to allow the distributed representation of news taking into account its associated features. To describe the cross-document linkages between news, a network consisting of news and its attributes is constructed. Moreover, the News2vec model treats the news node as a bag of features by developing the Subnode model. Based on the biased random walk and the skip-gram model, each news feature is mapped to a vector, and the news is thus represented as the sum of its features. This approach offers an easy solution to create embeddings for unseen news nodes based on its attributes. To evaluate our model, dimension reduction plots and correlation heat-maps are created to visualize the news vectors, together with the application of two downstream tasks, the stock movement prediction and news recommendation. By comparing with other established text/sentence embedding models, we show that News2vec achieves state-of-the-art performance on these news-related tasks.
BibTeX
@inproceedings{ma2019news,
  author = {Ye Ma and Lu Zong and Yikang Yang and Jionglong Su},
  title = {{News2vec: News Network Embedding with Subnode Information}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  doi = {10.18653/v1/d19-1490},
}
Figure