Abstract
Traditional NLP models often treat news articles as isolated text sequences, neglecting the rich structural connections between events. We present News2vec, a novel embedding framework that represents news articles as nodes within a heterogeneous network of shared entities, actions, and metadata. By decomposing articles into a "bag of features" via a Subnode model, News2vec captures inter-textual relationships and enables the inductive embedding of unseen articles without retraining the entire graph. Evaluations in stock movement prediction and news recommendation demonstrate that News2vec outperforms standard baselines like BERT and Doc2vec.
Introduction
Most standard NLP models—such as BERT or Word2vec—process news articles as independent sequences of words. While powerful, these models often miss the "big picture": the structural connections between different news events. For example, two articles about different corporate mergers are linked by the concept of "finance" or specific shared entities like "Alibaba."
Existing event embedding models attempt to capture this by extracting (Actor, Predicate, Object) tuples, but they often struggle with flexibility—specifically, they find it difficult to generate vectors for new, "unseen" news stories that weren't part of the original training graph. News2vec bridges this gap by creating a systematic approach that integrates inherited features (like sentiment and topic) with inter-textual knowledge (how stories relate to one another).
Methodology
The News2vec pipeline consists of three main stages designed to transform raw news into a structured vector space.
1. Network Construction
The framework builds a heterogeneous network where "News Nodes" are connected to "Element Nodes" (entities, actions, and nouns extracted via TF-IDF). The edges are weighted based on the importance of the element to the story, creating a web of semantic and structural relationships.
2. Biased Random Walks
Using a technique inspired by node2vec, the model performs random walks across the network. By tuning hyperparameters ($p$ and $q$), the walk can prioritize "local" context (similar news) or "global" exploration (latent relations across the network), ensuring the resulting embeddings capture both direct and indirect associations.
3. The Subnode Model
This is the paper's core innovation. Instead of learning a single monolithic vector for a news article, the model treats the article as a "bag of features" (semantic elements, word count, sentiment, publication date, etc.). The final news vector is calculated as the sum of its feature vectors.
This "sum-of-parts" approach, inspired by the Subword (FastText) model, allows the system to generate an embedding for a brand-new article simply by extracting its features and summing their pre-trained vectors, providing the model with vital inductive capabilities.
Experimental Evaluation
The researchers evaluated News2vec across three distinct domains: Chinese text classification (THUCTC), stock movement prediction (Sohu Financial News), and news recommendation (Browsing Records).
Key Results
- Stock Movement Prediction: Using an LSTM with an attention mechanism, News2vec+ (the version incorporating extra features like sentiment) achieved an accuracy of 46.24% and a Matthews Correlation Coefficient (MCC) of 0.1936, outperforming BERT (45.83% Acc / 0.1442 MCC).
- News Recommendation: In predicting user click-through behavior, News2vec reached a success rate of 7.53%, nearly doubling the performance of the standard Paragraph Vector (Doc2vec) baseline (4.31%).
Discussion and Insights
News2vec demonstrates that news is best understood as a composite rather than a monolith. By defining an article as the sum of its subnodes, the model gains the ability to understand articles it has never seen before, provided it has encountered their constituent entities or topics. Furthermore, the network approach captures "latent" connections; if two articles link to the same entity node (e.g., "Suning"), their vectors will be closer in the embedding space even if they share few words.
Limitations
The model's quality relies heavily on the initial extraction of elements via TF-IDF. If key entities are missed, network connections are lost. Additionally, the authors suggest that incorporating more "artificial labels," such as finer-grained topic tags or explicit supervised signals, could further sharpen the embeddings for specific downstream tasks.
Summary updated