Introduction
Network representation learning aims to transform complex graph nodes into low-dimensional vectors (embeddings) suitable for downstream machine learning tasks. Traditional methods often focus exclusively on network structure or node content in isolation, creating information silos. TriDNR (Tri-Party Deep Network Representation) addresses these limitations by jointly learning node representations from three distinct sources: network structure (links), node content (text), and node labels.
By coupling these "tri-party" signals into a unified deep learning architecture, TriDNR leverages available labels as anchors to guide the embedding process. This semi-supervised approach significantly improves performance in tasks like node classification by ensuring that representations are sensitive to both the topology and the semantics of the graph.
Methodology
TriDNR utilizes a coupled neural network architecture based on the Skip-gram model. The learning process integrates information at three levels simultaneously:
- Network Structure Level: Using random walks, the graph is treated as a collection of "sentences" where nodes are "words." The model maximizes the probability of observing a node's neighbors to capture local topology.
- Node Content Level: Text associated with a node (e.g., a paper title) is treated as a document. The model learns a node vector optimized to predict the specific words within that node's content, capturing the correlation between nodes and vocabulary.
- Node Label Level: For labeled nodes, a "class vector" is learned that also predicts the words associated with that class. This forces node embeddings to align with the underlying categorical structure.
The model is optimized using Hierarchical Softmax, ensuring scalability to large networks by reducing computational complexity in the output layer. Unlike previous matrix factorization methods, TriDNR's neural approach captures word order and context more effectively while scaling linearly with the number of words.
Experimental Evaluation
The authors evaluated TriDNR on two major citation network benchmarks: DBLP (over 60,000 papers across 4 classes) and CiteSeer-M10 (approximately 10,000 publications across 10 classes). In these datasets, paper titles served as node content, and research areas served as labels.
Key Results
TriDNR consistently outperformed state-of-the-art competitors, including DeepWalk, LINE, and Doc2Vec. On the CiteSeer-M10 dataset, TriDNR achieved a Macro F1 score of 0.777, a significant improvement over DeepWalk's 0.434 (approximately a 79% relative gain) and the previous state-of-the-art TADW (0.681).
Visual analysis using t-SNE confirmed that TriDNR produces cleaner, more distinct clusters compared to unsupervised methods. This suggests that even a small amount of label information acts as a powerful anchor, guiding the embedding process to better reflect class boundaries. The model also proved robust across different embedding dimensions, ranging from 50 to 300.
Discussion and Future Work
The success of TriDNR highlights the synergy between structure and content: structure compensates for sparse text, while content aids when links are few. However, the model's reliance on text content suggests potential limitations in networks where node features are non-existent or non-textual. Additionally, the model introduces a weight parameter (α) to balance structure versus content, which may require manual tuning for new datasets. Future research directions include extending the framework to heterogeneous networks and incorporating more complex node features beyond simple vocabulary.
Summary updated