Abstract
Classification of posts in social media such as Twitter is difficult due to the noisy and short nature of texts. Sequence classification models based on recurrent neural networks (RNN) are popular for classifying posts that are sequential in nature. RNNs assume the hidden representation dynamics to evolve in a discrete manner and do not consider the exact time of the posting. In this work, we propose to use recurrent neural ordinary differential equations (RNODE) for social media post classification which consider the time of posting and allow the computation of hidden representation to evolve in a time-sensitive continuous manner. In addition, we propose a novel model, Bi-directional RNODE (Bi-RNODE), which can consider the information flow in both the forward and backward directions of posting times to predict the post label. Our experiments demonstrate that RNODE and Bi-RNODE are effective for the problem of stance classification of rumours in social media.
Introduction
Social media conversations, particularly on platforms like Twitter, present unique challenges for automated classification. Beyond noisy text and informal grammar, the temporal dynamics of these threads are highly irregular. In a rumor thread, the time elapsed between a post and its reply carries significant contextual weight. Traditional sequence models like LSTMs or GRUs treat sequences as discrete steps, remaining "blind" to the actual time elapsed between interactions.
Methodology
To address these limitations, this research introduces a continuous-time approach leveraging Neural Ordinary Differential Equations (Neural ODEs).
Recurrent Neural Ordinary Differential Equations (RNODE)
Instead of defining a fixed number of layers, a Neural ODE uses a neural network to define the derivative of the hidden state. An ODE solver then integrates this derivative to find the state at any point in time. In the RNODE framework, the hidden state "evolves" continuously between two tweets at times $t_{i-1}$ and $t_i$. The number of update steps the solver takes is determined by the actual time gap, making the model naturally time-sensitive.
Bi-directional RNODE (Bi-RNODE)
While the standard RNODE processes information chronologically, the proposed Bi-RNODE incorporates a backward-running RNODE that processes the thread from future to past. The hidden states from both directions are aggregated (via concatenation or averaging) to provide a holistic view of a post's stance within the entire conversation.
Experimental Evaluation
The models were evaluated using the RumourEval-2019 dataset, focusing on rumor stance classification (Support, Query, Deny, or Comment) across four major events: Charlie Hebdo, Ferguson, Ottawa shooting, and Sydney siege.
Key Results
- Superior Performance: Bi-RNODE consistently outperformed traditional discrete models. For the Sydney siege event, Bi-RNODE achieved an AUC of 0.739, significantly higher than the 0.634 achieved by a standard LSTM.
- Parameter Efficiency: RNODE and Bi-RNODE are remarkably lean. RNODE uses only 0.22M parameters compared to 1.70M for an LSTM, as the model "depth" is handled by the ODE solver rather than additional weighted layers.
- Generalization: The models maintained their performance lead in "Unseen Event" scenarios, indicating better generalization to new topics.
- Latent Separation: t-SNE visualizations demonstrated that Bi-RNODE creates much clearer clusters for different stance classes compared to standard RNODE.
Conclusion
By treating the hidden state of a conversation as a continuous trajectory rather than a series of discrete jumps, RNODE and Bi-RNODE effectively capture the irregular temporal dynamics of social media. While the bi-directional approach is less suited for real-time classification due to its dependence on future context, it provides a powerful tool for retrospective thread analysis. Future work involves integrating these models with Conditional Random Fields (CRFs) to better capture structured label dependencies.
Summary updated