When modeling complex user behaviors, machine learning models typically face a choice between two distinct representations: sequences and graphs.
Sequences capture strict chronological logic. They excel at tracking immediate, short-term intent—such as the exact order of clicks in a web session or the step-by-step execution of a program. However, sequential models often treat items as isolated tokens, failing to capture the broader, high-order collaborative signals that emerge from the collective behavior of all users and items simultaneously.
Graphs, on the other hand, excel at capturing these structural and relational contexts. A knowledge graph or a user-item interaction network can map out complex, multi-hop relationships and long-term preferences. Yet, standard graph neural networks (GNNs) are often order-blind; they struggle to incorporate the continuous temporal evolution and immediate context that dictate real-time decisions.
As Qiannan Zhu et al. demonstrate in A Knowledge-Aware Attentional Reasoning Network for Recommendation, sequential history and structural graph paths are deeply interdependent. Relying on just one dimension forces a model to choose between chronological precision and relational depth. The research direction of sequential-graph fusion bridges this gap, integrating temporal dynamics with structural topology to build models that understand not just what happened, but where it fits within a broader relational ecosystem.
The Intuition: Filtering Structure Through Time
Why does fusing these two modalities yield such significant performance gains? The core intuition is that temporal context acts as a dynamic filter for static structural relationships.
Consider a user browsing an e-commerce platform. A static knowledge graph knows that a laptop is related to a laptop bag, a wireless mouse, and software licenses. However, without temporal context, a model cannot know which of these relationships matters right now. By integrating the user's recent sequential activity, the system can filter the static graph through the lens of current interests, prioritizing recommendations that align with immediate intent while leveraging established semantic connections.
As Deqing Yang et al. note in Incorporating User Micro-behaviors and Item Knowledge into Multi-task Learning for Session-based Recommendation, item relationships are inherently non-linear and bidirectional, whereas user actions follow a strict chronological logic. Fusing them allows models to balance two fundamental goals: exploitation and exploration.
[ Global Graph / Knowledge Base ] [ Local Chronological Sequence ]
(Relational Structure) (Temporal Context)
│ │
└───────────────────┬───────────────────┘
▼
[ Sequential-Graph Fusion ]
│
▼
Filtered Dynamic Predictions
As Bryan Hooi et al. show in STP-UDGAT: Spatial-Temporal-Preference User Dimensional Graph Attention Network for Next POI Recommendation, a sequential stream can exploit a user's highly specific, personalized historical trajectory, while a global graph stream explores spatial and temporal connectivity across the entire population to identify relevant but unvisited nodes. Similarly, Linmei Hu et al. leverage this balance in Graph Neural News Recommendation with Long-term and Short-term Interest Modeling to capture both fleeting short-term interests (via sequential networks) and stable long-term preferences (via global interaction graphs).
Architectural Blueprints for Fusion
To combine these two paradigms, researchers have converged on three primary architectural patterns:
1. Dual-Stream Parallel Architectures
In this setup, the model processes the sequence and the graph through separate, specialized encoders before fusing their representations. Typically, a GNN captures structural neighborhood information or global item co-occurrences, while a Transformer or RNN models chronological order.
For instance, Fedor Borisyuk et al. utilize this dual approach at scale in LiGNN: Graph Neural Networks at LinkedIn to represent both a node's static position in a professional network and its dynamic behavioral patterns over time. To ensure these two streams do not produce conflicting representations, Liangwei Yang et al. introduce a unified objective centered on "alignment" and "uniformity" in Graph-Sequential Alignment and Uniformity: Toward Enhanced Recommendation Systems, forcing the order-blind collaborative signals from the graph encoder to harmonize with the chronological patterns of the Transformer encoder.
Lianghao Xia et al. similarly combine self-attention with global message passing in Multi-Behavior Sequential Recommendation with Temporal Graph Transformer to capture multi-behavior user histories, while Ruihong Qiu et al. integrate a gated graph neural network with a transformer-based readout in Exploiting Positional Information for Session-Based Recommendation to capture local connectivity alongside global sequence dependencies.
2. Sequential Processing of Graph Snapshots
Instead of running streams in parallel, another approach discretizes time into windows or slices, building a structural graph for each interval. The model then uses a sequential network to track how these structural embeddings evolve over time.
Yozen Liu et al. apply this to predict user engagement in Characterizing and Forecasting User Engagement with In-App Action Graph: A Case Study of Snapchat, using GCNs to extract structural features from daily interaction graphs, and then passing those embeddings into an LSTM network to model temporal progression. A similar philosophy is explored in Learning Dual Dynamic Representations on Time-Sliced User-Item Interaction Graphs for Sequential Recommendation, where spatial graph aggregation within a time window extracts collaborative patterns, and a recurrent component links these windows to capture long-term individual trajectories.
3. Embedding Injection and Sequence Augmentation
A third pattern directly injects structural context into sequential models, or vice versa, treating one modality as an augmentation of the other.
For example, Xulin Ma et al. integrate graph-learned user embeddings directly into a Transformer model in GSRec: A Graph-Sequence Recommendation System Based on Reverse-Order Graph and User Embedding, allowing the sequential model to leverage a holistic understanding of user preferences. Alternatively, Yuxia Wu et al. propose a retrieval-augmented approach for dynamic graphs in Retrieval Augmented Generation for Dynamic Graph Modeling, where structural embeddings summarizing global context are prepended directly to temporal node sequences before being processed by a Transformer. In a similar vein, Wei Zhuo et al. incorporate sequential components into graph convolutional architectures in A Behavior-aware Graph Convolution Network Model for Video Recommendation to better track evolving user interests over structural graphs.
Generalizing Beyond Recommendations
While sequential-graph fusion is highly prominent in recommendation systems—ranging from session-based software adoption (Dengcheng Yan et al. in Session-based social and dependency-aware software recommendation) to conversational preference tracking (Mingjie Qian et al. in HutCRS: Hierarchical User-Interest Tracking for Conversational Recommender System) and clinical workflow prediction (Xia Ning et al. in Improving information retrieval from electronic health records using dynamic and multi-collaborative filtering)—the underlying principle is a general machine learning paradigm. It applies to any domain where data exhibits both local order and global structure.
- Cross-Domain Modeling: Jiangxia Cao et al. address cross-domain product suggestions in Contrastive Cross-Domain Sequential Recommendation by combining GNNs for global item co-occurrence with self-attention for chronological user actions across domains. In privacy-preserving contexts, Dongyi Zheng proposes a federated framework in FedCSR: A Federated Framework for Multi-Platform Cross-Domain Sequential Recommendation with Dual Contrastive Learning that uses a hybrid GNN and self-attention encoding architecture to capture multi-platform behaviors without requiring explicit user alignment.
- Social Networks & Event Sequences: Rizal Fathony et al. integrate individual user activity sequences with interpersonal relationship graphs in Integrating Sequential and Relational Modeling for User Events: Datasets and Prediction Tasks by treating personal events as primary data entities rather than secondary node attributes, capturing personal behavior and social influence simultaneously.
- Program Execution: Kevin Swersky et al. fuse static assembly code graphs with dynamic runtime execution sequences in Learning Execution through Neural Code fusion to predict branch behavior and memory prefetching, bridging local structural code logic with global execution history.
- Neuroscience: Xinxu Wei et al. combine graph transformers for topological brain data (fMRI) with standard transformers for sequential time-series data (EEG) in Multi-modal Cross-domain Self-supervised Pre-training for fMRI and EEG Fusion to capture both high spatial and temporal resolutions.
- Information Virality: Junliang Li et al. combine graph attention networks (for short-term cascade structures) with transformers (for long-term historical user patterns) in Public Opinion Field Effect and Hawkes Process Join Hands for Information Popularity Prediction to predict how information spreads across competitive social spaces.
Ultimately, sequential-graph fusion works because it acknowledges that real-world events do not happen in a vacuum, nor do they happen out of order. By unifying the chronological logic of sequences with the relational logic of graphs, models achieve a more complete, robust understanding of dynamic systems.
Key Foundational Work
A Knowledge-Aware Attentional Reasoning Network for Recommendation (2020) — contributed 4 concept elements to this direction.
Graph Neural News Recommendation with Long-term and Short-term Interest Modeling (2019) — contributed 2 concept elements to this direction.
Characterizing and Forecasting User Engagement with In-app Action Graph: A Case Study of Snapchat (2019) — contributed 1 concept elements to this direction.