Abstract
Machine learning systems typically apply the same model to both easy and tough cases. This is in stark contrast with humans, who tend to evoke either fast (instinctive) or slow (analytical) thinking depending on the problem difficulty, a property called the dual-process theory of mind. We present FLOWGEN, a graph-generation model inspired by the dual-process theory of mind that generates large graphs incrementally. Depending on the difficulty of completing the graph at the current step, graph generation is routed to either a fast (weaker) or a slow (stronger) model. These modules have identical architectures, but vary in the number of parameters and consequently differ in generative power. Experiments on real-world graphs show that ours can successfully generate graphs similar to those generated by a single large model, while being up to 2x faster.
Introduction
Most generative models for graphs—whether they are creating molecular structures or social networks—apply the same amount of computational power to every part of the graph. However, graph generation is not uniformly difficult. When building a graph piece-by-piece, often represented as a series of random walks, some steps are "easy" because they follow patterns frequently seen during training (exploitation). Other steps are "hard" because they require the model to generalize to novel neighborhoods not explicitly seen before (exploration). Using a massive, parameter-heavy model for the "easy" parts is computationally wasteful. FLOWGEN addresses this by creating a system that recognizes when it needs to "think harder" and switches resources accordingly.
Methodology
FLOWGEN treats graph generation as a language modeling problem, decomposing a graph into a set of second-order random walks where each node is treated like a "word" in a "sentence."
Dual Architectures
The system utilizes two decoder-only Transformer models. The FAST model is shallow (1–4 layers), designed for speed and local pattern recognition. The SLOW model is significantly deeper (6–36 layers), providing the capacity for complex structural reasoning.
The Handover Mechanism
To decide when to switch from FAST to SLOW, FLOWGEN employs a Bloom Filter. This memory-efficient data structure stores all 4-node neighborhoods from the training data. During inference, the FAST model initiates the walk. The system queries the Bloom Filter with the current neighborhood; if the neighborhood is "known," the FAST model continues. Once the walk enters an "unknown" neighborhood—signaling the start of exploration—the system hands the sequence over to the SLOW model to complete the walk.
Graph Reconstruction
After generating a sufficient number of walks, the system assembles them into a final graph by calculating edge probabilities based on the co-occurrence frequency of nodes in the generated sequences.
Experimental Evaluation
We evaluated FLOWGEN on four representative large-scale graph datasets: CORA-ML and CITESEER (citation networks), POLBLOGS (political blogs), and PUBMED (medical citations).
Key Results
- Speedup: FLOWGEN is up to 2x faster than using a standalone SLOW model.
- Performance Parity: Despite the speedup, FLOWGEN matches or exceeds the performance of the SLOW model on link prediction tasks. For example, on the POLBLOGS dataset, FLOWGEN achieved an AUC of 93.8, identical to the SLOW model, but in significantly less time.
- Structural Accuracy: The generated graphs successfully mimic the topological properties of the original data, such as triangle counts, max degree, and power-law exponents, outperforming several baseline methods like NetGAN and VGAE.
Discussion and Insights
The core innovation of FLOWGEN is applying the psychological concept of System 1 (instinctive) and System 2 (analytical) thinking to machine learning efficiency. The paper demonstrates that the difficulty of generating a random walk increases as the walk moves further from its starting node. The FAST model is effective at "regurgitating" known local structures, while the SLOW model is essential for maintaining structural integrity in novel areas. Using a Bloom Filter as a "surprise detector" allows the system to trigger the SLOW model only when the novelty of the generated graph exceeds a certain threshold.
Limitations and Future Work
Hardware latency remains a challenge; the benefits of FLOWGEN are most apparent when both models reside on the same GPU to avoid data transfer overhead. Future work could involve more granular, per-step dynamic switching and extending this "fast and slow" approach to general language modeling, where smaller models handle common phrases and larger models take over for complex reasoning.
Summary updated