Feature Interaction Modeling via Graph Topologies

65 concepts54 papers1 proposal

In traditional machine learning, features are typically treated as flat vectors or sequences. Whether in tabular data, recommendation systems, or computer vision, models like Multi-Layer Perceptrons (MLPs) or standard Transformers often rely on implicit mechanisms—such as bit-wise multiplication or self-attention—to learn how different variables influence one another. However, these approaches can struggle to capture the complex, non-linear, and often sparse relational structure inherent in high-dimensional data.

A growing body of research suggests a more structured alternative: representing individual features, channels, or fields as nodes in a graph. By treating feature interactions as edges in a topology, researchers can leverage Graph Neural Networks (GNNs) and graph theory to explicitly model the "web" of dependencies within a single data instance.

Features as Nodes

The fundamental shift in this research direction is the move from vector concatenation to graph construction. In tabular data, for example, Li et al. propose Fi-GNN, which represents multi-field categorical features as nodes in a fully connected graph. Instead of a flat vector, the model uses a Gated Graph Neural Network to let feature fields "talk" to one another, capturing sophisticated interactions that traditional models might miss.

This concept extends to other domains:

  • Computer Vision: Chang et al. and Yin introduce CoAtFormer, which treats feature channels as nodes. By using graph convolutional networks for inter-channel communication, the model dynamically weights feature maps based on complex relational dependencies rather than simple global pooling.
  • Categorical Data: Researchers have modeled categorical values as nodes in a network where edges represent statistical dependencies, such as conditional probabilities. This allows for the detection of outliers by analyzing the graph's topological measures rather than simple frequency counts.
  • Natural Language Processing: Sekhon et al. and Chen developed WIGRAPH, a layer that builds an explicit word interaction graph to improve both predictive accuracy and model transparency.

Capturing Beneficial Interactions

A primary challenge in modeling interactions is "noise." In high-dimensional spaces, most potential feature pairings are irrelevant; modeling every possible interaction can lead to overfitting. Graph-based approaches provide a natural framework for interaction selection.

Zhang et al. address this with $L_0$-SIGN, which treats the identification of beneficial interactions as a graph topology optimization problem. By using $L_0$ regularization to prune edges between feature nodes, the model discards noisy interactions and retains only those that contribute to predictive performance. Similarly, Tang uses an improved PageRank algorithm on a feature graph to identify and prune globally redundant features, viewing feature selection through the lens of network topology.

Beyond Simple Aggregation

Standard GNNs often rely on neighborhood averaging, which can suppress the very non-linearities needed for effective feature interaction. To solve this, Hu et al. propose GraphAIR, which explicitly calculates pairwise interactions between neighbor features using element-wise multiplication. This disentangles simple aggregation from complex interaction, allowing the network to represent relationships between neighbors that are mathematically lost in standard convolutional layers.

Wu et al. further bridge this gap with GraphFM, which integrates Factorization Machines with GNNs. By representing feature fields as nodes and learning to select beneficial interactions as edges, the model captures high-order relationships in sparse data more efficiently than traditional deep learning paradigms.

Handling Heterogeneity and Context

Tabular data is notoriously difficult for deep learning because features are heterogeneous (e.g., age, city, income). Yan et al. argue in their work on T2G-FORMER that organizing these features into a "Feature Relation Graph" helps the model filter out noise and discover subtle interactions that rigid feature grouping or fully connected attention mechanisms might overlook.

This structural approach also aids in cross-modal tasks. Cao et al. show that representing semantic relations as explicit nodes in a graph—rather than just edges—enables more effective message propagation. By treating actions and spatial relationships as distinct entities with their own feature vectors, the model can distinguish between highly similar video segments by focusing on the fine-grained interactions between objects.

Ultimately, the intuition behind this direction is that data has a shape. By moving away from flat representations and toward graph topologies, these models can more faithfully mirror the intricate, relational nature of the real-world phenomena they aim to predict.

Key Foundational Work

A Feature Selection Method from the View Of Complex Network (2014) — contributed 1 concept elements to this direction.
Query expansion using probabilistic local feedback with application to multimedia retrieval (2007) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_graph_based_feature_interaction_a9b9c73e,
  title = {graph-based feature interaction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/graph-based-feature-interaction/rdr_a9b9c73ec6bc47649f1bccb721258f18},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_graph_based_feature_interaction_a9b9c73e_v1,
  title = {graph-based feature interaction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/graph-based-feature-interaction/rdr_a9b9c73ec6bc47649f1bccb721258f18/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake