Network Modeling and Pathway Inference from Incomplete Data ("PathInf")

Xiang Li, Qitian Chen, Xing Wang, Ning Guo, Nan Wu, Quanzheng LiarXiv 2018 · arXiv

Introduction

In fields like oncology, understanding disease progression—such as how lung cancer spreads through the lymphatic system—is vital for treatment planning. However, modeling these "pathways" is notoriously difficult because real-world clinical data is often plagued by biased missing values. For instance, a surgeon might only biopsy lymph nodes that appear suspicious, leaving other nodes unrecorded. Standard tools like Bayesian Networks often fail in these scenarios due to biased missingness, small sample sizes, and the restrictive assumption of Directed Acyclic Graphs (DAGs), which cannot represent the cycles or reciprocal connections common in biological systems.

To address these challenges, researchers have developed PathInf, a two-stage framework designed to infer transition pathways from datasets with massive, non-uniformly missing values. By combining Maximum Likelihood Estimation (MLE) for data summarization with a greedy graph inference algorithm, PathInf identifies hidden connections in complex biological networks more effectively than traditional methods.

Methods: The Two-Stage Framework

PathInf avoids the pitfalls of individual data imputation by focusing on population-level distributions. The process is divided into two distinct stages:

  1. Stage 1: Data Summarization via MLE. Instead of guessing specific missing values for each patient, the model uses MLE to estimate a "state matrix." This matrix calculates the probability of different combinations of positive/negative nodes existing across the population. Crucially, it incorporates "priors"—assumptions about the likelihood of a positive node being missed versus a negative one—to account for the biased nature of clinical observations.
  2. Stage 2: Greedy Graph Inference. Once the state matrix is established, the model treats pathway discovery as a graph problem. Starting with an empty graph, it greedily adds edges that best explain the co-existence of positive nodes in the summarized states. The goal is to find the sparsest graph where all positive nodes in a given state are connected, without being forced into a strict, one-way tree structure.
The PathInf two-stage framework: (a) raw incomplete data, (b) summarized state matrix via MLE, (c) node identification, and (d) final inferred pathway graph.

Key Results and Clinical Discovery

The authors validated PathInf using both simulated data (10-node DAGs with up to 40% missingness) and a real-world dataset of 936 lung cancer patients tracking metastasis across 21 lymph node stations.

Superior Accuracy on Incomplete Data

In simulations with high missingness, PathInf consistently recovered true network edges with zero false negatives. In contrast, standard methods like Greedy Equivalent Search (GES) deviated significantly from the ground truth, often producing overly dense or inaccurate networks when faced with incomplete data.

Comparison of network recovery: (a) ground truth DAG, (b) PathInf result showing high fidelity to the original structure, and (c) standard GES method showing significant deviation and noise.

Clinical Insights

When applied to the lung cancer dataset, PathInf successfully mapped pathways that aligned with known anatomy while also revealing novel "jumping" connections. These findings suggest the existence of "sentinel node groups"—primary gateways for spread that skip nearby nodes to land on distant ones. Furthermore, by not forcing a DAG structure, PathInf can represent complex, reciprocal biological relationships that standard tools typically ignore.

Conclusion and Future Work

PathInf demonstrates that summarization is often superior to imputation when dealing with biased clinical data. While the model is excellent at identifying correlations and co-existence, its primary limitation is directionality; because the data is a "snapshot" from surgery, it lacks temporal information to explicitly define the chronological sequence of metastasis. Future work may incorporate tumor size and pathologic features to help infer the specific "arrows" of infection within these discovered pathways.

Summary updated

Abstract

In this work, we developed a network inference method from incomplete data ("PathInf") , as massive and non-uniformly distributed missing values is a common challenge in practical problems. PathInf is a two-stages inference model. In the first stage, it applies a data summarization model based on maximum likelihood to deal with the massive distributed missing values by transforming the observation-wise items in the data into state matrix. In the second stage, transition pattern (i.e. pathway) among variables is inferred as a graph inference problem solved by greedy algorithm with constraints. The proposed method was validated and compared with the state-of-art Bayesian network method on the simulation data, and shown consistently superior performance. By applying the PathInf on the lymph vascular metastasis data, we obtained the holistic pathways of the lymph node metastasis with novel discoveries on the jumping metastasis among nodes that are physically apart. The discovery indicates the possible presence of sentinel node groups in the lung lymph nodes which have been previously speculated yet never found. The pathway map can also improve the current dissection examination protocol for better individualized treatment planning, for higher diagnostic accuracy and reducing the patients trauma.
BibTeX
@misc{li2018network,
  author = {Xiang Li and Qitian Chen and Xing Wang and Ning Guo and Nan Wu and Quanzheng Li},
  title = {{Network Modeling and Pathway Inference from Incomplete Data ("PathInf")}},
  year = {2018},
  eprint = {1810.00839},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1810.00839},
}
Figure