Abstract
Probabilistic modeling often requires a choice between Directed Acyclic Graphs (DAGs) for causal relationships and Undirected Graphs for symmetric associations. This paper introduces a unified hierarchical framework for chain graphs that combines both, providing a cleaner mathematical definition and incorporating deterministic nodes to represent models like neural networks.
Introduction: Bridging Directed and Undirected Models
In probabilistic modeling, researchers typically choose between two paradigms: Directed Acyclic Graphs (DAGs), which are ideal for causal, parent-child relationships, and Undirected Graphs (Markov networks), which excel at representing symmetric associations or constraints.
However, real-world data often involves both. For example, a disease might "cause" symptoms (directed), but those symptoms might also have complex, non-causal correlations among themselves (undirected). Previous definitions of "chain graphs"—which mix these two types—were often mathematically cumbersome or struggled to handle deterministic relationships, such as the exact functional mappings found in neural networks. This paper aims to make chain graphs more accessible and practical for machine learning by providing a cleaner definition and a way to represent repeated data samples for learning.
Methods: Hierarchical Decomposition and Plates
The author's primary contribution is a hierarchical definition of chain graphs. Instead of treating the mixed graph as one giant, complex entity, the paper breaks it down into "component subgraphs."
Decomposition
The graph is partitioned into blocks. Each block is either a directed network or an undirected network, conditioned on its "parents" in the hierarchy. This modular approach allows the complex independence properties of a chain graph to be "read off" using simpler, well-known rules for directed and undirected networks.
Deterministic Nodes and Plates
The paper explicitly incorporates deterministic nodes (often represented by double circles), allowing the framework to describe standard feed-forward neural networks. To handle learning from data, the author uses "plates"—boxes drawn around variables to indicate they are repeated $N$ times. This allows the graph to represent not just a single instance, but the entire statistical process of estimating parameters from a dataset, effectively treating learning as inference on an expanded graph.
Key Insights: Neural Networks and Unified Interpretation
One of the most significant insights is that a standard feed-forward neural network is actually just a special case of a chain graph with deterministic nodes. By mapping diverse architectures—including Boltzmann machines and Bayesian classifiers—into this notation, the paper proves it is a truly general-purpose language for machine learning.
Results and Expressiveness
The author proves that this hierarchical decomposition is mathematically equivalent to the global Markov property for chain graphs (Theorem 2). This provides a rigorous foundation for using this simplified view. Furthermore, the paper provides explicit equations for the joint probability of these mixed models, showing how to normalize undirected components even when they are embedded within a larger directed structure.
Limitations and Future Work
While the paper provides the language to describe these models, it does not propose new algorithms for training them, suggesting instead the use of standard methods like Expectation-Maximization (EM) or Gradient Descent. Additionally, the author notes that the details of expanding plates for complex, multi-table datasets can become tedious, suggesting a need for better software abstractions. Finally, while the paper relaxes some constraints to allow for deterministic nodes, certain undirected subcomponents still require strictly positive distributions to maintain their standard Markov properties.
Summary updated