Self-Adaptive Neural Networks Based on a Poisson Approach for Knowledge Discovery

Haiying Wang, Huiru Zheng, Francisco AzuajeIJCAI 2007

Introduction

Self-Adaptive Neural Networks (SANNs) are increasingly utilized in data mining due to their ability to dynamically adjust their topology to fit complex datasets. However, traditional SANNs, such as the Self-Organizing Map (SOM) and the Self-Organizing Tree Algorithm (SOTA), typically rely on Euclidean distance as a matching criterion. This metric assumes a Gaussian distribution with constant variance. In many biological contexts, such as gene expression analysis using Serial Analysis of Gene Expression (SAGE), data consists of discrete counts that follow a Poisson distribution, where variance is proportional to the mean. Applying Gaussian-based metrics to such data often results in suboptimal clustering. This paper presents the Poisson-based Self-Organizing Tree Algorithm (PSOTA), designed specifically for count-based datasets.

Methodology: The PSOTA Framework

PSOTA builds upon the hierarchical binary tree structure of SOTA but introduces a statistical overhaul of the neuron matching and adaptation processes.

Tree Growth and Topology

The algorithm begins with a root node and two initial leaf neurons. It iteratively expands by identifying the leaf neuron with the highest heterogeneity (representation error) and splitting it into two new descendants. This process continues until a predefined stopping criterion or threshold is met.

Self-adaptive growth of the PSOTA binary tree structure from initial state to expanded hierarchy.

Statistical Matching Criteria

To account for the Poisson nature of the data, PSOTA replaces Euclidean distance with two primary metrics:

  1. Chi-square Statistics: Measures the deviation of observed counts from expected values.
  2. Maximum Joint Likelihood: Identifies the neuron most likely to have generated the observed data point under a Poisson model.

Relative Weight Adaptation

Unlike traditional networks that update weights based on absolute values, PSOTA focuses on relative profiles. By calculating expected values based on the sum of dimensions, the algorithm ensures that samples with similar proportional expression patterns are grouped together, regardless of their absolute magnitude.

Experimental Evaluation

The performance of PSOTA was evaluated using both synthetic and real-world biological datasets.

Datasets

  • Synthetic Data: 80 samples across five time points, generated using Poisson distributions to form four distinct groups (PA, PB, PC, and PD).
  • Mouse Retinal SAGE Data: 92 gene tags across 10 developmental stages, representing a challenging discrete count dataset.

Results

On synthetic data, PSOTA demonstrated superior accuracy compared to standard SOTA implementations. While Euclidean-based methods often conflated distinct groups (e.g., PA and PD), PSOTA achieved perfect separation, as evidenced by the confusion matrices and dendrogram structures.

Comparison of clustering performance and dendrogram structures across different distance metrics on synthetic Poisson data.

In the mouse retinal SAGE analysis, PSOTA identified biologically significant clusters corresponding to known functional stages of eye development, such as the embryonic and P10 clusters. The resulting dendrograms provided a clearer hierarchical representation of developmental lineages compared to traditional methods.

Conclusion

PSOTA addresses a critical gap in neural network-based clustering by incorporating Poisson statistics into a self-adaptive framework. By prioritizing relative profiles and acknowledging the mean-variance relationship of count data, PSOTA offers a more robust tool for knowledge discovery in bioinformatics and other fields dominated by discrete count data. Future work will focus on automating parameter selection through meta-heuristic approaches like genetic algorithms.

Summary updated

BibTeX
@inproceedings{wang2007self,
  author = {Haiying Wang and Huiru Zheng and Francisco Azuaje},
  title = {{Self-Adaptive Neural Networks Based on a Poisson Approach for Knowledge Discovery}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2007},
}
Figure