SEGEN: Sample-Ensemble Genetic Evolutional Network Model

Jiawei Zhang, Limeng Cui, Fisher B. GouzaarXiv 2018 · OpenReview

Abstract

Deep learning, a rebranding of deep neural network research works, has achieved a remarkable success in recent years. With multiple hidden layers, deep learning models aim at computing the hierarchical feature representations of the observational data. Meanwhile, due to its severe disadvantages in data consumption, computational resources, parameter tuning costs and the lack of result explainability, deep learning has also suffered from lots of criticism. In this paper, we will introduce a new representation learning model, namely "Sample-Ensemble Genetic Evolutionary Network" (SEGEN), which can serve as an alternative approach to deep learning models. Instead of building one single deep model, based on a set of sampled sub-instances, SEGEN adopts a genetic-evolutionary learning strategy to build a group of unit models generations by generations. The unit models incorporated in SEGEN can be either traditional machine learning models or the recent deep learning models with a much "narrower" and "shallower" architecture. The learning results of each instance at the final generation will be effectively combined from each unit model via diffusive propagation and ensemble learning strategies. From the computational perspective, SEGEN requires far less data, fewer computational resources and parameter tuning efforts, but has sound theoretic interpretability of the learning process and results. Extensive experiments have been done on several different real-world benchmark datasets, and the experimental results obtained by SEGEN have demonstrated its advantages over the state-of-the-art representation learning models.

Introduction

While deep learning has revolutionized artificial intelligence, its trajectory toward increasingly massive and complex architectures has introduced significant overhead. Training modern deep models typically requires extensive datasets, specialized hardware, and exhaustive hyper-parameter tuning. Furthermore, these "black-box" models often lack interpretability. The Sample-Ensemble Genetic Evolutionary Network (SEGEN) addresses these challenges by providing a "bionic" alternative: achieving hierarchical complexity not through vertical layer stacking, but by evolving a population of simpler models over time.

Methodology

The SEGEN framework operates through three distinct phases designed to maximize efficiency and interpretability:

  1. Network Sampling: To handle large-scale data, SEGEN samples small "sub-networks" using strategies such as Breadth-First Search (BFS) and Biased Edge Sampling. This approach captures both local neighborhoods and global structures while maintaining linear computational complexity $O(n)$ relative to the number of nodes.
  2. Genetic Evolutionary Network (GEN): This core engine evolves a population of "unit models" (e.g., shallow autoencoders or small CNNs). Through selection based on validation loss, crossover of internal variables, and random mutation, the population inherits and refines knowledge across generations.
  3. Result Ensemble: Final representations are synthesized from the evolved population. For nodes not included in the initial samples, SEGEN employs "diffusive propagation," leveraging neighbor information to estimate embeddings.
The SEGEN architecture: from network sampling to genetic evolution and final ensemble.

Experimental Results

The authors evaluated SEGEN across social networks (Twitter, Foursquare), image classification (MNIST), and tabular datasets (YEAST, ADULT).

In graph learning tasks, SEGEN consistently outperformed established baselines like DeepWalk and node2vec in network recovery and community detection. For image classification, using small CNNs as unit models, SEGEN achieved an accuracy of 99.37% on MNIST, surpassing LeNet-5 (99.05%). As shown in the convergence plots, the evolutionary strategy effectively minimizes the loss function $\mathcal{L}_c$ and stabilizes within a few dozen generations.

Convergence of the loss function $\mathcal{L}_c$ over 500 generations for Foursquare and Twitter datasets.

Discussion and Conclusion

SEGEN demonstrates that "deepening through time" via evolution is a viable alternative to deepening through space via layer stacking. By utilizing shallow unit models, the framework reduces computational costs and improves interpretability. While current results are promising, future work will focus on dynamic ensemble weighting and establishing more rigorous mathematical proofs for the convergence of the evolutionary layer. SEGEN stands as a flexible, model-agnostic framework capable of enhancing various base architectures through its evolutionary wrapper.

Summary updated

Abstract

Deep learning, a rebranding of deep neural network research works, has achieved a remarkable success in recent years. With multiple hidden layers, deep learning models aim at computing the hierarchical feature representations of the observational data. Meanwhile, due to its severe disadvantages in data consumption, computational resources, parameter tuning costs and the lack of result explainability, deep learning has also suffered from lots of criticism. In this paper, we will introduce a new representation learning model, namely "Sample-Ensemble Genetic Evolutionary Network" (SEGEN), which can serve as an alternative approach to deep learning models. Instead of building one single deep model, based on a set of sampled sub-instances, SEGEN adopts a genetic-evolutionary learning strategy to build a group of unit models generations by generations. The unit models incorporated in SEGEN can be either traditional machine learning models or the recent deep learning models with a much "narrower" and "shallower" architecture. The learning results of each instance at the final generation will be effectively combined from each unit model via diffusive propagation and ensemble learning strategies. From the computational perspective, SEGEN requires far less data, fewer computational resources and parameter tuning efforts, but has sound theoretic interpretability of the learning process and results. Extensive experiments have been done on several different real-world benchmark datasets, and the experimental results obtained by SEGEN have demonstrated its advantages over the state-of-the-art representation learning models.
BibTeX
@misc{zhang2018segen,
  author = {Jiawei Zhang and Limeng Cui and Fisher B. Gouza},
  title = {{SEGEN: Sample-Ensemble Genetic Evolutional Network Model}},
  year = {2018},
  eprint = {1803.08631},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1803.08631},
}
Figure