Abstract
Evolutionary deep intelligence has recently shown great promise for producing small, powerful deep neural network models via the synthesis of increasingly efficient architectures over successive generations. Despite recent research showing the efficacy of multi-parent evolutionary synthesis, little has been done to directly assess architectural similarity between networks during the synthesis process for improved parent network selection. In this work, we present a preliminary study into quantifying architectural similarity via the percentage overlap of architectural clusters. Results show that networks synthesized using architectural alignment (via gene tagging) maintain higher architectural similarities within each generation, potentially restricting the search space of highly efficient network architectures.
Introduction
Deep Neural Networks (DNNs) are powerful but often too massive for deployment on resource-constrained mobile or embedded devices. Evolutionary deep intelligence addresses this by "breeding" smaller, more efficient networks over successive generations. While early methods were "asexual" (single parent), recent research has moved toward "sexual" synthesis, where multiple parent networks combine to create offspring. However, a significant gap remains: the lack of a robust metric to measure architectural similarity within these populations. Without such a metric, it is difficult to determine if a population is exploring diverse designs or converging prematurely.
Methodology
The authors propose Architectural Cluster Overlap as a similarity metric. In $m$-parent evolutionary synthesis, a network is represented as a set of neurons and synapses. New networks are synthesized based on synaptic connectivity probabilities from parents, influenced by environmental factors like storage constraints. To ensure structural alignment during synthesis, gene tagging is employed, matching architectural clusters that share a common ancestral origin. Similarity is calculated as the percentage of shared architectural clusters between two networks:
$$\%overlap_{AB} = \frac{C_A \cap C_B}{C_A}$$
where $C_A$ and $C_B$ are the sets of architectural clusters in the networks being compared.
Experimental Results
The approach was evaluated using the MNIST dataset over seven generations. A key finding was that gene tagging acts as a constraint on architectural diversity. As shown in the data, populations using gene tagging maintained significantly higher similarity scores in early generations compared to those without.
By Generation 3, networks without tagging reached a diversity peak with only 68.84% overlap, whereas those with tagging remained at 83.49%. However, in later generations (Gen 6 and 7), the "No Gene Tagging" group saw a sharp spike in similarity, reaching 91.05% by the final generation. This spike indicates a "population collapse" where networks became so sparse and simple that they could no longer represent the problem, resulting in accuracy dropping to near-random levels (~10%).
The relationship between storage size and performance accuracy suggests a "sweet spot" around generations 3 and 4, where architectural variability is highest and the tradeoff between model size and accuracy is optimized. Networks with gene tagging generally showed a slightly slower decline in accuracy as storage size decreased compared to those without tagging.
Discussion and Conclusion
This preliminary study highlights the utility of architectural similarity metrics in monitoring evolutionary synthesis. While gene tagging preserves performance by preventing architectural mismatch, it may also restrict the search space by maintaining higher similarity within generations. The observed similarity spikes in later generations serve as a signal for population collapse rather than healthy convergence.
Future work will explore more complex measures like mutual information and the development of advanced mating policies to balance diversity and efficiency, ensuring that the evolutionary process continues to discover radically new and efficient structures.
Summary updated