Introduction
The design of Deep Neural Networks (DNNs) has traditionally relied on expert intuition and extensive trial-and-error. Determining the optimal number of layers, filter sizes, and architectural components like Batch Normalization or Dropout is a complex task. While Genetic Algorithms (GAs) have been employed to automate this "Neural Architecture Search" (NAS), they often struggle with the "exploitation" of continuous parameter spaces. Standard GAs typically select from a predefined discrete set of values, failing to converge effectively as the search space expands.
To address these limitations, this paper introduces a hybrid optimization framework. It integrates a Genetic Algorithm for discrete architectural decisions with a novel "Bi-Generative Adversarial Network" (Bi-GAN) to refine continuous parameters, enabling the discovery of high-performing, efficient network architectures.
Methodology
The proposed framework operates through a two-tiered optimization process that balances global exploration with local exploitation.
The Genetic Algorithm Tier
The GA tier manages the high-level "blueprint" of the network. It handles discrete architectural choices, such as selecting activation functions (e.g., ReLU vs. Tanh), kernel sizes, and toggling features like Batch Normalization, Dropout, or Max Pooling. This tier ensures broad exploration of the architectural search space.
The Bi-GAN Tier
The core innovation is the Bi-GAN tier, which refines continuous parameters—such as the specific number of neurons in a layer—within a nearly continuous range. Unlike standard GAs that pick from a list, the Bi-GAN "dreams up" precise values.
As shown in Figure 2, the Bi-GAN architecture consists of two generators ($G_1, G_2$) and two evaluators ($E_1, E_2$). The generators propose parameter sets, which are then used to build and train networks. The evaluators measure the resulting performance (accuracy). A discriminator then analyzes these results to distinguish between "good" and "bad" parameter sets. This competitive setup provides a stable training signal; the discriminator identifies which generator is performing better, forcing the other to improve.
Experimental Setup
The framework was evaluated using the ModelNet40 dataset, which contains 3D CAD models across 40 categories. These models were converted into $30 \times 30 \times 30$ voxel grids. The system's performance was benchmarked against the hand-crafted 3D ShapeNets architecture and traditional GA-only approaches.
Results and Discussion
The Bi-GAN + GA method demonstrated significant improvements over both manual and standard automated designs.
| Method | Accuracy (%) |
|---|---|
| 3D ShapeNets (Hand-crafted) | 84.17 |
| Small-set GA | 82.94 |
| Large-set GA | 36.41 |
| Proposed Bi-GAN + GA | 85.20 |
A key finding was the failure of standard GAs in large search spaces (Large-set GA), where accuracy plummeted to 36.41%. In contrast, the Bi-GAN successfully navigated this complexity. Furthermore, the system discovered that precise parameter tuning allowed a shallower 3-layer convolutional network to outperform the 4-layer 3D ShapeNets model. When allowed to search up to 5 layers, the proposed method achieved an accuracy of 86.62%.
Conclusion
By combining the exploratory power of Genetic Algorithms with the fine-tuning capabilities of a Bi-GAN, this framework offers a robust solution for autonomous DNN design. While computationally intensive, the approach moves beyond discrete search spaces, allowing for the discovery of optimized architectures that exceed the performance of human-designed models. Future work may involve scaling this method to deeper architectures and optimizing hyperparameters like learning rates alongside the network structure.
Summary updated