Abstract
ArcText is a unified framework designed to translate complex Convolutional Neural Network (CNN) architectures into standardized, unique text descriptions. By representing architectural graphs as structured sequences, ArcText enables the application of Natural Language Processing (NLP) and data mining techniques to discover architectural patterns. This approach addresses the limitations of manual design and computationally expensive Neural Architecture Search (NAS) by facilitating the analysis of existing high-performance models through a unique, lossless, and machine-readable format.
Introduction
Designing high-performing CNNs typically involves either manual trial-and-error or resource-intensive Neural Architecture Search (NAS). A significant untapped opportunity lies in data mining existing successful architectures to identify effective "motifs." However, the lack of a standardized, machine-readable description format for CNNs—which are essentially complex graphs—hinders this. Current descriptions are often inconsistent or omit critical hyperparameters. ArcText bridges this gap by treating architecture as a language, allowing researchers to vectorize model designs using tools like Word2Vec or Transformers.
Methods
The ArcText framework utilizes a specialized grammar to decompose any CNN into four standardized building blocks, termed ArcUnits:
- ConvArcUnit: Captures convolutional details such as kernel size, stride, padding, and dilation.
- PoolArcUnit: Describes pooling operations.
- FullArcUnit: Represents fully-connected layers.
- MFUnit (Multi-Function Unit): Handles auxiliary operations like ReLU, Batch Normalization, and skip connections.
To ensure a 1:1 mapping between an architecture and its text string (uniqueness), the authors implemented a deterministic ordering algorithm. By treating the CNN as a Directed Acyclic Graph (DAG), the algorithm traverses the network, using SHA-224 hashing to resolve ambiguities in parallel branches. This ensures that identical architectures always produce the same string, solving the graph isomorphism problem for neural networks.
Results
The robustness of ArcText was validated using complex benchmarks: GoogleNet and ResNet. For GoogleNet, the system successfully identified four distinct parallel paths within inception modules, assigning unique identifiers to 25 layers. In the case of ResNet, ArcText accurately captured skip connection logic using "connect_to" identifiers. These results demonstrate that the framework provides a lossless representation, containing all variables necessary to reconstruct the model programmatically, unlike traditional visualization tools.
Conclusion
ArcText provides a foundation for building a large-scale corpus of neural architectures. While the current implementation focuses on CNNs and requires an initial manual DAG construction, future work aims to extend the grammar to Transformers and RNNs. By standardizing architectural descriptions, ArcText paves the way for automated discovery of design patterns in the deep learning community.
Summary updated