Abstract
To convey neural network architectures in publications, appropriate visualizations are of great importance. While most current deep learning papers contain such visualizations, these are usually handcrafted just before publication, which results in a lack of a common visual grammar, significant time investment, errors, and ambiguities. Current automatic network visualization tools focus on debugging the network itself and are not ideal for generating publication visualizations. Therefore, we present an approach to automate this process by translating network architectures specified in Keras into visualizations that can directly be embedded into any publication. To do so, we propose a visual grammar for convolutional neural networks (CNNs), which has been derived from an analysis of such figures extracted from all ICCV and CVPR papers published between 2013 and 2019. The proposed grammar incorporates visual encoding, network layout, layer aggregation, and legend generation. We have further realized our approach in an online system available to the community, which we have evaluated through expert feedback, and a quantitative study. It not only reduces the time needed to generate network visualizations for publications, but also enables a unified and unambiguous visualization design.
Introduction
In the rapidly evolving field of deep learning, communicating complex model architectures is essential. However, researchers often face a "figure tax": spending hours in manual drawing tools like Illustrator or PowerPoint to create diagrams. This manual process is not only inefficient but also prone to inconsistencies and errors where labels may not match visual representations. While debugging tools like TensorBoard or Netron exist, they are designed for inspection rather than communication, often producing vertical or overly detailed graphs that do not fit well within the constraints of a two-column research paper. Net2Vis addresses this by providing a tool specifically designed for generating publication-ready visualizations directly from Keras code.
Empirical Analysis of the Visual Language
To establish a standardized visual language, the authors conducted an extensive analysis of existing literature. They compiled a corpus of 751 neural network architecture visualizations extracted from ICCV and CVPR papers published between 2013 and 2019. This study revealed a significant increase in the use of such visualizations over time, highlighting the growing need for a unified grammar.
The Net2Vis Visual Grammar
The core contribution of Net2Vis is a formal Visual Grammar for CNNs. This grammar is built on several key design principles:
- Layer Encodings: Net2Vis uses 2D glyphs where the height represents spatial resolution and the width represents the number of feature channels. This follows perceptual mapping principles where size is an effective channel for quantitative data.
- Trapezoid Glyphs: To resolve ambiguity regarding where downsampling (e.g., pooling) or upsampling occurs, the tool uses trapezoidal shapes. A layer that reduces resolution is wider on its input side than its output side, making the transformation explicit within the glyph itself.
- Layer Aggregation: To maintain readability for deep models like ResNet, Net2Vis allows users to group repeating sequences of layers into "aggregates" (e.g., a "Residual Block"). These are represented by glyphs with thicker borders, simplifying the overall flow without losing structural information.
- Automated Layout: The system employs a network simplex algorithm to generate a compact, horizontal, left-to-right layout optimized for paper publication.
Evaluation and Key Results
The effectiveness of Net2Vis was evaluated through expert interviews and a quantitative user study involving 10 participants. The results demonstrated several advantages over traditional methods:
- Higher Accuracy: Participants were significantly more accurate at extracting architectural details (e.g., identifying minimal spatial resolutions) using Net2Vis (83.75% accuracy) compared to TensorBoard (63.13%).
- Superior Interpretability: Net2Vis figures were rated higher in informativeness (86.6% preference) and interpretability (75% preference) compared to original handcrafted figures from published papers.
- Efficiency: While manual figures can take hours to iterate, Net2Vis generates standardized diagrams instantly from code, requiring only minimal user input for high-level styling or aggregation.
Conclusion and Future Work
Net2Vis successfully codifies the visual language of CNNs into an automated tool that bridges the gap between model implementation and publication. While currently tailored to the Keras framework, the underlying visual grammar is universal. Future work includes expanding the parser to support other frameworks like PyTorch and JAX, as well as providing more aesthetic flexibility for users to match specific publication styles.
Researchers can access the tool at https://viscom.net2vis.uni-ulm.de.
Summary updated