Abstract
To convey neural network architectures in publications, appropriate visualizations are of great importance. While most current deep learning papers contain such visualizations, these are usually handcrafted just before publication, which results in a lack of a common visual grammar, significant time investment, errors, and ambiguities. Current automatic network visualization tools focus on debugging the network itself and are not ideal for generating publication visualizations. Therefore, we present an approach to automate this process by translating network architectures specified in Keras into visualizations that can directly be embedded into any publication. To do so, we propose a visual grammar for convolutional neural networks (CNNs), derived from an analysis of figures extracted from all ICCV and CVPR papers published between 2013 and 2019. The proposed grammar incorporates visual encoding, network layout, layer aggregation, and legend generation. We have further realized our approach in an online system available to the community, which we have evaluated through expert feedback and a quantitative study. It not only reduces the time needed to generate network visualizations for publications but also enables a unified and unambiguous visualization design.
Introduction
The rapid expansion of deep learning research has led to a proliferation of convolutional neural network (CNN) architectures. Communicating these complex structures effectively is crucial for peer review and knowledge dissemination. However, researchers often spend significant time manually drawing architecture diagrams in general-purpose tools like PowerPoint or Illustrator. This manual process is error-prone, lacks standardization, and often results in diagrams that are difficult to compare across different publications.
As shown in the analysis of major computer vision conferences, the volume of papers requiring such visualizations is increasing exponentially. Existing automated tools like TensorBoard or Netron, while excellent for debugging, often produce vertical, overly detailed graphs that do not fit well within the space constraints of a standard research paper.
The Net2Vis Approach
Net2Vis addresses these challenges by providing a web-based tool that translates Keras code into standardized, publication-ready visualizations. The core of the system is a 'visual grammar' designed specifically for CNNs, emphasizing technical accuracy and readability.
Visual Grammar and Encoding
The visual grammar maps technical specifications directly to visual attributes. For instance, spatial resolution is represented by the height of the layer glyph, while the number of feature channels is represented by the glyph's width. Layers that change resolution, such as pooling or upsampling, are depicted as trapezoids to clearly indicate the compression or expansion of data. A 'network simplex' algorithm arranges layers horizontally to match standard reading directions and fit page widths.
Handling Complexity through Aggregation
Modern architectures like ResNet or Inception contain dozens or hundreds of layers, making raw visualizations unreadable. Net2Vis automatically identifies repeating sequences and groups them into 'aggregates.' These can be collapsed to provide a high-level overview or expanded for detail.
This aggregation capability allows a massive ResNet50 architecture to be reduced from 177 individual glyphs to just 21, significantly improving legibility without losing essential structural information. The tool also automatically generates a legend to explain both individual layers and complex aggregates.
Evaluation and Results
The authors evaluated Net2Vis through a quantitative study and expert feedback. Participants using Net2Vis visualizations were significantly more accurate (83.75%) at answering technical questions about network structures compared to those using TensorBoard (63.13%). Furthermore, in 86.6% of cases, researchers preferred Net2Vis outputs over original handcrafted figures for their informativeness and clarity. The tool received an 'excellent' System Usability Scale (SUS) score of 83.44.
Conclusion
Net2Vis provides a practical, automated solution for generating high-quality CNN visualizations. By standardizing the visual language of neural architectures, it reduces the manual effort required by researchers and improves the clarity and technical accuracy of deep learning publications. Future work includes expanding support to other frameworks like PyTorch and enhancing customization options for diverse aesthetic requirements.
Summary updated