Deep Paper Gestalt

Jia-Bin HuangarXiv 2018 · arXiv

Abstract

The computer vision community faces a crisis due to the exponential growth of conference submissions. This paper investigates whether a deep learning model can predict paper acceptance based purely on visual "gestalt"—the overall look and layout of a paper. By representing papers as 2x4 page grids and training a ResNet-18 classifier, we achieve 92% accuracy on the CVPR 2018 dataset. Our results highlight the importance of visual polish, such as teaser figures and full page utilization, in successful submissions.

Introduction

The success of computer vision has led to a "reviewer fatigue" crisis. Conferences like CVPR and ICCV have seen submissions grow at a rate that far outpaces the availability of experienced reviewers. This imbalance leads to inconsistent and poorly justified feedback, as the pool of qualified reviewers is stretched thin.

Exponential growth in computer vision paper submissions vs. available reviewers.

Experienced researchers often develop an intuition for paper quality based on a quick glance at the layout—the "gestalt." This work attempts to automate this intuition, creating a potential pre-filter to reduce reviewer workload by identifying papers that lack the visual polish typically associated with high-quality research.

Methodology

We treat paper review as an image classification task. Each paper is converted into a single image by arranging its first eight pages into a 2x4 grid. This representation allows a convolutional neural network to analyze the spatial distribution of text, figures, and tables without needing to parse the underlying text.

Examples of paper layouts represented as 2x4 page grids used for training the classifier.

We utilized a ResNet-18 architecture pre-trained on ImageNet, modified for binary classification ("accepted" vs. "rejected"). To train the model, we introduced the Computer Vision Paper Gestalt (CVPG) dataset, consisting of 5,618 accepted papers from CVPR/ICCV (2013–2018) as positive examples and 1,503 workshop papers as a proxy for rejected submissions. Preprocessing steps included removing headers to prevent the model from identifying "Open Access" watermarks that only appear on accepted papers. Additionally, the paper explores generative tools like Progressively Growing GANs and CycleGAN to synthesize ideal layouts and suggest visual improvements, such as adding a teaser figure or filling out the final page.

Results

The model achieved a remarkable 92% accuracy on the CVPR 2018 test set. By tuning the classification threshold, the system could effectively reject 50% of "bad" papers while maintaining a false rejection rate of only 0.4% for "good" papers. Analysis via Class Activation Maps (CAMs) revealed that the model prioritizes features such as teaser figures on the first page, dense tables (suggesting comprehensive evaluation), and the use of the full page limit. Conversely, papers that end prematurely (e.g., halfway through the final page) are strongly associated with rejection. In a notable test of "self-irony," the model predicted that this very paper should be rejected with 97.4% probability, likely due to its shorter length during the testing phase.

Discussion and Limitations

While highly accurate, this approach is intentionally superficial. It ignores scientific content in favor of visual polish. A visually perfect paper with flawed logic might pass, while a groundbreaking paper with poor formatting might be rejected. Furthermore, the model is sensitive to specific LaTeX templates and may not generalize to other fields without retraining. Future work should explore higher-level structural abstractions to improve robustness and generalization across different academic disciplines.

Summary updated

BibTeX
@misc{huang2018deep,
  author = {Jia-Bin Huang},
  title = {{Deep Paper Gestalt}},
  year = {2018},
  eprint = {1812.08775},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1812.08775},
}
Figure