Peer review is the primary mechanism for quality control and scientific validation across academic conferences and journals, as highlighted by David Poole 0001 et al. in both the Preface (aaai 2013) and Preface (aaai 2014). However, because evaluation is performed by human experts, it is inherently vulnerable to cognitive shortcuts and social biases. When reviewers know who wrote a paper, they may unintentionally judge the work based on the authors' institutional prestige, gender, nationality, or past reputation rather than the text itself.
To combat this, the scientific community designs specific information-masking protocols. By systematically controlling who knows what during the evaluation process, these mechanisms decouple the technical merit of a submission from the social status of its creators.
The Spectrum of Information Masking
The peer review landscape operates along a spectrum of anonymity, with each tier targeting a different vector of potential bias.
Single-Blind Review
In a single-blind workflow—traditionally used by journals such as the IEEE Transactions on Control of Network Systems (see Anuradha Annaswamy et al. in Information for Authors (IEEE Transactions on Control of Network Systems 2020) and Thomas Parisini et al. in IEEE Transactions on Control of Network Systems Information for Authors (IEEE Transactions on Control of Network Systems 2023))—the reviewers know the identities of the authors, but the authors do not know who reviewed their paper.
While this asymmetry protects reviewers from professional retaliation and allows them to provide candid critiques, it leaves the evaluation completely exposed to prestige bias. Reviewers may give well-known laboratories the benefit of the doubt while scrutinizing newcomers with disproportionate skepticism.
Double-Blind (Double-Anonymous) Review
To eliminate prestige and demographic biases, many technical communities have shifted to a double-blind model. Here, identity information is withheld from both sides: authors do not know their reviewers, and reviewers do not know the authors.
As explained by Joseph L. Gabbard et al. in Message from the ISMAR 2019 Science and Technology Program Chairs and TVCG Guest Editors (IEEE Transactions on Visualization and Computer Graphics 2019), concealing author identities ensures that research is evaluated strictly on its technical content rather than the reputation of the contributing individuals or institutions. This protocol has been widely adopted across diverse, highly competitive fields to maintain fairness, including:
- Photogrammetry and Remote Sensing: As detailed by C. Mallet et al. in THE REVIEWING PROCESS FOR ISPRS EVENTS (ISPRS 2018) and J. D. Wegner et al. in PREFACE: TECHNICAL COMMISSION II (ISPRS 2022).
- Mixed and Augmented Reality: Discussed by Hideo Saito et al. in Message from the ISMAR 2020 Science and Technology Program Chairs (ISMAR 2020).
- Services and Computational Intelligence: Noted by Ernesto Damiani et al. in IEEE 2020 World Congress on Services Message from the Program Chairs in Chief (SERVICES 2020) and Marco Gori et al. in Conference Report on 2022 IEEE World Congress on Computational Intelligence (IEEE WCCI 2022) (IEEE Comput. Intell. Mag. 2022).
- Information Retrieval and Image Computing: Documented by Ruihua Song et al. in Preface (EVIA@NTCIR 2013) and Ajmal Saeed Mian et al. in Neural computing and applications (NCAA) special issue on best of DICTA 2019 papers (Neural Comput. Appl. 2021).
By stripping away names and affiliations, the double-blind process acts as a cognitive filter. It forces reviewers to build their evaluation entirely from the paper's internal logic, methodology, and data, minimizing social bias (Ruihua Song et al.).
Triple-Blind Review
Even in double-blind systems, editors who handle the papers and choose reviewers still see the authors' identities, which can introduce bias during the initial screening or reviewer assignment phases. To mitigate this, some venues implement a triple-blind protocol. As described by Jinman Kim et al. in Editorial (Vis. Comput. 2014), this mechanism anonymizes authors, reviewers, and editors, ensuring that the entire administrative pipeline remains focused solely on technical novelty.
Why Masking Works: Information Containment
The core intuition behind identity masking is information containment. Human brains are highly efficient at using heuristics (like an author's famous institution) as a proxy for quality. When these heuristics are available, they pollute the evaluation.
[With Affiliation Metadata]
Author Name/Institution ──> Triggers Heuristic Bias ──> Distorted Evaluation of Merit
[With Identity Masking]
Anonymized Text Only ──> Forces Deep Cognitive Processing ──> Objective Evaluation of Merit
By removing this metadata, the reviewer experiences an information asymmetry that can only be resolved by deeply processing the text. Mechanisms like reviewer self-selection (as noted by Biplav Srivastava in Preface (aaai 2012)) further ensure that the unmasked technical content aligns with the reviewer's actual expertise.
However, masking is not a perfect solution. In niche research communities, unique writing styles, specific datasets, or heavy self-citations can allow reviewers to guess the authors' identities anyway. Furthermore, Frank C. Park et al. in Rethinking the Research Paper [President's Message] (IEEE Robotics Autom. Mag. 2022) point out that while double-anonymous reviews are designed to reduce demographic and institutional bias, their long-term empirical effectiveness in fostering true innovation remains unproven and requires ongoing study.
The Alternative Paradigm: Transparency via Open Evaluation
Instead of hiding information to prevent bias, an alternative movement argues for the exact opposite: complete transparency.
The traditional closed-door peer review process can mask low-quality critiques, hidden biases, and self-serving gatekeeping because reviewers operate in secret. To solve this, the "Open Evaluation" paradigm moves the review process into the public eye.
As synthesized by Nikolaus Kriegeskorte et al. in An emerging consensus for open evaluation: 18 visions for the future of scientific publishing (Frontiers in Computational Neuroscience 2012), making reviews and ratings publicly accessible subjects the evaluators themselves to community scrutiny. This shift changes the incentive structure: when a reviewer's reputation is tied to the quality of their critique, they are highly motivated to provide objective, constructive, and evidence-based feedback, mitigating bias through public accountability rather than anonymity.
Key Foundational Work
An emerging consensus for open evaluation: 18 visions for the future of scientific publishing (2012) — contributed 2 concept elements to this direction.
Preface (2013) — contributed 1 concept elements to this direction.
Preface (2013) — contributed 1 concept elements to this direction.