Introduction
In the evolution of Natural Language Processing (NLP), certain experimental paradigms have become so entrenched that they define the "standard" way of doing science. This prototypical setup—training a standard architecture on labeled English data and optimizing solely for accuracy—is what researchers call the Square One Bias. While this focus helped the field mature, it has created a "one-dimensional" rut where innovation typically moves along only a single axis at a time, leaving the complex interactions between fairness, efficiency, and multilinguality largely unexplored.
The One-Dimensional Trap
The "research manifold" represents the space of all possible NLP discoveries across multiple dimensions. Ideally, research would explore the interior of this space, looking at how different goals interact. However, a manual classification of recent literature reveals that most papers that depart from the "Square One" prototype do so in a strictly linear fashion.
As seen in the visualization above, papers tend to cluster either at the origin (Square One) or along the primary axes. A paper might explore fairness, but it will do so only in English. Another might explore multilinguality but ignore computational efficiency. This one-dimensionality is often a survival strategy for researchers; the community rewards concise, incremental progress that is easy for reviewers to digest. Unfortunately, this means we miss critical "blind spots." For example, weight pruning (an efficiency technique) has been shown to disproportionately hurt performance on minority demographics—a failure mode that is invisible if you only study efficiency in a vacuum.
Quantitative Evidence from ACL 2021
To quantify this bias, the authors annotated 461 oral papers from ACL 2021 across four key dimensions: multilinguality, fairness/bias, efficiency, and interpretability. The results paint a stark picture of a field tethered to its origins.
The meta-analysis revealed that approximately 70% of papers evaluate only on English, and nearly 40% use only Accuracy or F1 as their primary metric. Most tellingly, only 6.1% of papers made a contribution along two or more dimensions simultaneously. Even in specialized tracks, the silos remain: as shown in the table, papers in the "Ethics in NLP" track were 100% focused on fairness but 0% on multilinguality or efficiency.
Systemic Bottlenecks
Why is the field so resistant to multi-dimensional research? The authors identify several systemic barriers:
- Education: Introductory NLP courses often frame the field through the lens of English-centric tasks like sentiment analysis. Survey data shows that 75% of teachers start with these "Square One" examples, imprinting them as the default scientific method for new researchers.
- Organizational Silos: Conference tracks are organized by topic (e.g., "Fairness" vs. "Machine Translation"). A paper that explores the intersection of both might be seen as "not deep enough" for either track, leading to rejection in favor of narrower, more specialized work.
- The "State-of-the-Art" (SOTA) Race: The pressure to beat a single accuracy number on a leaderboard discourages researchers from considering trade-offs, such as a model that is slightly less accurate but significantly more efficient or fair.
Recommendations for a Multi-Dimensional Future
To break out of the Square One Bias, the authors propose several practical shifts:
- Multi-Dimensional Leaderboards: We should move toward "Pareto frontiers" that visualize the trade-offs between speed, fairness, and language coverage rather than a single-column ranking.
- Reviewer Reform: Conferences should incentivize "inter-areal" reviews, ensuring that papers exploring intersections are evaluated by people who value the breadth of the contribution.
- Curriculum Updates: Educators should introduce multilinguality and ethics as core components of "NLP 101" rather than optional, advanced topics.
By acknowledging the Square One Bias, the NLP community can begin to explore the full manifold of research, moving beyond simple accuracy to build systems that are robust, equitable, and efficient across the world's languages.
Summary updated