The Bell Boundary of Shared Representations

Yang Wu, Hanyu Zhao, Yazhou Zhang, Yuexian HouarXiv 2026 · arXiv

TL;DR

This paper introduces the Bell Boundary, a novel diagnostic tool for multi-task learning derived from quantum physics. By applying Bell’s theorem to neural networks, the authors demonstrate that when a shared representation lacks the capacity to support multiple tasks, task correlations violate the classical CHSH bound ($|S| \leq 2$), providing a principled way to detect representational bottlenecks.

What Problem Does This Paper Address?

Multi-task learning (MTL) is built on the hope that a single shared representation can support many different objectives. However, we often see "task interference" or "negative transfer," where learning one task hurts another. While researchers have developed many ways to mitigate this (like gradient balancing), we still lack a fundamental understanding of when and why a shared representation becomes insufficient.

The authors ask: Is there a measurable, principled boundary beyond which a single representation can no longer jointly support multiple tasks? They tackle this by looking at representation capacity not just as a performance metric, but as a constraint on the joint probability distribution of task outcomes.

Methods

The researchers draw a brilliant analogy between the CHSH inequality (a cornerstone of Bell’s theorem in quantum mechanics) and shared representations in AI.

In physics, the CHSH inequality tests if correlations between two particles can be explained by a "local hidden variable." In MTL, the "hidden variable" is the shared representation $h$. The authors propose that if four binary task outcomes (e.g., "is the answer correct?") are jointly supported by one representation, their pairwise correlations must satisfy $|S_{NN}| \leq 2$.

To test this, they developed NCnet, a controlled toy model where they could precisely vary the number of hidden neurons. They also established a pairwise training protocol:

  1. Define two task branches ($A$ and $B$), each with two settings ($a_1, a_2$ and $b_1, b_2$).
  2. Train four separate models for each task pair combination (e.g., $a_1$ with $b_1$).
  3. Calculate the correlations $E_{ij}$ and combine them into the statistic $S_{NN} = E_{11} + E_{12} + E_{21} - E_{22}$.
Overview of the Bell Boundary framework showing the analogy between CHSH experiments and NCnet.

Datasets

The authors validated their theory across several domains:

  • Synthetic: A 4-bit binary feature dataset designed for XOR and identity tasks.
  • Vision (CNNs): A two-digit MNIST variant where tasks involved detecting or classifying digits on the left and right sides of an image.
  • NLP (BERT): Homogeneous tasks using PAWS-X (multilingual paraphrase) and heterogeneous tasks combining SST-2 (sentiment), MRPC (paraphrase), CommonsenseQA, and MathQA.
  • Reasoning (LLMs): Using Qwen3-8B with LoRA adapters on a mix of MathQA, MATH, RACE-Middle, and LogiQA2.0.

Key Insights and Ideas

  • The Bell Boundary: The authors identify a specific transition. At very low capacity, the model underperforms (low $S_{NN}$). As capacity increases to a "critical" level, the model begins to fit the tasks but the representation is too cramped, causing $|S_{NN}|$ to exceed 2. As capacity increases further, the tasks become compatible again, and $S_{NN}$ returns to the bounded region ($\leq 2$).
  • Incompatibility is Capacity-Dependent: A violation ($|S_{NN}| > 2$) is a mathematical "smoking gun" that the shared representation is insufficient to jointly account for the tasks.
  • Heterogeneity Matters: Homogeneous tasks (like paraphrasing in different languages) rarely cross the boundary because they "want" the same features. Heterogeneous tasks (like math vs. sentiment) compete for space, making them much more likely to hit the Bell Boundary.
  • Connection to Gradient Conflict: The peak violation of the Bell inequality coincides with the highest levels of gradient conflict, suggesting that the Bell Boundary is a representation-level manifestation of optimization struggles.
The Bell Boundary across different model scales, showing the characteristic "hump" where S_NN exceeds 2.

Key Results

  • NCnet: In the toy setting, $S_{NN}$ peaked at 2.31 when the hidden layer had only 3 neurons, falling back to 2.0 as neurons increased to 7+.
  • CNNs: The vision models showed a massive violation, peaking at $S_{NN} = \mathbf{2.96}$ at specific depth/width combinations before returning to the bound as the network grew.
  • LLMs: Even in massive models like Qwen3-8B, using a low LoRA rank (e.g., $r=2$ or $4$) for heterogeneous tasks caused $S_{NN}$ to exceed 2 (reaching 2.05). Increasing the rank to 64 restored compatibility ($S_{NN} \approx 2.0$).

Limitations and Future Work

The current framework is limited to binary task outcomes (correct vs. incorrect). While this simplifies the math, it might discard nuanced information present in continuous outputs or multi-class distributions. Additionally, the requirement to train four separate task-pair models to compute the statistic is computationally expensive for very large-scale applications.

Future work could extend this to continuous variables, explore adaptive task grouping based on Bell violations, and use the Bell Boundary as a real-time signal to trigger "MoE-style" (Mixture of Experts) capacity expansion in foundation models.

Summary updated

Abstract

When does a shared representation become insufficient for jointly supporting multiple tasks? This question lies at the heart of multi-task learning models. Although task interference in shared representations has been widely observed, its onset, measurable form, and capacity boundary remain poorly understood. In this paper, we revisit this problem through the lens of Bell's theorem. We show that if four binary task outcomes can be jointly represented by a common shared representation, their correlations must satisfy the classical CHSH bound $|S|\leq2$. We introduce NCnet, a controlled multi-task model for studying how representational capacity affects task compatibility. Experiments reveal a clear capacity-dependent transition. Once nontrivial task fitting emerges, constrained capacity can yield $|S|>2$. As capacity further increases, these exceedances progressively disappear and $S$ returns to the bounded region. This systematic pattern provides evidence that limited representational capacity is an important source of the observed incompatibility. We observe similar transitions across CNNs, PLMs, and LLMs, suggesting that the phenomenon extends beyond the controlled NCnet setting. We refer to this capacity-dependent transition as the Bell Boundary, providing a measurable diagnostic for representation bottlenecks, task incompatibility, and capacity allocation in multi-task models.
BibTeX
@misc{wu2026bell,
  author = {Yang Wu and Hanyu Zhao and Yazhou Zhang and Yuexian Hou},
  title = {{The Bell Boundary of Shared Representations}},
  year = {2026},
  eprint = {2610.16828},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.16828},
}
Figure