Finding where something is in an image, such as the corner of an eye or a wrist, contains a hidden "pick the largest" step: somewhere the network must compare many candidate positions and keep the best one. Theory about ReLU networks (networks built from the piecewise-linear unit max(0, x)) shows that computing an exact maximum over $n$ inputs requires depth that grows with $\log n$. With real-valued weights, extra width can approximate this computation away. Real deployed models are different. Their weights are low-bit fixed-point or floating-point numbers, which are rational numbers with a fixed base. For such weights, the depth barrier is provably unaffected by width or by added precision. This note proposes that the barrier becomes a practical problem once weights are quantized aggressively, because the wide, fine-coefficient approximations that normally let shallow networks cheat need exactly the precision that low-bit weights lack.
Coarse-to-fine cascades are models that first estimate a rough location on the whole image and then crop a small patch around it and refine. Seen through this lens, a cascade does two things that theory says matter. The crop is a data-dependent indexing operation, not a piecewise-linear function, so the depth bound does not apply across it. And each refinement stage only has to choose among the $m \ll n$ positions inside its crop. The proposal is to test, first on controlled synthetic tasks and then on face and body landmark benchmarks, whether low-bit localization accuracy follows the depth-versus-$\log(\text{candidates})$ law. It also tests whether cropping beats the other ways of escaping the bound (max-pooling, a soft-argmax head, simply going deeper) at matched compute.
Core Claim
We claim that under low-bit weights, the error of a shallow ReLU localizer has a floor that width cannot remove once depth falls below roughly $\log$ of the number of candidate positions it must discriminate. We further claim that coarse-to-fine cropping lowers the depth each stage needs, because it replaces $n$ with the crop size $m$. The practical prediction is falsifiable. As bit-width drops, monolithic single-stage regressors should lose accuracy faster than cascades of equal total compute, and the refinement stage's required depth should track $\log m$ rather than $\log n$.
The theoretical starting point comes from three papers. Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes shows that integer-weight networks need at least $\lceil \log_2 n \rceil$ hidden layers to compute $\max\{0, x_1, \dots, x_n\}$, using parity of polytope volumes. On the Expressiveness of Rational ReLU Neural Networks With Bounded Depth extends this to weights that are base-$N$ fractions, using volumes modulo a prime $p$ that does not divide $N$. For decimal weights the bound is
$$ k \;\ge\; \lceil \log_3(n+1) \rceil, $$
where $k$ is the number of hidden layers and $n$ the number of inputs. In general the bound is $\Omega(\ln n / \ln\ln N)$. That paper also shows that neither width nor finer denominators compensate. Hardware weights are binary fractions ($N=2$), and 3 does not divide 2, so we expect the same logarithmic form to hold. Confirming that the paper's statement covers $N=2$ is the first check we will do, not an assumption we make.
Benefits of Max Pooling in Neural Networks supplies the approximate, real-weight counterpart. Below $\lceil \log_2 d \rceil$ layers, a ReLU network approximating a $d$-way maximum keeps a residual that shrinks only with exponential width. These results motivate the proposal but are also its limit. They concern exact or approximate scalar maxima in flat networks. None of them studies spatial regression, cropping, or trained quantized models. That gap is the one we fill.
Research Program
The work has three layers.
Theory. We will write down the approximation analogue of the rational result. The question is whether the subpool-max-average residual from the max-pooling paper stays bounded away from zero at depth below $\log n$ when the weights are $b$-bit fixed-point with power-of-two scales and width is arbitrary. If proving this fully is out of reach, we will compute it numerically for small $n$ by exhaustive search over quantized weights. We will also state formally why a crop escapes the bound. The cascade is a composition of piecewise-linear maps with a discontinuous gather, so the polytope argument applies to each stage separately, with $n$ replaced by the crop's candidate count.
Synthetic localization. We will build a task where $n$ is controlled exactly. A 1D or 2D grid of $n$ cells contains one peak plus clutter, and the network regresses the peak's sub-cell coordinate. We will train with quantization-aware training (training in which the forward pass uses rounded weights) at 32, 8, 4, and 2 bits, sweeping depth and width. The key picture is a heatmap of error over depth and width at each bit-width. Theory predicts a vertical wall near depth $\approx \log n$ that width fails to cross at low bits but does cross at 32 bits. We then add the three escape routes. A max-pooling stage computes the maximum directly, which the rational-weights paper notes sidesteps the bound. A soft-argmax head computes a softmax-weighted average of positions. A two-stage crop cascade picks an $m$-cell window and refines within it. We compare each route at matched multiply-accumulate count.
Real benchmarks. We will repeat the comparison on WFLW (98 facial points under blur, occlusion, and pose), on 300W with its Challenging subset, and on LSP for body pose. The single-stage and cascade detectors are quantized to the same bit-widths, and we fit the observed accuracy-versus-depth curves against the synthetic law.
Baselines
The naive reference point is the full-precision model of each architecture with no quantization, together with the same model after plain post-training rounding to each bit-width. These two tell us how much accuracy quantization costs before anyone tries to fix it.
Strongest single-stage competitor. This is PFLD, a single-stage MobileNet-V2 regressor whose training is regularized by an auxiliary head that predicts head pose and is dropped at inference. It is the strongest challenger because it shows a cascade is unnecessary at full precision. The file reports NME 4.98 on 300W Challenging for PFLD 1X+, against 6.60 for SAN and 6.98 for LAB, from the PFLD paper. We found no reported result on Lacuna for PFLD on WFLW.
Strongest cascade competitor. This is the Sparse Local Patch Transformer (SLPT). It crops small patches around current landmark estimates, shrinks them across stages, and relates landmarks with attention. It reports NME 4.14% on WFLW with an HRNetW18C-lite backbone. We found no reported result on Lacuna for SLPT on 300W or COFW, and no reported low-bit result for any landmark model.
Fairness. Both models are reimplemented and trained on identical splits with identical augmentation. Each gets the same hyperparameter search budget per bit-width, uses the same quantization-aware training recipe, and is compared at matched inference multiply-accumulates and measured mobile latency.
What counts as winning. Beating a baseline means that, at 4 bits and below, the cascade's NME degradation relative to its own full-precision score is significantly smaller than PFLD's, with confidence intervals from at least three seeds, at equal compute. If the cascade only wins at full precision, the thesis is not supported.
Ablations. The ablation that isolates the key ingredient keeps the SLPT architecture but replaces the hard crop with full-image cross-attention at the same compute. This removes the reduction from $n$ to $m$ while keeping everything else. A second ablation varies crop size $m$ at fixed stage depth, to test whether required depth tracks $\log m$.
Why not just…?
Why not just use standard quantization-aware training with more width or mixed precision? This is the most common fix, and the synthetic width sweep answers it directly. The rational-weights result says width cannot restore exact representation. The open question is whether it restores good approximation at low bits. If the error wall disappears with width at 4 bits, our claim is wrong and we will report that. Mixed precision, meaning keeping the last few layers at 16 bits, is included as a baseline at an equal total bit budget. If it matches the cascade, the practical lesson becomes "keep the comparison layers precise," which is still a finding the theory explains.
Why not just add a soft-argmax or max-pooling head instead of a cascade? This is a real threat. Both are primitives outside the piecewise-linear family and should escape the bound as well as a crop does. The synthetic and benchmark comparisons at matched compute are designed to settle this. Our prior is that a head over the full map still has to discriminate all $n$ positions in a low-bit feature map, while a crop reduces $n$. If a quantized PFLD with a soft-argmax head matches the cascade, we will narrow the claim to "any non-piecewise-linear max step suffices" and drop the cascade-specific part.
Why not just make the single-stage network deeper? Theory agrees that depth works, so this is not a strawman. The comparison is at matched latency and multiply-accumulates. If a deeper monolithic network matches the cascade at equal cost, cropping offers no advantage beyond depth. What would remain is a depth-selection rule derived from $\log n$, which Deep Convolutional Network Cascade for Facial Point Detection and DeepPose chose by hand.
Decisive Checks
Synthetic wall. The first decisive check is whether the synthetic depth-width error map shows a wall near $\log n$ that width crosses at 32 bits but not at 2–4 bits. Without this wall, the rest of the program is unmotivated and stops.
Shift with $n$. The second check doubles $n$ repeatedly and asks whether the wall moves by a constant depth increment, as a logarithmic law predicts.
Benchmark ordering. On WFLW and 300W at 4 bits, the cascade should degrade less than PFLD at matched compute. The full-attention ablation should lose that advantage. In the crop-size ablation, refinement depth needed for a fixed NME should grow with $\log m$. If all three hold, we will have given the first explanation, grounded in theory, of why coarse-to-fine design matters more when weights are cheap. If any fails, the specific failure will tell us which escape route from the depth barrier matters in practice.
References
- Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes
- On the Expressiveness of Rational ReLU Neural Networks With Bounded Depth
- Benefits of Max Pooling in Neural Networks: Theoretical and Experimental Evidence
- PFLD: A Practical Facial Landmark Detector
- Sparse Local Patch Transformer for Robust Face Alignment and Landmarks Inherent Relation Learning
- Deep Convolutional Network Cascade for Facial Point Detection
- DeepPose: Human Pose Estimation via Deep Neural Networks