Mobile vision networks such as MobileNetV2 are built from inverted residual blocks. Each block widens a narrow feature map about sixfold, filters each widened channel spatially on its own, then compresses back down. Because the spatial filtering never mixes channels, every widened channel is a self-contained path through the block: one row of the expansion layer, one private spatial filter, and one column of the compression layer. We propose that channel-wise dropout belongs on exactly these paths. Channel-wise dropout zeroes whole feature maps at random during training rather than individual pixels. Applied this way, it is equivalent to classic dropout on the hidden units of a two-layer network, the setting where dropout is best understood. The mask goes after the block's last normalization and before its final linear projection. That keeps it clear of the normalization-variance problem that makes dropout unreliable in modern convolutional networks.
The question matters because the literature gives two opposite instructions for these networks. The MobileNet authors report that small separable networks need little regularization and little or no dropout (MobileNets). Channel-dropout work on large networks finds strong co-adaptation among sparse deep channels, especially on fine-grained tasks (Weighted Channel Dropout; Channel DropBlock). Co-adaptation here means channels that learn to rely on each other and on a single distinctive cue. Our claim is that both are right about different parts of the network. Separable layers with no widening have little redundancy, but the widened interior of an inverted residual block is wide enough to co-adapt. If that is true, the benefit of path dropout should grow with the expansion factor and with data scarcity.
Core Claim
Write the inverted residual block as
$$ y = x + \sum_{j=1}^{tC} m_j \, p_j \, h_j(x), $$
where $x$ is the block input with $C$ channels and $t$ is the expansion factor. The term $h_j(x)$ is the $j$-th widened channel after its expansion row, depthwise (per-channel) spatial filter, batch normalization and ReLU6. The vector $p_j$ is the $j$-th column of the final linear projection, and $m_j$ is a Bernoulli keep-mask scaled by its inverse keep probability. The sum makes the structure explicit: the block's residual update is a sum of $tC$ independent rank-one contributions, and $m_j$ removes whole contributions.
This placement satisfies the guideline of Kim et al.. They showed that dropout distorts batch normalization statistics unless it sits after the last normalization and before the last weight layer of a residual branch. Their positions P5/P6 gained 0.44% on CIFAR-100, while dropout at the block input cost over 55%. Their analysis covered only standard ResNet blocks; for MobileNetV2 they tested only element-wise dropout in the classification head. The masked channel is also already behind a nonlinearity. We therefore expect it to avoid the information loss Xception reported when adding nonlinearities between depthwise and pointwise layers in unexpanded blocks. That expectation is a hypothesis we test, not a premise.
The idea is new in one specific way. Prior channel-dropout methods treat a channel as a feature map inside a dense network: SpatialDropout drops channels uniformly, WCD drops them weighted by activation, and Channel DropBlock drops correlated groups. None has been run on separable architectures. None uses the fact that in a separable block a channel is an entire independent subnetwork, which is what makes one placement principled and lets us predict where the method helps and where it hurts.
Research Program
We will first build path dropout into MobileNetV2, and into DPDNet-style blocks (PSDNet and DPDNet) as a second widened separable family, inside the apple/ml-cvnets training library. Following DropBlock, the drop rate ramps linearly from zero during training (DropBlock). The only new hyperparameter is a single peak drop rate, shared across the last two network stages.
The second step tests the mechanism, not just the accuracy. The path view implies that a channel masked before the depthwise filter and a channel masked after it remove the same path. The two placements differ only in what the depthwise batch normalization sees during training. Comparing them therefore isolates the variance-shift effect inside separable blocks: Kim's theory predicts that the pre-filter mask hurts and the post-normalization mask does not. A third placement, at the narrow linear bottleneck output, tests whether masking low-dimensional features damages them as MobileNetV2 warns that ReLU does. A fourth placement, between depthwise and pointwise layers in an unexpanded MobileNetV1-style block, tests the tension with Chollet's result.
The third step tests the reconciliation claim. We sweep the expansion factor $t \in \{1, 3, 6\}$ and the training-set fraction. The prediction is that gains are near zero or negative at $t=1$ and on full ImageNet, consistent with Howard et al. They should be positive and growing at $t=6$ on fine-grained data.
The fourth step uses the same masks at test time for Monte Carlo uncertainty: averaging predictions over several random masks. This extends the calibration result of Zhang et al. from PreAct-ResNet to mobile networks.
Baselines
The naive reference is MobileNetV2 trained with no dropout beyond the standard recipe. Sandler et al. report 72.0% ImageNet top-1 for this model; we retrain it ourselves under our exact schedule, so comparisons are within our runs.
The cheapest serious competitor is element-wise dropout placed before global average pooling in the head, the configuration Kim et al. found reliably helpful on MobileNetV2. The file gives no exact ImageNet number for it, so no reported result was found on Lacuna for MobileNetV2 with pre-pooling dropout, and we will measure it.
The strongest existing internal method is DropBlock applied in the last two stages of MobileNetV2 with its schedule. On ResNet-50 it lifted ImageNet top-1 from 76.51% to 78.13% (Ghiasi et al.), but no reported result was found on Lacuna for DropBlock on any separable network. We also include two methods at the same block positions as our mask: SpatialDropout applied to block outputs, and WCD applied at our position. These test whether weighting adds anything over uniform path masks. Finally, we run stochastic depth, the DropLayer variant that gave the best calibration (ECE 3.08% on CIFAR-100 with PreAct-ResNet, Zhang et al.).
Two ablations isolate the key ingredient. Element-wise dropout at the identical position tests whether the whole-path structure matters. The pre-depthwise mask tests whether the placement matters.
Every method gets the same data, schedule, and number of epochs, plus an equal tuning budget of six drop-rate or block-size settings chosen on a held-out split, with three seeds each. On fine-grained data, beating a baseline means a mean gain larger than two seed standard deviations over the best-tuned competitor. On ImageNet, we treat a gain of at least 0.3 top-1 over tuned head dropout as meaningful. Matching DropBlock with one hyperparameter instead of two would be a partial win; losing to it would falsify the claim that path structure matters.
Why not just…?
Why not just shrink the width multiplier if the network overfits? Thinning removes capacity at inference for every input, and Howard et al. show accuracy falls steeply when networks are thinned aggressively. Path dropout keeps full width at test time. The concern is still fair, so we include a width-matched comparison: a model trained with path dropout at keep rate $k$ against a model with width multiplier chosen to match its average active width during training. If the thinner model matches, the method is merely a costly capacity reduction.
Why not just put dropout before pooling in the head, as Kim et al. recommend? It is the cheapest fix and the most likely to tie on full ImageNet, where our own hypothesis predicts little internal co-adaptation. DropBlock's gains on ResNet came from masking throughout the convolutional hierarchy, not only at the head. Whether that carries over to separable networks is exactly what the fine-grained and data-fraction sweeps measure. If head dropout ties there too, we will report that internal masking is unnecessary for mobile networks.
Why not just use DropBlock inside MobileNetV2? This is the real threat. DropBlock masks contiguous spatial regions, which attacks spatial leakage within a channel. Path dropout attacks co-adaptation across widened channels. These are different failure modes, and the head-to-head comparison, plus a combined DropBlock-plus-path arm, will show whether they are complementary or redundant.
Why not just use stochastic depth, which already drops whole residual branches? Stochastic depth removes all $tC$ paths of a block at once; path dropout removes a random subset. The calibration and accuracy comparison against stochastic depth decides whether finer granularity buys anything. If it does not, the simpler block-level method wins.
Proposed Experiments
ImageNet training of MobileNetV2 at $t=6$ establishes whether path dropout is harmless at scale. We then fine-tune ImageNet-pretrained MobileNetV2 and DPDNet on CUB-200-2011 and Stanford Cars. These are the benchmarks where WCD (VGG-16 on CUB, 72.31% to 76.10%) and Channel DropBlock (ResNet-50 on CUB, 86.6% to 87.2%) found channel co-adaptation in dense networks. CIFAR-100 serves for the expansion-factor and data-fraction sweeps and for calibration, measured as expected calibration error under Monte Carlo masking. Class-activation maps on CUB check whether the evidence the model uses spreads across object parts, as reported for dense networks. We will also report training-time overhead, since masking cost is a known drawback of structured dropout.
Three results are decisive. The first is whether the post-normalization mask beats the pre-depthwise mask, which the path view says differ only through normalization statistics. The second is whether gains scale with the expansion factor, vanishing at $t=1$. The third is whether path dropout beats tuned DropBlock and stochastic depth on fine-grained data. If all three come out as predicted, the Howard-versus-WCD disagreement has a structural answer. If the second fails, the expansion-space co-adaptation story is wrong, even if accuracy improves.
References
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Weighted Channel Dropout for Regularization of Deep Convolutional Neural Network
- Channel DropBlock: An Improved Regularization Method for Fine-Grained Visual Classification
- How to use dropout correctly on residual networks with batch normalization
- Xception: Deep Learning with Depthwise Separable Convolutions
- Efficient object localization using Convolutional Networks
- PSDNet and DPDNet: Efficient channel expansion, Depthwise-Pointwise-Depthwise Inverted Bottleneck Block
- DropBlock: A regularization method for convolutional networks
- MobileNetV2: Inverted Residuals and Linear Bottlenecks
- Confidence Calibration for Convolutional Neural Networks Using Structured Dropout.