BCN: Batch Channel Normalization

Afifa Khaled, Chao Li, Kun HeOpenReview 2024 · Submission to ICLR 2024 · Withdrawn · OpenReview

Its authors withdrew this paper, so there is no full text to read or summarize.

Abstract

Normalization techniques enable higher learning rates and are less careful in initialization. Unlike the standard Batch Normalization (BN) and Layer Normalization (LN), where BN computes the mean and variance along the (N, H, W) axes (N is the batch axes, H and W are the spatial height and width axes) and LN computes the mean and variance along the (C, H, W) axes (C is the channel axes), this paper presents a simple normalization technique called Batch Channel Normalization (BCN). BCN separately normalizes inputs along the (N, H, W) and (C, H, W) axes, then combine the normalized outputs based on adaptive parameters. BCN exploits both the channel and batch dependence and adaptively combines the advantages of BN and LN. As a basic block, BCN can be easily integrated into existing models for various applications in the field of computer vision. Empirical results show that the proposed (BCN) technique improves the generalization performance of various models.
BibTeX
@misc{khaled2024bcn,
  author = {Afifa Khaled and Chao Li and Kun He},
  title = {{BCN: Batch Channel Normalization}},
  year = {2024},
  url = {https://openreview.net/forum?id=lM2In9x2c7},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2024},
}
Figure