Introduction
Deep Neural Networks (DNNs) are highly susceptible to adversarial perturbations—small, often imperceptible changes to input data that cause misclassification. Randomized smoothing has become a prominent defense strategy, providing provable robustness by adding random noise to inputs and using a majority vote for prediction. This process creates a "smoothed" classifier that is guaranteed to be robust within a specific radius.
However, traditional randomized smoothing typically assumes isotropic noise, resulting in spherical "safe" regions. While later improvements like ANCER introduced anisotropic (ellipsoidal) regions, these were restricted to being axis-aligned. Real-world decision boundaries are rarely aligned with pixel axes, meaning these methods often fail to capture the full volume of the safe region, leading to overly conservative robustness guarantees.
The RANCER Approach
To overcome these limitations, we introduce RANCER (Rotation-based ANCER). The core innovation is the ability to orient the certification ellipsoid in any direction, allowing it to better fit the local geometry of the classifier's decision boundary.
Hessian-Guided Alignment
Finding the optimal orientation for every input could be computationally prohibitive. RANCER utilizes the Hessian (the matrix of second-order derivatives) of the loss function at the input point as a mathematical guide. The eigenvectors of the Hessian identify directions of high and low curvature, effectively mapping out the local "valleys" and "ridges" of the loss landscape.
By aligning the axes of the smoothing distribution with these eigenvectors, RANCER ensures that the ellipsoid stretches along the "safe" directions (low curvature) and remains narrow in unstable directions (high curvature). Following rotation, the algorithm optimizes the lengths of these axes to maximize the certified volume. To maintain consistency across the data space and prevent overlapping regions with conflicting labels, a memory-based approach is employed to store and check transformation matrices.
Key Insights and Visual Intuition
The primary advantage of RANCER is its ability to exploit the "valley" structure of neural network loss landscapes. If a safe region is a long, tilted corridor, an axis-aligned oval must remain small to avoid crossing the boundary. RANCER’s rotated ellipsoids can expand significantly further into these regions.
As shown in the 2D toy examples above, the pink RANCER ellipsoids hug the contours of the class boundaries much more effectively than the isotropic or axis-aligned alternatives, resulting in a larger certified volume.
Experimental Results
We evaluated RANCER on the CIFAR-10 dataset using a ResNet-18 architecture. The results demonstrate a significant leap in certified robustness:
- Certified Accuracy: At an $\ell_2$ radius of 0.25, RANCER achieved 81% certified accuracy, outperforming ANCER (75%) and standard isotropic smoothing (56%).
- Average Certified Radius (ACR): RANCER improved the ACR to 0.53, compared to 0.46 for ANCER.
Limitations and Future Work
While RANCER sets a new state-of-the-art for certified accuracy, it introduces higher computational costs. Calculating the Hessian and performing eigendecomposition makes RANCER approximately 2.5x slower than ANCER and 5x slower than isotropic smoothing. Additionally, the memory-based consistency check requires storing transformation matrices for every certified point, which poses a scalability challenge for high-resolution datasets like ImageNet. Future research will focus on optimizing these memory requirements and speed through more efficient data structures.
Summary updated