Abstract
Traditional image filters often blur edges because they center the processing window on the pixel, inevitably incorporating data from both sides of a boundary. Side Window Filtering (SWF) addresses this by shifting the window so the pixel sits on its side or corner, allowing the filter to "stay on one side" of an edge and preserve sharpness. It is a versatile, plug-and-play framework that significantly improves classic filters like Box, Gaussian, and Median filters.
The Problem with Centered Windows
In computer vision, local window-based filtering is a fundamental operation used for denoising, smoothing, and detail enhancement. For decades, the standard approach has been to place the pixel being processed at the exact center of the window.
However, this approach has a fundamental flaw: it blurs edges. When a pixel lies on an edge, a centered window captures data from both sides of that boundary. Averaging these pixels (as in Box or Gaussian filters) blurs the edge, creating "halos" or artifacts. While complex edge-preserving filters like the Bilateral Filter exist, they can be computationally expensive or fail in tricky scenarios. The authors of this paper propose a simple alternative: why must the pixel be in the center?
The Side Window Filtering (SWF) Method
The authors propose Side Window Filtering (SWF). Instead of one centered window, they define eight different "side windows" where the target pixel is located on a side or a corner:
- 4 Side-aligned: Up, Down, Left, Right.
- 4 Corner-aligned: North-West, North-East, South-West, South-East.
The SWF Algorithm:
- Filter: Apply your chosen filter (e.g., a Box filter) to all eight side windows independently.
- Compare: Calculate the difference (L2 distance) between the original pixel value and the output of each of the eight filters.
- Select: Choose the output from the window that resulted in the smallest change to the pixel.
The logic is elegant: if a pixel is on an edge, the windows that "cross" the edge will produce a value very different from the original pixel. The window that stays entirely on one side of the edge will produce a value very close to the original. By picking the "closest" result, the algorithm naturally preserves the edge.
Key Insights and Ideas
- The Geometry of Edges: Using Taylor expansion, the authors prove mathematically that to accurately reconstruct a pixel on an edge, you must only use neighbors from one side of that edge.
- A Universal Framework: SWF isn't a new filter; it’s a meta-filter. You can take almost any existing local filter (Box, Gaussian, Median, Bilateral, Guided Filter) and turn it into a "Side Window" version (S-BOX, S-GAU, etc.).
- Heuristic Intelligence: The "minimum distance" selection criteria is a powerful heuristic. It allows the filter to adaptively choose the best local geometry without needing complex global optimization.
- Colorization Fix: SWF can prevent "color leakage" in optimization-based colorization, where colors from one object accidentally bleed into another across a boundary.
Key Results
- Edge Preservation: In qualitative tests, S-BOX and S-GAU filters produced significantly sharper edges and fewer artifacts than their traditional centered counterparts.
- Quantitative Gains: Using the SSIM (Structural Similarity Index), the SWF versions consistently outperformed standard filters. For example, in image smoothing, S-GUI (Side Window Guided Filter) achieved an SSIM of 0.948 compared to 0.815 for the standard version.
- Texture Removal: The "Iterated Side Window Weighted Guided Filter" (IS-WGF) was shown to be the only filter capable of preserving complex structures (like a boat's hull) while completely removing high-frequency textures (like ocean waves).
- Speed: While SWF requires more computation (roughly 8x the operations of a standard filter), it remains highly efficient. An S-BOX filter on a 1-megapixel image takes about 0.215 seconds on a standard CPU, making it viable for real-time applications.
Limitations and Future Work
- Computational Overhead: Although efficient, calculating eight windows is slower than calculating one. However, these calculations are independent and can be easily parallelized on a GPU.
- Roof Edges: While SWF is nearly perfect for "step" and "ramp" edges, it still struggles slightly with "roof" edges (sharp peaks), though it still performs better than traditional centered windows.
- Deep Learning Integration: The authors suggest that the SWF principle—aligning windows with edges—could be integrated into Convolutional Neural Networks (CNNs), potentially improving how neural networks handle boundaries and fine details.
Summary updated