TL;DR
This paper introduces a novel, parameterless algorithm for automatically segmenting the object of interest (OOI) in low depth-of-field (DOF) images. By analyzing pixel sharpness and color, the method effectively separates the focused OOI from the blurred background, outperforming existing techniques in robustness and accuracy, especially on complex images.
What Problem Does This Paper Address?
In photography, low depth-of-field (DOF) is a common technique to draw attention to a specific subject by rendering it sharp while blurring the background. This creates an implicit segmentation that humans easily perceive. However, automatically segmenting these "objects of interest" (OOIs) from low DOF images is challenging for algorithms. Unlike images with clear edges, the transition between sharp and blurred regions can be gradual, making it difficult to define precise boundaries. This automatic segmentation is crucial for applications like image retrieval, adaptive compression, and content-based image analysis, where understanding the OOI is key.
Methods
The proposed algorithm employs a five-stage process:
- Deviation Scoring: This stage identifies sharp pixels by comparing the original image with a blurred version. The difference in edge information between the two provides a score, with higher scores indicating sharper regions. This method leverages the L*a*b* color space for more accurate color difference calculations compared to RGB.
- Score Clustering: Pixels with high sharpness scores are clustered using a density-based clustering algorithm (DBSCAN). This helps to distinguish genuine sharp regions of the OOI from isolated noise pixels. DBSCAN is chosen for its ability to handle non-convex shapes.
- Mask Approximation: A preliminary binary mask of the OOI is generated by computing the convex hull of the clustered sharp pixels and then filling these hulls. Morphological operations (closing and dilation by reconstruction) are applied to smooth this mask and close small holes.
- Color Segmentation: The approximate mask is further refined by grouping pixels within the mask that have similar colors in the original image, using the L*a*b* color space and a color distance threshold.
- Region Scoring: Finally, each color region is assigned a relevance score based on its surrounding sharpness scores. Regions with low relevance are removed, iteratively refining the mask to accurately delineate the OOI.
Datasets
The experiments were conducted on a diverse dataset of 65 images downloaded from Flickr and created by the authors. This dataset includes images from various categories with varying degrees of DOF, fuzziness, and background complexity, aiming to avoid overfitting to specific image types.
Key Insights and Ideas
- Leveraging Perceived Sharpness: The core idea is to mimic human perception by focusing on regions of sharpness, but doing so in a robust, automated way that goes beyond simple edge detection.
- L*a*b* Color Space for Robustness: Using the L*a*b* color space for color difference calculations provides a more perceptually uniform measure, leading to better results than traditional RGB.
- Density-Based Clustering for Complex Shapes: DBSCAN's ability to identify clusters of arbitrary shapes is crucial for segmenting OOIs that are not necessarily convex.
- Parameterless Design: A significant contribution is the algorithm's parameterless nature, with parameters being either image-independent or automatically determined, making it highly adaptable.
- Impact of DOF on Image Similarity: The paper demonstrates that segmenting OOIs in low DOF images significantly improves image similarity measures, particularly for content-based image retrieval.
Key Results
- The proposed algorithm achieves superior robustness and accuracy compared to three reference methods ([7], [11], [18]) across a diverse test set.
- The average spatial distortion error was 0.21, significantly lower than the best competing algorithm's average of 0.51.
- The algorithm performs well even on images with complex backgrounds and higher resolutions, outperforming competitors that struggle with these conditions.
- The paper shows that increasing image size generally leads to improved segmentation quality, with the algorithm maintaining accuracy.
- The algorithm's runtime increases with image size, but it remains competitive, especially for larger images where other methods lose accuracy.
Limitations and Future Work
- Runtime for Large Images: While accurate, the algorithm's runtime can be substantial for very large images, though it scales better than some competitors.
- Future Work: The authors plan to further improve processing speed and accuracy, apply the algorithm to video, and develop an automatic detection mechanism for low DOF images. They also intend to release their dataset and code as ImageJ plugins.
Summary updated