Web-based clothing searcher utilizing convolutional neural networks and dissimilarity rules for color classification in the HSL color space and ORB algorithm for garment characterization

Leonardo Ayala, Martín Montes Rivera, Alberto Ochoa Zezzatti, Julio Ponce, José Ignacio Guzman MontotoInternational Journal of Combinatorial Optimization Problems and Informatics 2025 · DOI

Abstract

This study proposes a methodology for searching for clothing on the web using convolutional neural networks (CNN) and a color obtaining module. The system is based on the creation of a color base in the HSL (Hue, Saturation, Luminosity) space. It determines the color of an image based on dissimilarity rules. The base image, previously processed by a CNN, along with the result of obtaining the color, generates a string of characters. This string is entered into a search engine, resulting in garments that resemble the base image. These garments are processed with the ORB algorithm to extract their features and compare them with those of the base image. In this way, it can be determined which of the garments is most like the base image. This work offers new perspectives and useful techniques for obtaining colors, significantly improving the accuracy and efficiency of object search on the web.

Introduction

Finding a specific piece of clothing online using only an image is a non-trivial task. While "search by image" tools exist, they often struggle with the nuance of specific cuts, textures, and precise color matching. The authors identify a gap in how automated systems handle the "semantic-to-visual" bridge: a search engine needs a text query (e.g., "blue trousers"), but a user only has pixels. Furthermore, once a search engine returns hundreds of results, determining which one matches the style and cut of the original remains difficult. This paper proposes a hybrid methodology to automate this entire loop, combining deep learning for high-level semantics with classical computer vision for fine-grained morphological matching.

Methodology

The researchers developed a four-step "identify-search-refine" architecture to bridge the gap between user photos and e-commerce inventory.

1. Classification via CNN

A sequential Convolutional Neural Network (CNN) is employed to identify the garment category (e.g., shirt, dress, pants). The model utilizes standard layers for feature extraction and dimensionality reduction.

The sequential CNN architecture used for garment classification, featuring Conv2D, Maxpooling, and Dense layers with ReLU activation functions.

2. Color Extraction in HSL Space

To avoid the lighting sensitivity of the standard RGB color space, the system converts images to HSL (Hue, Saturation, Lightness). By defining 12 specific "color centers" and calculating the Euclidean distance between image pixels and these centers, the system determines the dominant color.

3. The Search Bridge

The system concatenates the identified color and garment type into a text string (e.g., "black shirt"). This string is fed into the SerpApi (a Google Search API) to scrape relevant product images from the web.

4. Morphological Ranking

To find the best match among search results, the authors use the ORB (Oriented FAST and Rotated BRIEF) algorithm. ORB detects keypoints to compare the shape and morphology of the garments. A final scoring formula combines these features with RGB sum comparisons to rank the results.

Datasets and Preprocessing

The primary training data was the Apparel Dataset sourced from Kaggle, containing thousands of images across categories like Dress, Pants, Shirt, Shoes, and Skirt. To ensure the CNN's robustness against various photographic angles and conditions, the authors utilized extensive data augmentation.

Examples of data augmentation techniques, including rotation and scaling, applied to the training dataset to prevent overfitting.

Key Results and Insights

  • High Accuracy: The CNN achieved a 90% accuracy rate across garment categories, with confidence levels as high as 99.53% for specific test cases like black t-shirts.
  • Effective Ranking: The ORB-based ranking system successfully sorted web-scraped images, placing the most morphologically similar items at the top of the list.
  • Hybrid Intelligence: The system effectively acts as a translator, turning visual data into a structured text query that leverages existing web search infrastructure rather than building a global image index from scratch.

Conclusion and Future Work

This methodology significantly improves the accuracy of object search on the web by combining semantic classification with morphological refinement. Future work will focus on reducing computational latency for real-time analysis and improving segmentation for "in-the-wild" photos where garments are set against complex backgrounds.

Summary updated

Abstract

This study proposes a methodology for searching for clothing on the web using convolutional neural networks (CNN) and a color obtaining module. The system is based on the creation of a color base in the HSL (Hue, Saturation, Luminosity) space. It determines the color of an image based on dissimilarity rules. The base image, previously processed by a CNN, along with the result of obtaining the color, generates a string of characters. This string is entered into a search engine, resulting in garments that resemble the base image. These garments are processed with the ORB algorithm to extract their features and compare them with those of the base image. In this way, it can be determined which of the garments is most like the base image. This work offers new perspectives and useful techniques for obtaining colors, significantly improving the accuracy and efficiency of object search on the web.
BibTeX
@article{ayala2025web,
  author = {Leonardo Ayala and Martín Montes Rivera and Alberto Ochoa Zezzatti and Julio Ponce and José Ignacio Guzman Montoto},
  title = {{Web-based clothing searcher utilizing convolutional neural networks and dissimilarity rules for color classification in the HSL color space and ORB algorithm for garment characterization}},
  journal = {International Journal of Combinatorial Optimization Problems and Informatics.},
  year = {2025},
  doi = {10.61467/2007.1558.2025.v16i3.514},
}
Figure