Peer Review #5 of "A novel CAPTCHA solver framework using deep skipping Convolutional Neural Networks (v0.3)"

Shida Lu, Kai Huang, Talha Meraj, Hafiz Tayyab Rauf, S Ahmed, K Anand, H Ahn, C Yim, F Alqahtani, F Alsulaiman, S Azad, K Jain, H Baird, K Popat, O Bostik, K Horak, L Kratochvila, T Zemcik, S Bilik, J Klecka, E Bursztein, M Martin, J Mitchell, A Moscicki, C Fabry, S Bethard, D Jurafsky, Y Cao, A Che, Y Liu, H Xiao, H Wang, K Zhang, H.-n Dai, K Chellapilla, K Larson, P Simard, M Czerwinski, J Chen, X Luo, J Wang, Y Ma, C Cruz-Perez, O Starostenko, F Uceda-Ponga, V Alarcon-Aquino, L Reyes-Cabrera, S Dankwa, L Yang, F.-l Du, J.-x Li, Z Yang, P Chen, B Wang, J Zhang, D Ferreira, L Leira, P Mihaylova, P Georgieva, J Gao, H Shen, Y Gao, H Gao, S Luo, Y Zi, S Zhang, W Mao, P Wang, Y Shen, J Yan, D George, W Lehrach, K Kansky, M Lzaro-Gredilla, C Laan, B Marthi, X Lou, Z Meng, M Gheisari, H Najafabadi, J Alzubi, G Wang, A Abbasi, A Castiglione, G Goswami, B PowellOpenalex Unknown 2022 · DOI · OpenAlex

TL;DR

Researchers have developed a "Skip-CNN" framework that significantly improves the accuracy of text-based CAPTCHA solvers by using skip connections to preserve fine-grained pictorial details that are often lost in standard deep learning models. By testing on datasets with complex features like overlapping characters and intersecting lines, the team achieved up to 98.82% accuracy, outperforming several existing state-of-the-art methods.

What Problem Does This Paper Address?

The "Completely Automated Public Turing test to tell Computers and Humans Apart" (CAPTCHA) is a cornerstone of web security, designed to block malicious bots while letting humans through. However, it’s a constant arms race. As CAPTCHA designers add noise, overlapping text, and intersecting lines to thwart bots, traditional Optical Character Recognition (OCR) and standard Convolutional Neural Networks (CNNs) begin to struggle.

The authors identify a specific technical gap: conventional CNNs use "pooling" layers to reduce data dimensionality, but this process can be "lossy," discarding subtle spatial features necessary to distinguish characters in a highly distorted image. Furthermore, many existing solvers are "dataset-specific," meaning they perform well on one type of CAPTCHA but fail when faced with a new style or a different number of characters.

Methods

The core innovation is a 16-layer Skip-connection-based CNN (Skip-CNN).

Standard CNNs process information linearly, but a "skip connection" allows information from an earlier layer to "jump" over intermediate layers and concatenate with later feature maps. In this model, the researchers saved the information from the first ReLU (Rectified Linear Unit) layer and injected it back into the network after the third ReLU layer. This ensures that the "pictorial information"—the raw shapes and edges of the characters—is preserved even as the deeper layers extract more abstract features.

The pipeline includes several critical preprocessing steps:

  1. Grayscale & Binarization: Using Bradley’s adaptive thresholding method to turn messy color images into clean black-and-white data.
  2. Erosion & Noise Removal: Specifically for CAPTCHAs with intersecting lines, the authors used a line-type structuring element to "erode" away the background lines while keeping the character shapes intact.
  3. 5-Fold Cross-Validation: Unlike many studies that use a simple train/test split, this paper employs 5-fold validation to ensure the results are robust and not biased by a specific subset of data.

Datasets

The study utilized two publicly available datasets from Kaggle:

  • $d_1$ (5-Character Dataset): Contains 1,040 images with 19 different character types. This dataset is particularly challenging because it features a horizontal line intersecting the text and overlapping characters.
  • $d_2$ (4-Character Dataset): A larger set of 9,955 images with 32 character types. While it has more variety in characters, it lacks the complex intersecting lines found in $d_1$.

Key Insights and Ideas

  • Feature Preservation is Key: The "aha moment" here is that for OCR-style tasks, the high-level abstraction of deep layers isn't enough. By re-introducing early-layer spatial data via skip connections, the model can better handle "segmentation" challenges where characters are touching or obscured by noise.
  • The "M" Challenge: Interestingly, the character 'M' proved to be the most difficult to recognize across both datasets (achieving the lowest accuracy). The authors suggest this is due to its complex structural morphology, which requires even more precise "structural polishing" during preprocessing.
  • Reliability through Folds: The authors argue that many previous CAPTCHA solvers overstate their success by using 1-fold validation. By using 5-fold validation, they provide a more "confident" and realistic metric for how these models would perform in the wild.

Key Results

The Skip-CNN framework showed impressive performance, particularly on the 4-character dataset:

  • 4-Character Dataset ($d_2$): Achieved a mean accuracy of 98.82% and an F1-score of 98.84%.
  • 5-Character Dataset ($d_1$): Achieved a mean accuracy of 85.52% and an F1-score of 85.97%. The lower score here reflects the significantly higher difficulty of the intersecting lines and overlapping text.
  • Comparative Success: The model outperformed several recent benchmarks, including a 15-layer CNN (80% accuracy) and various KNN/SVM approaches.

Limitations and Future Work

While the Skip-CNN is powerful, the 85% accuracy on the 5-character dataset shows there is still room for improvement when dealing with "unclear" or highly intersected text.

The authors suggest several future directions:

  • Data Augmentation: Implementing more robust data generation techniques to train the model on even more diverse noise patterns.
  • Complex Intersections: Developing better methods to handle "intersecting line-based" CAPTCHAs, which remain the most difficult for the model to "clean" during preprocessing.
  • Language Expansion: Applying the framework to CAPTCHAs using different alphabets or local languages.

Summary updated

Abstract

A Completely Automated Public Turing Test to tell Computers and Humans Apart (CAPTCHA) is used in web systems to secure authentication purposes; it may break using Optical Character Recognition (OCR) type methods.CAPTCHA breakers make web systems highly insecure.However, several techniques to break CAPTCHA suggest CAPTCHA designers about their designed CAPTCHA's need improvement to prevent computer visionbased malicious attacks.This research primarily used deep learning methods to break state-of-the-art CAPTCHA codes; however, the validation scheme and conventional Convolutional Neural Network (CNN) design still need more confident validation and multiaspect covering feature schemes.Several public datasets are available of text-based CAPTCHa, including Kaggle and other dataset repositories where self-generation of CAPTCHA datasets are available.The previous studies are dataset-specific only and cannot perform well on other CAPTCHA's.Therefore, the proposed study uses two publicly available datasets of 4-and 5-character text-based CAPTCHA images to propose a CAPTCHA solver.Furthermore, the proposed study used a skip-connection-based CNN model to solve a CAPTCHA.The proposed research employed 5-folds on data that delivers 10 Different CNN models on two datasets with promising results compared to the other studies.
BibTeX
@inproceedings{lu2022peer,
  author = {Shida Lu and Kai Huang and Talha Meraj and Hafiz Tayyab Rauf and S Ahmed and K Anand and H Ahn and C Yim and F Alqahtani and F Alsulaiman and S Azad and K Jain and H Baird and K Popat and O Bostik and K Horak and L Kratochvila and T Zemcik and S Bilik and O Bostik and J Klecka and E Bursztein and M Martin and J Mitchell and E Bursztein and A Moscicki and C Fabry and S Bethard and J Mitchell and D Jurafsky and Y Cao and A Che and Y Liu and H Xiao and H Wang and K Zhang and H.-N Dai and K Chellapilla and K Larson and P Simard and M Czerwinski and J Chen and X Luo and Y Liu and J Wang and Y Ma and C Cruz-Perez and O Starostenko and F Uceda-Ponga and V Alarcon-Aquino and L Reyes-Cabrera and S Dankwa and L Yang and F.-L Du and J.-X Li and Z Yang and P Chen and B Wang and J Zhang and D Ferreira and L Leira and P Mihaylova and P Georgieva and J Gao and H Wang and H Shen and J Gao and H Wang and H Shen and J Gao and H Wang and H Shen and Y Gao and H Gao and S Luo and Y Zi and S Zhang and W Mao and P Wang and Y Shen and J Yan and D George and W Lehrach and K Kansky and M Lzaro-Gredilla and C Laan and B Marthi and X Lou and Z Meng and Y Liu and H Wang and M Gheisari and H Najafabadi and J Alzubi and J Gao and G Wang and A Abbasi and A Castiglione and G Goswami and B Powell},
  title = {{Peer Review \#5 of "A novel CAPTCHA solver framework using deep skipping Convolutional Neural Networks (v0.3)"}},
  booktitle = {Openalex Unknown},
  year = {2022},
  doi = {10.7287/peerj-cs.879v0.3/reviews/5},
}

Loading related content...

Figure