Abstract
Sign language recognition is a critical field for enhancing accessibility, yet many current systems are hindered by high computational costs or reliance on expensive sensors. This paper introduces the Dual-path Background Erasure Convolutional Neural Network (DPCNN), a lightweight model designed to isolate hand gestures by explicitly modeling and subtracting background noise. Evaluated on the ASL Finger Spelling dataset, the DPCNN achieves a 99.52% accuracy and a Macro-F1 score of 0.997. With only 60,280 parameters, the model is optimized for deployment on resource-constrained mobile and IoT devices.
Introduction
Effective sign language recognition (SLR) systems often face a trade-off between accuracy and accessibility. High-performance systems frequently utilize depth sensors or wearable gloves, which are costly and limit portability. Conversely, vision-based systems using standard RGB cameras often struggle with "noisy" real-world backgrounds or require massive deep learning architectures that exceed the processing power of mobile devices. This study proposes a novel DPCNN architecture that addresses these challenges through a physically motivated "background erasure" mechanism.
Methodology: Dual-Path Background Erasure
The core innovation of the DPCNN is its dual-path architecture, which processes input images through two parallel pipelines:
- The Primary Path: This path extracts "overall features," encompassing both the hand gesture and the surrounding environment. It utilizes Average Pooling to capture global data trends.
- The Auxiliary Path: Specifically designed to model the background, this path uses Max Pooling to identify the most salient local features, such as the sharp edges of background objects.
The Erasure Mechanism
At four distinct stages within the network, the features extracted by the auxiliary path are subtracted from those in the primary path ($M_{main} - M_{auxiliary}$). This explicit subtraction "erases" the background noise, leaving a refined representation of the hand gesture for final classification. The resulting features are flattened and passed through a fully connected layer (128 units) before reaching the final 24-unit output layer.
Experimental Setup and Datasets
The model was benchmarked using the ASL Finger Spelling dataset, which consists of approximately 65,702 RGB images across 24 static alphabetic categories (A-Y, excluding dynamic gestures 'J' and 'Z').
Unlike laboratory datasets with uniform backgrounds, this dataset features complex, non-uniform environments, providing a rigorous test for the model's generalization capabilities.
Results and Discussion
The DPCNN demonstrated state-of-the-art performance while maintaining an exceptionally small footprint:
- Accuracy and Precision: The model achieved a total accuracy of 99.52% and a Macro-F1 score of 0.997, indicating high reliability across all gesture categories.
- Efficiency: The architecture contains only 60,280 parameters (0.06M). This is significantly smaller than alternative models, which range from 1.88M to 21.24M parameters, making the DPCNN ideal for "small terminals" like smartphones.
- Ablation Study: Comparisons between feature subtraction and standard feature addition confirmed that "erasing" noise is superior to merging it with the signal, resulting in lower loss and higher accuracy.
Conclusion
The DPCNN provides a robust, lightweight solution for vision-based sign language recognition. By leveraging a dual-path architecture to subtract background noise, the model achieves high accuracy on complex datasets without the need for massive parameter counts. Future work will focus on extending this "erasure" concept to dynamic gestures and full-sentence recognition.
Summary updated