A Novel Short-Memory Sequence-Based Model for Variable-Length Reading Recognition of Multi-Type Digital Instruments in Industrial Scenarios

Shenghan Wei, Xiang Li, Yong Yao, Suixian YangAlgorithms 2023 · DOI

TL;DR

This paper introduces a novel "short-memory" sequence-based model for recognizing variable-length readings from diverse digital instruments in industrial settings. By combining a residual network-inspired feature extractor with a BasicRNN sequence module, the model effectively handles variations in fonts, spacing, and lengths, achieving robust performance even on unseen data.

What Problem Does This Paper Address?

In industrial environments, digital instruments (like multimeters, gauges, etc.) are ubiquitous for displaying readings. However, many lack direct digital interfaces, necessitating manual reading. This manual process is inefficient, prone to errors, and risky in hazardous conditions. While Optical Character Recognition (OCR) offers a solution, existing methods struggle with the unique challenges of digital instrument readings: variable lengths, diverse fonts, inconsistent spacing, and the presence of special characters. Unlike standardized tasks like license plate recognition, these readings are often non-standardized, making robust automated recognition difficult. This paper aims to bridge this gap by developing an OCR system specifically tailored for these complex industrial scenarios.

Methods

The proposed model is a three-part system:

  1. Feature Extractor: This component uses a Convolutional Neural Network (CNN) enhanced with "shortcut connections" (inspired by ResNet). This design helps capture rich, high-dimensional features from the instrument images, particularly addressing the loss of small character details that can occur in standard CNNs.
  2. Sequence Modeling Module: This module employs a Bidirectional BasicRNN (Bi-BasicRNN). The key innovation here is its focus on "short-distance dependencies." Unlike LSTMs or GRUs that might overemphasize long-term patterns, this model prioritizes local relationships between characters, which is argued to be more suitable for the relatively independent nature of characters in instrument readings. This approach aims to improve robustness and generalization.
  3. Decoding Module: Connectionist Temporal Classification (CTC) is used for decoding. CTC is a standard technique for sequence-to-sequence tasks where alignment between input and output is not explicitly known. It allows for end-to-end training and handles variable-length outputs.

The authors specifically chose BasicRNN over LSTM/GRU because they hypothesize that focusing on short-distance dependencies, rather than long-distance ones, leads to better generalization and fewer "memorial errors" (misrecognizing unseen data as similar to seen data).

Datasets

The study utilized two datasets, A and B, constructed from a combination of collected and crawled images of digital instruments.

  • Dataset A: Created based on the principle of independent and identically distributed data, with training and testing sets having an equilibrium distribution of string types, fonts, spacing, and aspect ratios. The training-to-testing ratio was 10:1.
  • Dataset B: Designed to test robustness and generalization, especially for "invisible data" (readings not present in the training set). Approximately 23% of the reading strings in the test set were absent from the training set, and the remaining 77% featured significant differences in fonts, spacing, and aspect ratios compared to the training data. The training-to-testing ratio was also approximately 10:1.

Both datasets contained readings of variable lengths from 1 to 6 digits. The character set included digits 0-9, along with special characters like "E", "-", and ".".

Key Insights and Ideas

  • Short-Memory is Key: The paper strongly advocates for prioritizing short-distance dependencies in sequence modeling for this specific task. The intuition is that characters in instrument readings are often more locally related than globally, and over-reliance on long-term memory can lead to misinterpretations, especially with unseen data.
  • Residual Connections for Feature Richness: Incorporating shortcut connections into the CNN feature extractor significantly improves the ability to capture fine-grained features, crucial for distinguishing between similar-looking characters or handling variations in fonts and styles.
  • "Invisible Data" Matters: The design of Dataset B highlights the critical need for models that generalize well to data not encountered during training, a common challenge in real-world industrial applications.
  • BasicRNN's Niche: While LSTMs and GRUs are powerful, the study suggests that simpler RNNs like BasicRNN can be more effective when the task benefits from a focus on local context and avoids the pitfalls of over-generalization to similar, but incorrect, training examples.

Key Results

  • Feature Extractor Effectiveness: The skip-connection CNN feature extractor outperformed a plain CNN structure, achieving higher String-Level Recognition Accuracy (SRA) and Character-Level Recognition Accuracy (CRA) on both datasets. For the complex Dataset B, SRA improved from 0.740 to 0.831.
  • Sequence Modeling Necessity: Adding the Bi-BasicRNN sequence module improved SRA on Dataset A from 0.969 to 0.990, demonstrating its value in capturing sequential dependencies.
  • BasicRNN Superiority for Generalization: On the challenging Dataset B (designed for invisible data), the BasicRNN-adopted model achieved an SRA of 0.897, significantly outperforming LSTM (0.815), GRU (0.839), and SRU (0.808). It also produced substantially fewer memorial errors (6 vs. 44, 36, 35 respectively).
  • Robustness: The model demonstrated strong generalization and robustness, particularly for unseen data, as evidenced by its performance on Dataset B.

Limitations and Future Work

  • Dataset Size: While efforts were made to create comprehensive datasets, the authors acknowledge that the number of collected images (1723) and crawled images (10,520) might still be limited for extremely diverse industrial scenarios.
  • Computational Cost: Although the model aims for efficiency, deep learning models can still be computationally intensive. Further optimization for real-time deployment in resource-constrained industrial settings might be necessary.
  • Future Work: The paper implicitly suggests that further exploration of short-memory RNN architectures or hybrid approaches could yield even better results. Expanding the dataset with more diverse instrument types and challenging conditions would also be beneficial.

Summary updated

Abstract

As a practical application of Optical Character Recognition (OCR) for the digital situation, the digital instrument recognition is significant to achieve automatic information management in real-industrial scenarios. However, different from the normal digital recognition task such as license plate recognition, CAPTCHA recognition and handwritten digit recognition, the recognition task of multi-type digital instruments faces greater challenges due to the reading strings are variable-length with different fonts, different spacing and aspect ratios. In order to overcome this shortcoming, we propose a novel short-memory sequence-based model for variable-length reading recognition. First, we involve shortcut connection strategy into traditional convolutional structure to form a feature extractor for capturing effective features from characters with different fonts of multi-type digital instruments images. Then, we apply an RNN-based sequence module, which strengthens short-distance dependencies while reducing the long-distance trending memory of the reading string, to greatly improve the robustness and generalization of the model for invisible data. Finally, a novel short-memory sequence-based model consisting of a feature extractor, an RNN-based sequence module and the CTC, is proposed for variable-length reading recognition of multi-type digital instruments. Experimental results show that this method is effective on variable-length instrument reading recognition task, especially for invisible data, which proves that our method has outstanding generalization and robustness in real-industrial applications.
BibTeX
@inproceedings{wei2023novel,
  author = {Shenghan Wei and Xiang Li and Yong Yao and Suixian Yang},
  title = {{A Novel Short-Memory Sequence-Based Model for Variable-Length Reading Recognition of Multi-Type Digital Instruments in Industrial Scenarios}},
  booktitle = {Algorithms},
  year = {2023},
  doi = {10.3390/a16040192},
}
Figure