Abstract
Chemical structure recognition is a critical task at the intersection of chemistry and computer vision. Traditional Optical Chemical Structure Recognition (OCSR) techniques often suffer from information loss or misidentification in complex tasks due to structural diversity. Deep learning models, while promising, frequently encounter issues with high architectural complexity, loss of contextual information, and low recognition rates. This paper proposes a novel chemical structure recognition method combining an attention mechanism with an encoder-decoder architecture. The approach utilizes a modified ResNet50 for feature extraction, a Bi-directional Long Short-Term Memory (BLSTM) network for row-wise encoding to enhance spatial information, and a coverage-attention-based LSTM decoder to translate images into Simplified Molecular Input Line Entry System (SMILES) sequences. Experimental results across multiple datasets, including CASIA-CSDB and USPTO, demonstrate that the proposed method achieves superior accuracy compared to state-of-the-art models like Image2Mol and ChemPix, even with limited training data.
Introduction
Converting chemical structures from images into machine-readable formats like SMILES strings is essential for digitizing chemical knowledge and accelerating drug discovery. However, traditional rule-based methods struggle with the complexity of molecular layouts and noisy backgrounds. Deep learning approaches offer a solution but often require immense data and struggle with maintaining fine-grained spatial relationships or avoiding repetitive outputs in the decoding phase.
Proposed Methodology
The proposed framework employs a specialized encoder-decoder architecture designed to capture intricate molecular details and maintain global context.
Encoder: Feature Extraction and Spatial Reinforcement
The encoder consists of two primary components:
- Modified ResNet50: To preserve high-resolution spatial features necessary for identifying small chemical details (e.g., double bonds, atom labels), the standard ResNet50 is truncated by removing the final two stages and global pooling layers.
- Row-wise BLSTM: A Bi-directional LSTM processes the feature maps row-by-row. This step reinforces the spatial connectivity and context of the molecular structure before it is passed to the decoder.
Decoder: Coverage-Attention Mechanism
The decoder uses an LSTM network integrated with a "coverage" attention mechanism. This mechanism tracks previously attended regions of the image, preventing the model from generating redundant characters or skipping parts of the molecule—a common failure mode in standard attention decoders.
Experimental Evaluation
The model was evaluated on a wide range of datasets, including large-scale benchmarks like CASIA-CSDB (~480,000 images) and various external patent and journal datasets (USPTO, ACS, Indigo).
Key Results
The method demonstrated significant performance gains:
- Accuracy: Achieved an Exact Match (EM) accuracy of 84.42% on CASIA-CSDB and 85.78% on a smaller subset (Mini CASIA-CSDB), indicating high data efficiency.
- Generalization: On the USPTO patent dataset, it reached an EM of 58.21%, outperforming baseline models like SwimOCSR and ChemPix.
- Efficiency: The architecture achieves competitive results with significantly fewer training samples than many existing deep learning models.
Conclusion
By combining a high-resolution CNN backbone with row-wise spatial encoding and a coverage-aware decoder, this method provides a robust solution for chemical structure recognition. Future work will focus on improving the recognition of stereochemistry and enhancing robustness against low-quality or highly complex molecular scans.
Summary updated