Japanese Translation Quality Evaluation System Based on Deep Neural Network Algorithm

Rui ZhangInternational Journal of Circuits Systems and Signal Processing 2022 · DOI

Abstract

The current translation quality evaluation system relies on the combination of manual and text comparison for evaluation, which has the defects of low efficiency and large evaluation errors. In order to optimize the defects of the current quality evaluation system, a Japanese translation quality evaluation system based on deep neural network algorithm will be designed. In order to improve the processing efficiency of the system, the USB3.0 communication module of the hardware system will be optimized. Based on the hardware design, the reference translation map is used to extend the reference translation of Japanese translation. The evaluation indexes of over- and under-translation are set, and the evaluation of Japanese translation quality is realized after the parameters are determined by training the deep neural network using the sample set. The system functional test results show that the average data transmission processing time of the system is improved by about 31.27%, and the evaluation error interval is smaller and the evaluation is more reliable.

Introduction

Evaluating the quality of machine translation (MT) is a critical bottleneck in natural language processing. While manual evaluation remains the gold standard, it is prohibitively slow and expensive for large-scale datasets. Existing automated metrics, such as BLEU, often rely on rigid text comparisons that fail to account for the linguistic flexibility inherent in Japanese, where word order and synonyms can vary significantly without altering meaning. Furthermore, many MT models suffer from specific pathologies like "over-translation" (repetition) or "under-translation" (omission), which standard metrics struggle to quantify. This research introduces a high-performance evaluation system that combines hardware-accelerated throughput with a deep neural network (DNN) algorithm to provide more accurate and efficient translation assessments.

Hardware Architecture and Throughput Optimization

To address the efficiency limitations of software-only evaluation systems, this research implements a custom hardware architecture designed for high-speed data processing. The system utilizes an Altera Cyclone III FPGA (EP3C10E144C8N) as the main control module, integrated with a CYUSB3014 controller to implement a high-speed USB 3.0 communication interface.

The hardware architecture of the system, featuring an FPGA-based main control module and optimized communication interfaces.

This hardware-centric approach allows the system to ingest massive translation datasets with minimal latency. A 128GB Flash chip array is employed to ensure that large translation dictionaries and reference sets are readily accessible, bypassing traditional disk I/O bottlenecks and enabling real-time data playback and processing.

Algorithmic Methodology

1. Reference Translation Map

To overcome the rigidity of static string matching, the system employs a "reference translation map" represented as a directed graph. This graph captures multiple valid paths for a single translation, accounting for synonyms and varied grammatical structures. A candidate translation is considered correct if it aligns with a valid path within this map.

2. Specialized Metrics: OTEM and UTEM

The system introduces two quantitative metrics to target specific translation errors:

  • Over-Translation Evaluation Metric (OTEM): Utilizes N-gram matching to detect redundant repetitions in the output.
  • Under-Translation Evaluation Metric (UTEM): Identifies missing essential N-grams from the source text.

These metrics, combined with a length penalty (LP), are fed into a multi-layer Deep Neural Network. The DNN is trained on a corpus of 1.25 million Chinese-Japanese sentence pairs from the LDC database, learning to correlate these N-gram mismatches with actual translation quality scores.

Experimental Results

The system was benchmarked against standard automated evaluation systems using the MT02, MT03, and MT04 datasets. The hardware optimizations resulted in a significant performance boost, reducing the average data transmission and processing time from 7.61 seconds to 5.23 seconds—a 31.27% improvement in efficiency.

Comparative analysis of evaluation errors, showing the proposed system's improved reliability and smaller error interval compared to a contrast system.

As shown in the comparative analysis, the proposed system exhibits a smaller evaluation error interval and higher consistency. By aligning more closely with human judgment and providing stable scores across complex Japanese syntax, the system proves to be a more reliable tool for MT development than traditional contrast systems.

Conclusion

By integrating FPGA-based hardware acceleration with a flexible DNN-driven evaluation algorithm, this research provides a robust solution for Japanese translation quality assessment. The system successfully balances high throughput with linguistic nuance, offering a scalable framework for future multi-language performance models.

Summary updated

Abstract

The current translation quality evaluation system relies on the combination of manual and text comparison for evaluation, which has the defects of low efficiency and large evaluation errors. In order to optimize the defects of the current quality evaluation system, a Japanese translation quality evaluation system based on deep neural network algorithm will be designed. In order to improve the processing efficiency of the system, the USB3.0 communication module of the hardware system will be optimized. Based on the hardware design, the reference translation map is used to extend the reference translation of Japanese translation. The evaluation indexes of over- and under-translation are set, and the evaluation of Japanese translation quality is realized after the parameters are determined by training the deep neural network using the sample set. The system functional test results show that the average data transmission processing time of the system is improved by about 31.27%, and the evaluation error interval is smaller and the evaluation is more reliable.
BibTeX
@article{zhang2022japanese,
  author = {Rui Zhang},
  title = {{Japanese Translation Quality Evaluation System Based on Deep Neural Network Algorithm}},
  journal = {International Journal of Circuits Systems and Signal Processing},
  year = {2022},
  doi = {10.46300/9106.2022.16.53},
}
Figure