A Parallel Corpus of Theses and Dissertations Abstracts

Felipe Soares, Gabrielli Harumi Yamashita, Michel Jose Anzanello

This paper presents the development and evaluation of a new, large-scale Portuguese-English parallel corpus derived from academic abstracts. By leveraging open data from the Brazilian government, the authors have created a significant resource for the Natural Language Processing (NLP) community, particularly for those working on Machine Translation (MT) in scientific domains.

TL;DR

The authors introduced a new parallel corpus of ~1.3 million Portuguese-English sentence pairs extracted from over 240,000 Brazilian thesis and dissertation abstracts. Models trained on this domain-specific data outperformed Google Translate in translating academic text, highlighting the continued importance of specialized datasets in the era of "general-purpose" AI.

What Problem Does This Paper Address?

Modern Machine Translation—whether Statistical (SMT) or Neural (NMT)—is incredibly data-hungry. To translate accurately, these systems require "parallel corpora": massive collections of sentences that are direct translations of one another.

While general-purpose corpora (like movie subtitles or parliamentary proceedings) exist, they often fail to capture the nuances of scientific writing. Academic prose has a specific "register"—it is formal, uses specialized terminology, and follows rigid structural norms. There is a persistent shortage of high-quality, large-scale parallel data specifically for the Portuguese-English language pair in the scientific domain. This paper fills that gap by tapping into the CAPES Theses and Dissertations Catalog (TDC).

Methods

The researchers followed a multi-stage pipeline to transform raw database records into a clean, aligned corpus:

  1. Data Retrieval: They downloaded CSV files from the CAPES open data portal covering the years 2013–2016.
  2. Filtering & Cleaning: They removed entries missing either the Portuguese or English abstract. A crucial step involved "language checking" to ensure abstracts weren't accidentally swapped into the wrong fields.
  3. Preprocessing: Because the source data was entirely in uppercase, they performed "case folding" to lowercase. They also had to modify standard sentence segmentation algorithms, which typically rely on capitalization to identify sentence boundaries.
  4. Alignment: They used the LF Aligner (a wrapper for the popular Hunalign tool). This algorithm uses a combination of sentence-length heuristics and bilingual dictionaries to match a Portuguese sentence with its English counterpart.
  5. Evaluation: To prove the corpus's worth, they trained two types of translation models: a Moses-based SMT and an OpenNMT-based neural model.

Datasets

The primary contribution is the CAPES TDC Parallel Corpus.

  • Size: 239,018 documents resulting in 1,289,372 aligned sentence pairs.
  • Volume: Approximately 34 million tokens per language.
  • Diversity: The data is categorized into 9 knowledge areas, including Health Sciences (the largest segment), Engineering, Humanities, and Agricultural Sciences.
  • Availability: The corpus is provided in TMX (Translation Memory eXchange) format, the industry standard for translation data, and as an SQLite database containing rich metadata (university, year, keywords).

Key Insights and Ideas

  • Domain Specificity is King: The most striking insight is that a model trained on 1.3 million domain-specific sentences can beat a "state-of-the-art" general model like Google Translate (GT) when tested on that same domain. GT is trained on billions of sentences, but its "jack-of-all-trades" nature makes it less precise for the specific verbal regency and formal style of academic abstracts.
  • The "Janitorial" Work of NLP: The paper highlights the non-trivial challenges of data cleaning. For instance, the fact that the source data was all-caps required the authors to re-think sentence segmentation, as traditional tools often look for a capital letter following a period to define a boundary.
  • Metadata Adds Multi-Dimensional Value: By including metadata like "Knowledge Area," the authors have created more than just a translation resource; they've created a dataset for sociolinguistic research, allowing others to study how scientific language differs across disciplines (e.g., how an abstract in "Humanities" differs structurally from one in "Exact Sciences").

Key Results

The authors used the BLEU score (a standard metric for MT quality, where higher is better) to evaluate their models against Google Translate:

  • NMT Superiority: The Neural Machine Translation (OpenNMT) model was the top performer, particularly in the English $\rightarrow$ Portuguese direction.
  • Outperforming Google: In the English $\rightarrow$ Portuguese test, OpenNMT achieved a BLEU score of 43.22, significantly higher than Google Translate’s 38.92.
  • Alignment Quality: A manual evaluation of 400 random sentence pairs showed that 82.30% were perfectly aligned. Only 4.35% were complete mismatches, usually due to the alignment tool getting "lost" in very long or unusually formatted abstracts.

Limitations and Future Work

The authors acknowledge that segmentation errors remain the primary source of noise. If a Portuguese sentence is split into two but the English translation remains as one, the alignment tool often produces "partial" matches (13.33% of the corpus).

Future work suggests using this corpus for tasks beyond translation, such as:

  • Cross-language text mining: Identifying trends in research topics across borders.
  • Automatic classification: Training models to identify the "Knowledge Area" of a text based on its abstract.
  • Hybrid Models: Concatenating this scientific data with other multi-domain corpora to create more robust, general-purpose Portuguese-English translators.

Summary updated

Abstract

In Brazil, the governmental body responsible for overseeing and coordinating post-graduate programs, CAPES, keeps records of all theses and dissertations presented in the country. Information regarding such documents can be accessed online in the Theses and Dissertations Catalog (TDC), which contains abstracts in Portuguese and English, and additional metadata. Thus, this database can be a potential source of parallel corpora for the Portuguese and English languages. In this article, we present the development of a parallel corpus from TDC, which is made available by CAPES under the open data initiative. Approximately 240,000 documents were collected and aligned using the Hunalign tool. We demonstrate the capability of our developed corpus by training Statistical Machine Translation (SMT) and Neural Machine Translation (NMT) models for both language directions, followed by a comparison with Google Translate (GT). Both translation models presented better BLEU scores than GT, with NMT system being the most accurate one. Sentence alignment was also manually evaluated, presenting an average of 82.30% correctly aligned sentences. Our parallel corpus is freely available in TMX format, with complementary information regarding document metadata.
BibTeX
@inproceedings{soares2018parallel,
  author = {Felipe Soares and Gabrielli Harumi Yamashita and Michel Jose Anzanello},
  title = {{A Parallel Corpus of Theses and Dissertations Abstracts}},
  booktitle = {PROPOR},
  year = {2018},
  url = {https://openreview.net/forum?id=XpmfN6nHAtz},
}

Loading related content...

Figure