Introduction
Access to information is a fundamental right, yet linguistic complexity remains a significant barrier for many. In the Netherlands, approximately 2.5 million citizens struggle with low literacy, making essential communications—from government mandates to news articles—difficult to comprehend. While Text Simplification (TS) is a mature field in Natural Language Processing for English, it remains underdeveloped for "low-resource" languages like Dutch. The ARTIST project (ARTificial Intelligence for Simplified Text) explores the potential and pitfalls of using generative AI to bridge this accessibility gap.
The ARTIST Pipeline
The researchers developed a configurable web-based pipeline to test two primary simplification strategies for Dutch text:
- Dutch_T5: A Text-to-Text Transfer Transformer model fine-tuned specifically on Dutch news datasets.
- GoogleTransl (The Pivot Strategy): This method translates complex Dutch into English, applies a high-performing English simplification model (
TS_T5), and translates the result back into Dutch.
The system is designed to be highly interactive, allowing users to toggle various readability metrics—such as Flesch-Kincaid, SMOG, and the Dutch-specific Flesch Douma—to evaluate the output in real-time.
Methodology and Datasets
To benchmark these models, the study utilized the Canon van Nederland (CvN) dataset, which consists of 50 historical topics. This dataset provides a "gold standard" by offering manually aligned pairs of complex text (secondary education level) and simplified text (primary education level). By comparing AI-generated outputs against these human-written benchmarks, the researchers could assess the models' ability to maintain factual integrity while reducing complexity.
The Challenge of Factual Integrity
The study revealed a sobering reality regarding generative models: "aggressive" simplification often leads to the loss of critical nuance or the introduction of outright errors.
- Nuance Loss: In one instance, a model simplified a passage about historical voting rights by removing the specific detail that only men were allowed to vote, thereby generalizing the fact and losing historical accuracy.
- Hallucinations: The models frequently exhibited "temporal drift," such as incorrectly changing the date of a historical congress from 1915 to 2015.
- Inference Errors: Due to a lack of deep "world knowledge" regarding Dutch culture and history compared to English-centric models, the AI often merged distinct biographical facts into incorrect "alternative truths."
Evaluation and Results
Both quantitative and qualitative evaluations suggest that fully automated Dutch simplification is not yet viable for sensitive real-world applications. Quantitative BLEU scores were remarkably low, with Dutch_T5 peaking at only 0.138 on a 0-to-1 scale.
Qualitative human evaluations further highlighted these deficiencies. On a 5-point scale, both models received poor marks for simplicity, fluency, and adequacy.
As shown in the results, while Dutch_T5 slightly outperformed the translation-pivot method (GoogleTransl), neither model reached a level of quality sufficient for use without significant human intervention.
Conclusion and Future Directions
The ARTIST project highlights that while AI can reduce linguistic complexity, it cannot yet be trusted to maintain factual consistency in low-resource languages like Dutch. The researchers propose that future work should move toward neuro-symbolic AI—combining generative power with hard factual constraints—and human-in-the-loop pipelines. In such systems, AI would provide an initial draft that human experts refine to ensure the final text is both accessible and accurate.
Summary updated