TL;DR
This paper tackles the challenge of making public administration language accessible to citizens by developing a specialized corpus of simplified texts from ombudsman institutions. The research introduces the OmbudsCorpus, a parallel dataset of complex-simple text pairs, and a methodology for annotating simplification operations, aiming to advance Automatic Text Simplification (ATS) systems for administrative texts.
What Problem Does This Paper Address?
Public administration language, particularly legal and administrative texts, is often characterized by syntactic complexity, jargon, and archaic phrasing, making it difficult for ordinary citizens to understand. This lack of comprehension can hinder citizens' ability to engage with public services and exercise their rights. While Automatic Text Simplification (ATS) offers a potential solution, its effectiveness in specialized domains like public administration is limited by the lack of appropriate training data. Existing ATS datasets are often general-purpose and do not capture the unique linguistic features of administrative language. This paper addresses this gap by focusing on the language used by ombudsman institutions, which act as crucial intermediaries between citizens and public bodies, and aims to create a specialized resource for improving ATS in this domain.
Methods
The research employs a multi-faceted approach:
- Linguistic Analysis: The project first involves an extensive analysis of the syntactical, lexical, and discursive features of English-speaking ombudsmen's texts. This aims to identify specific characteristics that contribute to complexity and pose a threat to comprehension.
- Corpus Compilation: The OmbudsCorpus is being built from publicly available annual reports and case reports of ombudsman offices. To ensure representativeness and allow for variational studies, texts are selected from three common thematic areas (Education, Health, and Housing) across various English-speaking countries. The corpus is structured into subcorpora to manage the volume of data and ensure proportional representation.
- Parallel Corpus Creation: A key contribution is the creation of a parallel corpus consisting of original complex fragments and their manually simplified versions.
- Annotation of Simplification Operations: The simplified fragments are meticulously annotated with standardized simplification operations using XML tags. This detailed annotation allows for a deeper understanding of how simplification is achieved, moving beyond simple text transformations.
- Complexity Measurement: To identify the most complex texts for simplification, the researchers utilize psycholinguistic metrics such as word frequency, familiarity, concreteness, and imageability (using the TAALES tool), as well as syntactic complexity metrics like subordinating conjunctions per clause, passive auxiliary verbs per clause, and dependents per clause (using TAASSC). The Flesch-Kincaid grade level is also used for comparison with existing datasets.
Datasets
The paper introduces the OmbudsCorpus, a novel parallel corpus specifically designed for the simplification of public administrative language.
- OmbudsCorpus: This corpus is composed of original complex fragments from ombudsman case reports and their manually simplified counterparts. It is annotated with detailed simplification operations. The corpus is substantial, comprising approximately 12,600,000 tokens in total, with a significant portion from annual reports and a smaller but crucial part from case reports.
- Existing Datasets Mentioned: The paper references several existing datasets for context and comparison, including:
- Newsela and TurkCorpus (general English text simplification)
- ASSETann corpus (standardized annotation for simplification)
- Dsim corpus (Danish)
- Brazilian Portuguese corpus
- Japanese news simplification corpus
- Italian corpora (Terence, SIMPITIKI)
- German corpus
- French biomedical corpus (CLEAR)
- Spanish medical corpus (CLARA-MeD)
- Spanish domain-independent corpus (EASIER)
- SimPA (English, public administration domain)
- SIMPITIKI (Italian, public administration domain)
The OmbudsCorpus distinguishes itself by focusing on the specific domain of public administration and by providing detailed annotations of simplification operations at a supra-sentential level.
Key Insights and Ideas
- Complexity is a Continuum, Not a Binary: The paper emphasizes that text complexity is not absolute but exists on a continuum. Factors like lexicon familiarity and syntactic structure contribute to difficulty, but the perception of complexity is also highly dependent on the target audience's background and needs.
- Plain Language Recommendations are a Starting Point, Not an End Goal: While standard plain language recommendations (e.g., short sentences, simple words) are valuable, they are often too generic. True simplification requires a deeper understanding of psycholinguistic factors and syntactic structures that influence comprehension.
- Annotation of Simplification Operations is Crucial: Simply providing complex-simple pairs is insufficient. Annotating the specific operations (e.g., replacing a synonym, deleting a modifier, changing to active voice) is vital for training sophisticated ATS systems and understanding the simplification process itself.
- Supra-sentential Simplification is Necessary: Many existing datasets focus on sentence-level simplification. However, administrative texts often require simplification at a higher level, involving the restructuring of multiple sentences or paragraphs to improve coherence and flow.
- The "Why" Behind Simplification Matters: The research highlights that the goal is not just to make text shorter or use simpler words, but to achieve genuine comprehensibility for the target audience, which involves a nuanced understanding of linguistic features and their impact on readers.
Key Results
The paper presents preliminary results from the evaluation of the simplified versions of the OmbudsCorpus fragments:
- Lexical Improvement: Metrics for word frequency, familiarity, and concreteness showed statistically significant improvements in the simplified versions (p-value <0.05), indicating that complex words were replaced with more frequent and familiar alternatives.
- Syntactic Improvement: The average number of clausal components per clause (ccomp_per_cl) and passive verbs per clause (auxpass_per_cl) were significantly reduced in the simplified fragments, indicating a shift towards simpler sentence structures and more active voice.
- Flesch-Kincaid Grade Level Reduction: A significant reduction in the Flesch-Kincaid Grade Level was observed in the simplified texts, dropping from an average of 14.960 to 8.857 (p-value <0.05).
- Limitations in Lexicon: Despite improvements, the lexicon chosen for substitution still retained high levels of abstraction, and imageability showed only marginal, non-statistically significant improvement.
- Subordinate Conjunctions: While the number of subordinate clauses decreased, the difference in the number of subordinate conjunctions per clause (mark_per_cl) was not statistically significant.
Limitations and Future Work
- Subjectivity in Annotation: The authors acknowledge that human annotation, even by professionals, can be subjective. They propose a "perspectivist approach" to corpus annotation, viewing disagreements between annotators not as errors but as different, valid interpretations.
- Lexical Abstraction: The simplified lexicon, while more frequent and familiar, still exhibited a degree of abstraction, suggesting room for further improvement in lexical choice.
- Future Work:
- ATS System Training and Evaluation: The OmbudsCorpus is intended to be used for training and evaluating ATS systems specialized in public administrative language.
- Enrichment with Target Audience Ratings: The authors suggest that incorporating ratings from actual target audiences would further enhance the validation and assessment of the reference data.
- Diachronic and Diatopic Variation: The corpus is annotated to allow for the exploration of variations in language over time and across different regions.
- Further Refinement of Simplification: The analysis of specific fragment pairs aims to identify areas where further simplification might be needed to achieve optimal clarity.
Summary updated