Population Characteristics in Justice Health Research Based on PubMed Abstracts From 1963 to 2023: Text Mining Study (Preprint)

Wilson Lukmanjaya, Tony Butler, Patricia Taflan, Paul Simpson, Natasha Ginnivan, Iain Buchan, Goran Nenadić, George Karystianis2024 · DOI

Introduction

Justice health research, or epidemiological criminology, sits at the critical intersection of public health and the legal system. To develop effective interventions for justice-involved populations, it is essential to understand who is being studied. However, the sheer volume of literature makes manual review nearly impossible, and inconsistent reporting standards often obscure whether research is equitable or representative. This study addresses this "knowledge gap" by automating the extraction of population characteristics from over 34,000 PubMed abstracts published between 1963 and 2023.

Methods

The researchers employed a rule-based text mining approach using the General Architecture for Text Engineering (GATE) and its Java Annotations Pattern Engine (JAPE). Unlike black-box machine learning models, this method used 140 specific rules to identify lexical patterns—anchored by specific verbs, nouns, and prepositions—to extract data on sex, age, offender type, and nationality.

Flowchart of the study methodology for identifying and screening PubMed abstracts.

The primary corpus consisted of 34,481 PubMed abstracts. To ensure accuracy, raw extractions were processed via Python scripts to map varied terms (e.g., "adolescent," "teen," "juvenile") into standardized bins. The system was evaluated against a manually annotated "gold standard" of 100 random abstracts, achieving a micro-F1 score of 89.9%, with high precision (94.3%) and respectable recall (85.9%).

Results

The analysis revealed a significant reporting crisis: 49.4% of abstracts mentioned none of the four key characteristics, and only 1.3% reported all four. This suggests that the "front door" of most papers—the abstract—often fails to inform readers about the study's target population.

Shifting Demographics

A major finding was the "Gender Paradox." While men represent roughly 95% of the global incarcerated population, research focus has shifted dramatically. Since 1990, female-only studies have surged, accounting for nearly half (48.6%) of research outputs in 2023. While studying women is vital, the authors suggest that the overwhelming majority of prisoners (men) may now be under-researched.

Regarding age, 69.3% of abstracts reporting age focused on minors (<18), reflecting a global research priority on youth justice. In terms of offender types, mentally ill offenders were the most studied subgroup (17.4%), reflecting a 30-year trend where prisons have increasingly become de facto mental health facilities.

Geographic Disparities

The United States leads in sheer volume (31.4% of articles), but when research output is scaled per 1,000 prisoners, Nordic countries (Sweden, Norway, Finland, and Denmark) produce the most research. This likely reflects their progressive, data-driven approach to offender rehabilitation.

Discussion and Conclusion

While the study is limited by its focus on abstracts rather than full-text bodies (due to paywalls) and its restriction to English-language articles, the findings are clear. There is a critical need for better reporting. The authors call for journals to enforce stricter adherence to reporting guidelines (like STROBE or CONSORT) specifically within abstracts to ensure justice health research remains transparent and actionable. By promoting diversity in participant selection and improving reporting clarity, the field can better address the needs of all justice-involved individuals.

Summary updated

Abstract

<sec> <title>BACKGROUND</title> The field of epidemiological criminology (or justice health research) has emerged in the past decade, studying the intersection between the public health and justice systems. To ensure research efforts are focused and equitable, it is important to reflect on the outputs in this area and address knowledge gaps. </sec> <sec> <title>OBJECTIVE</title> This study aimed to examine the characteristics of populations researched in a large sample of published outputs and identify research gaps and biases. </sec> <sec> <title>METHODS</title> A rule-based, text mining method was applied to 34,481 PubMed abstracts published from 1963 to 2023 to identify 4 population characteristics (sex, age, offender type, and nationality). </sec> <sec> <title>RESULTS</title> We evaluated our method in a random sample of 100 PubMed abstracts. Microprecision was 94.3%, with microrecall at 85.9% and micro–&lt;i&gt;F&lt;/i&gt;&lt;sub&gt;1&lt;/sub&gt;-score at 89.9% across the 4 characteristics. Half (n=17,039, 49.4%) of the 34,481 abstracts did not have any characteristic mentions and only 1.3% (n=443) reported sex, age, offender type, and nationality. From the 5170 (14.9%) abstracts that reported age, 3581 (69.3%) mentioned young people (younger than 18 years) and 3037 (58.7%) mentioned adults. Since 1990, studies reporting female-only populations increased, and in 2023, these accounted for almost half (105/216, 48.6%) of the research outputs, as opposed to 33.3% (72/216) for male-only populations. Nordic countries (Sweden, Norway, Finland, and Denmark) had the highest number of abstracts proportional to their incarcerated populations. Offenders with mental illness were the most common group of interest (840/4814, 17.4%), with an increase from 1990 onward. </sec> <sec> <title>CONCLUSIONS</title> Research reporting on female populations increased, surpassing that involving male individuals, despite female individuals representing 5% of the incarcerated population; this suggests that male prisoners are underresearched. Although calls have been made for the justice health area to focus more on young people, our results showed that among the abstracts reporting age, most mentioned a population aged &amp;lt;18 years, reflecting a rise of youth involvement in the youth justice system. Those convicted of sex offenses and crimes relating to children were not as researched as the existing literature suggests, with a focus instead on populations with mental illness, whose rates rose steadily in the last 30 years. After adjusting for the size of the incarcerated population, Nordic countries have conducted proportionately the most research. Our findings highlight that despite the presence of several research reporting guidelines, justice health abstracts still do not adequately describe the investigated populations. Our study offers new insights in the field of justice health with implications for promoting diversity in the selection of research participants. </sec>
BibTeX
@misc{lukmanjaya2024population,
  author = {Wilson Lukmanjaya and Tony Butler and Patricia Taflan and Paul Simpson and Natasha Ginnivan and Iain Buchan and Goran Nenadić and George Karystianis},
  title = {{Population Characteristics in Justice Health Research Based on PubMed Abstracts From 1963 to 2023: Text Mining Study (Preprint)}},
  year = {2024},
  doi = {10.2196/preprints.60878},
}
Figure