FRACCO: A gold-standard annotated corpus of oncological entities with ICD-O-3.1 normalisation

Johann Pignat, Milena Vucetic, Christophe Gaudet-Blavignac, Jamil Zaghir, Amandine Stettler, Fanny Amrein, Jonatan Bonjour, Jean-Philippe Goldman, Olivier Michielin, Christian Lovis, Mina BjelogrlicarXiv 2025 · DOI · arXiv

TL;DR

The authors present FRACCO, the first gold-standard, expert-annotated French corpus specifically designed for clinical oncology. Comprising 1,301 clinical cases and over 70,000 annotations, it normalizes complex medical concepts using the ICD-O-3.1 standard and introduces a novel composite layer to capture multi-attribute clinical expressions.

What Problem Does This Paper Address?

Building effective Natural Language Processing (NLP) tools for the medical domain requires massive amounts of high-quality, annotated data. While English-language clinical datasets are relatively abundant, French clinical resources remain scarce—especially in highly specialized fields like oncology.

Furthermore, existing medical Named Entity Recognition (NER) datasets often focus on isolated, atomic terms (e.g., just tagging a body part or a disease name). In real clinical narratives, however, diagnoses are composite and multi-attribute (e.g., "poorly differentiated adenocarcinoma of the lung"). The lack of a standardized French corpus that captures these complex, interconnected clinical concepts severely limits the development of robust oncology-specific NLP tools.

Methods

The creation of FRACCO relied on a hybrid pipeline combining machine translation, automated annotation projection, and extensive manual refinement by medical experts:

  1. Translation & Baseline Projection: The authors started with the Spanish CANTEMIST corpus (synthetic oncological case reports) and translated it into French using DeepL Pro as part of the FRASIMED initiative. They automatically projected the original Spanish morphology annotations onto the French text using character-level string matching.
  2. Manual Re-annotation & Expansion: Domain experts manually reviewed the entire corpus to fix translation errors and alignment shifts. Crucially, they annotated two entirely new layers de novo: topography (anatomical site) and histology differentiation (tumour grading).
  3. Composite Expression Layer (expression_CIM): To capture full clinical concepts, the authors introduced a high-level composite label that unifies morphology, topography, and differentiation into a single expression.
  4. ICD-O Normalization: Annotations were mapped to standardized International Classification of Diseases for Oncology (ICD-O) codes using a semi-automated workflow of dictionary matching followed by expert validation.
  5. Technical Validation: The dataset's quality was validated by measuring inter-annotator agreement (IAA) and establishing baseline performance benchmarks by fine-tuning five different BERT-based language models (including CamemBERT and multilingual BERT).

Datasets

  • FRACCO (FRench Annotated Corpus for Clinical Oncology): The newly introduced dataset containing 1,301 synthetic French clinical cases and 71,065 total entity annotations.
  • It maps to 399 unique morphology codes, 272 topography codes, and 2,043 unique composite expressions.
  • It is derived from and structurally aligned with the Spanish CANTEMIST corpus.

Key Insights and Ideas

  • The Value of Composite Labeling: By introducing the expression_CIM label, the dataset bridges the gap between simple entity recognition and true semantic understanding. It models how clinicians actually write, linking descriptive attributes directly to the core diagnosis.
  • General vs. Domain-Specific Models: Surprisingly, domain-specific models like CamemBERT-bio did not significantly outperform general-domain models like CamemBERT-base (both achieved an identical 89.4% weighted-F1). The authors hypothesize that because oncological terms are highly repetitive and frequently appear in general corpora (like Wikipedia or web crawls), additional biomedical pre-training provides less marginal benefit than it might in rarer medical sub-fields.
  • The Challenge of Span Boundaries: During validation, the lowest inter-annotator agreement occurred in the "differentiation" category (a hard F1 of 0.48). Error analysis revealed this wasn't due to clinical disagreement, but rather stylistic differences in text span selection (e.g., one annotator highlighting "de grade 2" while another highlighted only "grade 2").

Key Results

  • Massive Data Expansion: Through manual enrichment, the total number of annotations grew from roughly 16,000 in the baseline translated corpus to over 70,000.
  • Annotation Distribution: The dataset is well-balanced between core morphology and composite expressions, though highly imbalanced toward differentiation grades, which make up only 1.5% of the data.
Figure 1: label distribution, n = 71’065
  • High Model Learnability: Despite the class imbalance, baseline NER models proved highly capable of learning the annotation schema. Top-performing models achieved a 89.4% weighted-F1 score on the test set.

The table below illustrates how individual atomic entities are successfully synthesized into a unified composite code:

Example of how morphology, topography, and differentiation are combined into a composite expression_CIM label.

Limitations and Future Work

  • Ontology Constraints: The authors noted that the ICD-O framework occasionally lacks precise codes for highly complex composite expressions, leading to multiple potential mappings or forcing a reliance on consensus interpretations for edge cases.
  • Synthetic Source Text: Because the underlying text consists of synthetic clinical cases translated from Spanish, it may not fully capture the idiosyncratic shorthand, typos, and stylistic variations found in native, real-world French Electronic Health Records (EHRs).
  • Future Directions: A natural next step will be testing the models trained on FRACCO against unstructured, native French hospital data to evaluate real-world clinical transferability.

Summary updated

Abstract

Developing natural language processing tools for clinical text requires annotated datasets, yet French oncology resources remain scarce. We present FRACCO (FRench Annotated Corpus for Clinical Oncology) an expert-annotated corpus of 1301 synthetic French clinical cases, initially translated from the Spanish CANTEMIST corpus as part of the FRASIMED initiative. Each document is annotated with terms related to morphology, topography, and histologic differentiation, using the International Classification of Diseases for Oncology (ICD-O) as reference. An additional annotation layer captures composite expression-level normalisations that combine multiple ICD-O elements into unified clinical concepts. Annotation quality was ensured through expert review: 1301 texts were manually annotated for entity spans by two domain experts. A total of 71127 ICD-O normalisations were produced through a combination of automated matching and manual validation by a team of five annotators. The final dataset representing 399 unique morphology codes (from 2549 different expressions), 272 topography codes (from 3143 different expressions), and 2043 unique composite expressions (from 11144 different expressions). This dataset provides a reference standard for named entity recognition and concept normalisation in French oncology texts.
BibTeX
@misc{pignat2025fracco,
  author = {Johann Pignat and Milena Vucetic and Christophe Gaudet-Blavignac and Jamil Zaghir and Amandine Stettler and Fanny Amrein and Jonatan Bonjour and Jean-Philippe Goldman and Olivier Michielin and Christian Lovis and Mina Bjelogrlic},
  title = {{FRACCO: A gold-standard annotated corpus of oncological entities with ICD-O-3.1 normalisation}},
  year = {2025},
  eprint = {2510.13873},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.13873},
}
Figure