Explainability and transparency in the realm of digital humanities: toward a historian XAI

Hassan El-Hajj, Oliver Eberle, Anika Merklein, Anna Siebold, Noga Shlomi, Jochen Büttner, Julius Martinetz, Klaus‐Robert Müller, Grégoire Montavon, Matteo VallerianiInternational Journal of Digital Humanities 2023 · DOI

Abstract

The recent advancements in the field of Artificial Intelligence (AI) translated to an increased adoption of AI technology in the humanities, which is often challenged by the limited amount of annotated data, as well as its heterogeneity. Despite the scarcity of data it has become common practice to design increasingly complex AI models, usually at the expense of human readability, explainability, and trust. This in turn has led to an increased need for tools to help humanities scholars better explain and validate their models as well as their hypotheses. In this paper, we discuss the importance of employing Explainable AI (XAI) methods within the humanities to gain insights into historical processes as well as ensure model reproducibility and a trustworthy scientific result. To drive our point, we present several representative case studies from the Sphaera project where we analyze a large, well-curated corpus of early modern textbooks using an AI model, and rely on the XAI explanatory outputs to generate historical insights concerning their visual content. More specifically, we show that XAI can be used as a partner when investigating debated subjects in the history of science, such as what strategies were used in the early modern period to showcase mathematical instruments and machines.

The Black Box Problem in Digital Humanities

In the Digital Humanities (DH), Artificial Intelligence is frequently deployed as a "black box" for automated tasks such as Optical Character Recognition (OCR) or document layout analysis. While these models often achieve high performance, their internal logic remains opaque. For a historian, knowing that a model classified an image as a "mathematical instrument" is of limited utility if the underlying reasoning is hidden.

This opacity creates a significant trust gap. If a scholar cannot scrutinize the logic behind a machine’s decision, they cannot integrate that decision into a rigorous historical argument. This research advocates for a shift from "operational" AI—using machines merely to perform labor—to "interpretative" AI, where machines serve as partners in generating and testing historical hypotheses.

Methodology: Opening the Box with LRP

To bridge this gap, the researchers employed a two-step technical framework designed to provide granular transparency. First, a VGG-16 convolutional neural network (CNN), pre-trained on ImageNet, was fine-tuned on a specific corpus of historical illustrations to perform classification.

Second, the team applied Layer-wise Relevance Propagation (LRP) to explain the model's outputs. Unlike standard saliency maps (such as Grad-CAM) which often produce blurry regions of interest, LRP traces the model's decision back to individual pixels. This allows researchers to identify the specific lines, textures, or compositional elements in a 16th-century woodcut that triggered a classification.

The XAI workflow: from domain expert input to model classification and pixel-level explanation.

The Sphaera Dataset

The study utilized the Sphaera dataset, a curated collection of over 350 early modern textbooks based on Johannes de Sacrobosco’s Tractatus de Sphaera. This corpus represents a critical period in the history of science and education.

The dataset consists of 5,879 pages categorized into four primary classes:

  • Mathematical Instruments (657): Depictions of quadrants, armillary spheres, and other measuring tools.
  • Scientific Illustrations (1,870): Abstract diagrams representing orbits, eclipses, or celestial mechanics.
  • Machines (352): Complex mechanical devices such as mills and pumps, often sourced from contemporary "Theaters of Machines."
  • Other (3,000): Contrast classes including text-only pages.
Representative samples from the Sphaera dataset: (a) machines, (b) scientific diagrams, (c) text pages, and (d) mathematical instruments.

Discovering the Visual Language of Early Modern Science

The application of XAI revealed "visual conventions" that defined how 16th and 17th-century illustrators communicated technical information. By analyzing the LRP heatmaps, historians identified several key insights:

  • The Role of Scales: The model learned that the defining feature of a "mathematical instrument" was the graduated scale. Even in abstract diagrams, the presence of a scale for measurement prompted the model to classify the object as an instrument, mirroring the priorities of historical actors who valued precision.
  • Contextual Cues for Machines: Interestingly, the model did not rely solely on gears or cogs to identify machines. Instead, it focused on scaffolding, poles, and architectural context (such as floor tiles or windows). This suggests that early modern illustrators used these environmental elements to provide a sense of physical scale, helping readers grasp the actual size of large mechanical devices.
  • Materiality and Shared Conventions: A lathe (a machine) was frequently misclassified as an instrument. LRP analysis showed the model was triggered by the footstand. This revealed a shared visual convention of "materiality"—both instruments and machines were depicted with sturdy bases to signal their status as real, physical objects rather than abstract geometric concepts.
  • Negative Signals: For machines, the presence of multiple concentric circles—ubiquitous in astronomical diagrams—acted as a "negative" contribution, helping the model distinguish between a mechanical gear and a celestial orbit.

Conclusion

The fine-tuned VGG-16 achieved a high test set accuracy of 0.96, but the primary value of the research is qualitative. XAI allowed historians to move beyond dictionary definitions toward a "visual definition" based on actual print practices. By acting as a "derivation of the eye," the model forced researchers to notice minute details they might otherwise have overlooked. While currently constrained by the specific corpus of geocentric astronomy, this workflow demonstrates that XAI can serve as a powerful "research companion" for the humanities, ensuring that AI-driven insights are both interpretable and scientifically robust.

Summary updated

Abstract

Abstract The recent advancements in the field of Artificial Intelligence (AI) translated to an increased adoption of AI technology in the humanities, which is often challenged by the limited amount of annotated data, as well as its heterogeneity. Despite the scarcity of data it has become common practice to design increasingly complex AI models, usually at the expense of human readability, explainability, and trust. This in turn has led to an increased need for tools to help humanities scholars better explain and validate their models as well as their hypotheses. In this paper, we discuss the importance of employing Explainable AI (XAI) methods within the humanities to gain insights into historical processes as well as ensure model reproducibility and a trustworthy scientific result. To drive our point, we present several representative case studies from the Sphaera project where we analyze a large, well-curated corpus of early modern textbooks using an AI model, and rely on the XAI explanatory outputs to generate historical insights concerning their visual content. More specifically, we show that XAI can be used as a partner when investigating debated subjects in the history of science, such as what strategies were used in the early modern period to showcase mathematical instruments and machines.
BibTeX
@article{elhajj2023explainability,
  author = {Hassan El-Hajj and Oliver Eberle and Anika Merklein and Anna Siebold and Noga Shlomi and Jochen Büttner and Julius Martinetz and Klaus‐Robert Müller and Grégoire Montavon and Matteo Valleriani},
  title = {{Explainability and transparency in the realm of digital humanities: toward a historian XAI}},
  journal = {International Journal of Digital Humanities},
  year = {2023},
  doi = {10.1007/s42803-023-00070-1},
}
Figure