Automated multi-scale computational pathotyping (AMSCP) of inflamed synovial tissue

Richard D. Bell, Matthew Brendel, Maxwell A. Konnaris, Justin Xiang, Miguel Otero, Mark Alan Fontana, Zilong Bai, Jennifer S. Albrecht, William Apruzzese, Brendan F. Boyce, David L. Boyle, Michael B. Brenner, S. Louis Bridges, Christopher D. Buckley, Jane H. Buckner, Vivian P. Bykerk, James P. Dolan, Thomas Eisenhaure, Andrew Filer, Gary S. Firestein, Chamith Y. Fonseka, Ellen M. Gravallese, Peter K. Gregersen, Joel M. Guthridge, María Gutiérrez‐Arcelus, Nir Hacohen, V. Michael Holers, Laura B. Hughes, Eddie A. James, Judith A. James, A. Helena Jonsson, Josh Keegan, Stephen Kelly, James A. Lederer, Yvonne Lee, David Lieb, Arthur M. Mandelin, Mandy J. McGeachy, Michael A. McNamara, Joseph Mears, Fumitaka Mizoguchi, Larry W. Moreland, Jennifer P. Nguyen, Akiko Noma, Chad Nusbaum, Harris Perlman, Christopher T. Ritchlin, William H. Robinson, Mina Rohani‐Pichavant, Cristina Rozo, Karen Salomon-Escoto, Jennifer Seifert, Anupamaa Seshadri, Kamil Slowikowski, Danielle Sutherby, Darren Tabechian, Jason D. Turner, Paul J. Utz, Gerald F. Watts, Kevin Wei, Costantino Pitzalis, Deepak A. Rao, Soumya Raychaudhuri, Daria M. Krenitsky, Nida Meednu, Javier Rangel‐Moreno, Dagmar Scheel‐Toellner, Hayley Carr, Saba Nayar, Jack L. McMurray, Edward F. DiCarlo, Jennifer H. Anolik, Laura T. Donlin, Dana E. Orange, H. Mark Kenney, Edward M. Schwarz, Lionel B. Ivashkiv, Fei WangNature Communications 2024 · DOI

Abstract: Rheumatoid arthritis (RA) is a complex immune-mediated inflammatory disorder in which patients suffer from inflammatory-erosive arthritis. Recent advances on histopathology heterogeneity of RA synovial tissue revealed three distinct phenotypes based on cellular composition (pauci-immune, diffuse and lymphoid), suggesting that distinct etiologies warrant specific targeted therapy which motivates a need for cost effective phenotyping tools in preclinical and clinical settings. To this end, we developed an automated multi-scale computational pathotyping (AMSCP) pipeline for both human and mouse synovial tissue with two distinct components that can be leveraged together or independently: (1) segmentation of different tissue types to characterize tissue-level changes, and (2) cell type classification within each tissue compartment that assesses change across disease states. Here, we demonstrate the efficacy, efficiency, and robustness of the AMSCP pipeline as well as the ability to discover novel phenotypes. Taken together, we find AMSCP to be a valuable cost-effective method for both pre-clinical and clinical research.

Introduction

Rheumatoid Arthritis (RA) is a highly heterogeneous disease, typically categorized into three distinct "pathotypes" based on synovial tissue composition: lymphoid (immune cell-rich), myeloid/diffuse, and pauci-immune (fibroblast-rich). These subtypes are critical because they respond differently to targeted therapies. Traditionally, identifying these pathotypes requires expert pathologists to manually score tissue slides—a process that is slow, expensive, and prone to inter-observer variation. There is a critical need for objective, scalable tools that can automate this "pathotyping" in both preclinical research and clinical trials to accelerate drug discovery and precision medicine.

Methods and Pipeline Architecture

The Automated Multi-Scale Computational Pathotyping (AMSCP) pipeline consists of two primary stages designed to bridge the gap between structural tissue changes and cellular-level inflammation.

  1. Tissue Segmentation: The pipeline utilizes UNet++, a deep learning architecture, to partition whole-slide images into distinct regions such as synovium, cartilage, and bone. To overcome "batch effects"—variations in staining color and intensity common in histology—the authors employed a 66% patch overlap and heavy data augmentation.
  2. Cell Classification: Individual cell nuclei are detected and segmented using HoverNet. For each cell, 856 features are extracted, including critical "neighborhood features" that capture spatial context, such as the density of surrounding cells. These features are then processed by a Gradient Boosted Decision Tree (XGBoost) to classify cells into types like fibroblasts, lymphocytes, and lining cells.
  3. Active Learning: To reduce the manual annotation burden for human samples, an active learning loop was implemented. The model identifies cells it is most "uncertain" about for pathologist labeling, significantly streamlining the training process.
AMSCP segmentation performance and visual comparison across different models.

Datasets and Validation

The pipeline was validated across diverse datasets to ensure robustness:

  • Mouse Models: TNF-Tg (transgenic) and Zymosan-induced arthritis models were used to simulate chronic and acute inflammation.
  • Human Biopsies: 58 synovial tissue biopsies from the AMP-RA consortium provided a diverse clinical cohort for testing.
  • Validation: Computational results were compared against historical hand-drawn histomorphometry and expert-derived Krenn inflammation scores.

Key Insights

  • Spatial Context is Essential: The inclusion of neighborhood features was a breakthrough for cell classification, improving the F1 score from 0.75 to 0.88. This demonstrates that a cell's identity is often defined by its local environment.
  • Cross-Species Utility: The pipeline's ability to work on both mouse and human tissue demonstrates its potential to bridge the gap between laboratory research and clinical application.
  • Biomarker Discovery: In human samples, the researchers identified that a threshold of 0.82% plasma cells could accurately distinguish between "diffuse" and "lymphoid" pathotypes (AUROC = 0.82), offering a potential objective biomarker for patient stratification.

Results

The AMSCP pipeline demonstrated high technical accuracy and biological fidelity. The segmentation model achieved a mean Intersection over Union (mIOU) of up to 0.95. Biologically, the model successfully recapitulated known phenomena, such as the reduction of synovitis following anti-TNF therapy and sexual dimorphism in disease severity in TNF-Tg mice.

Efficiency gains were particularly striking: the automated pipeline processed 174 slides in roughly 30 hours of compute time with only 2–3 hours of human labor. This represents a 120-fold increase in efficiency compared to the estimated 200+ hours required for manual annotation. Furthermore, computational lymphocyte counts showed a strong correlation (ρ = 0.88) with the gold-standard Krenn inflammation score.

Biological validation of the AMSCP pipeline in a mouse model, showing correlation with manual methods and sensitivity to treatment response.

Limitations and Future Work

While highly effective, the model faced challenges in consistently identifying macrophages due to their high heterogeneity and the limited specificity of H&E staining. Future iterations may integrate spatial transcriptomics or multiplexed imaging to provide deeper molecular insights. Additionally, further validation on larger, independent clinical cohorts is necessary before the pipeline can be adopted for routine diagnostic use.

Summary updated

Abstract

Rheumatoid arthritis (RA) is a complex immune-mediated inflammatory disorder in which patients suffer from inflammatory-erosive arthritis. Recent advances on histopathology heterogeneity of RA synovial tissue revealed three distinct phenotypes based on cellular composition (pauci-immune, diffuse and lymphoid), suggesting that distinct etiologies warrant specific targeted therapy which motivates a need for cost effective phenotyping tools in preclinical and clinical settings. To this end, we developed an automated multi-scale computational pathotyping (AMSCP) pipeline for both human and mouse synovial tissue with two distinct components that can be leveraged together or independently: (1) segmentation of different tissue types to characterize tissue-level changes, and (2) cell type classification within each tissue compartment that assesses change across disease states. Here, we demonstrate the efficacy, efficiency, and robustness of the AMSCP pipeline as well as the ability to discover novel phenotypes. Taken together, we find AMSCP to be a valuable cost-effective method for both pre-clinical and clinical research.
BibTeX
@inproceedings{bell2024automated,
  author = {Richard D. Bell and Matthew Brendel and Maxwell A. Konnaris and Justin Xiang and Miguel Otero and Mark Alan Fontana and Zilong Bai and Jennifer S. Albrecht and William Apruzzese and Brendan F. Boyce and David L. Boyle and Michael B. Brenner and S. Louis Bridges and Christopher D. Buckley and Jane H. Buckner and Vivian P. Bykerk and James P. Dolan and Thomas Eisenhaure and Andrew Filer and Gary S. Firestein and Chamith Y. Fonseka and Ellen M. Gravallese and Peter K. Gregersen and Joel M. Guthridge and María Gutiérrez‐Arcelus and Nir Hacohen and V. Michael Holers and Laura B. Hughes and Eddie A. James and Judith A. James and A. Helena Jonsson and Josh Keegan and Stephen Kelly and James A. Lederer and Yvonne Lee and David Lieb and Arthur M. Mandelin and Mandy J. McGeachy and Michael A. McNamara and Joseph Mears and Fumitaka Mizoguchi and Larry W. Moreland and Jennifer P. Nguyen and Akiko Noma and Chad Nusbaum and Harris Perlman and Christopher T. Ritchlin and William H. Robinson and Mina Rohani‐Pichavant and Cristina Rozo and Karen Salomon-Escoto and Jennifer Seifert and Anupamaa Seshadri and Kamil Slowikowski and Danielle Sutherby and Darren Tabechian and Jason D. Turner and Paul J. Utz and Gerald F. Watts and Kevin Wei and Costantino Pitzalis and Deepak A. Rao and Soumya Raychaudhuri and Daria M. Krenitsky and Nida Meednu and Javier Rangel‐Moreno and Dagmar Scheel‐Toellner and Hayley Carr and Saba Nayar and Jack L. McMurray and Edward F. DiCarlo and Jennifer H. Anolik and Laura T. Donlin and Dana E. Orange and H. Mark Kenney and Edward M. Schwarz and Andrew Filer and Lionel B. Ivashkiv and Fei Wang},
  title = {{Automated multi-scale computational pathotyping (AMSCP) of inflamed synovial tissue}},
  booktitle = {Nature Communications},
  year = {2024},
  doi = {10.1038/s41467-024-51012-6},
}
Figure