A Modular Framework for ESG Prediction through Domain-Specific Language Modeling and Graph Learning

Sofia Baiocco, Gianluca Bonifazi, Christopher Buratti, Michele Marchetti, Camilla Mazzoli, Federica Parlapiano, Giulia Quaglieri, Davide Traini, Domenico Ursino, Luca VirgiliarXiv 2026 · arXiv

TL;DR

ESGPred is a new modular framework that predicts corporate Environmental, Social, and Governance (ESG) scores by combining domain-specific language modeling with Graph Neural Networks (GNNs). By modeling companies not as isolated entities but as nodes in a "sustainability graph," the system achieves significantly higher accuracy than traditional text-only approaches, reaching up to 85% accuracy in predicting ESG rating quartiles.

What Problem Does This Paper Address?

ESG scores have become critical for investment decisions and regulatory compliance, yet the industry faces a major "consistency" problem. Different rating agencies (like MSCI or S&P Global) often provide wildly different scores for the same company because they rely on proprietary, subjective, and manual assessment methodologies.

The challenge in automating this process lies in the data: sustainability reports are lengthy, qualitative, and filled with specialized financial jargon. While standard NLP models can extract some signal, they often ignore the relational context—such as industry peers, shared business models, or similar sustainability narratives—that helps define a company's true ESG standing.

Methods

The authors propose ESGPred, a four-step modular pipeline designed to turn unstructured reports into precise ESG classifications:

  1. Supervised Fine-Tuning: They start with FinBERT, a language model pre-trained on financial corpora, and fine-tune it specifically on ESG-related labels using a Weighted Focal Loss to handle class imbalances.
  2. Embedding Generation: Since reports are too long for standard Transformers, ESGPred segments them into overlapping chunks. These chunks are embedded via the fine-tuned FinBERT and then aggregated using mean pooling to create a single "document-level" vector.
  3. Graph Construction: This is the core innovation. The framework builds "sustainability graphs" where nodes are reports and edges represent relationships. They tested three strategies:
  4. Semantic Similarity: Edges based on the cosine similarity of report embeddings.
  5. Company Description Similarity: Using NLP (SpaCy) to extract entities and keywords from company profiles, then linking them via Jaccard similarity.
  6. Sector-Based: Linking all companies within the same industrial sector.
  7. Graph-Based Prediction: Finally, a GNN (like a Graph Transformer or GraphSAGE) processes the graph. This allows the model to "smooth" a company's prediction based on the features of its neighbors in the graph.
Figure 1: Overview of the ESGPred’s workflow

Datasets

The study utilized a substantial real-world dataset:

  • Sustainability Reports: 5,266 PDF reports from 2,134 publicly listed companies, covering the years 2020–2023.
  • Metadata: Sector classifications, country of origin, and descriptive profiles sourced from ResponsibilityReports.
  • Ground Truth: ESG ratings from LSEG Data & Analytics (formerly Refinitiv), mapped into four quartiles (A, B, C, D).

Key Insights and Ideas

  • Relational Context Matters: The biggest "aha" moment is that a company’s ESG performance is better understood by looking at its peers. Adding graph-based relational learning improved accuracy from 77% (text-only) to 85% (graph-based).
  • Descriptions over Sectors: Interestingly, graphs built from company descriptions (capturing specific products, technologies, and markets) performed better than simple sector-based graphs. This suggests that "industry" is too coarse a filter, while detailed business descriptions capture more nuanced ESG commonalities.
  • Graph Transformers Win: Among various GNN architectures, the Graph Transformer was the most effective. Its multi-head attention mechanism allows it to weigh different types of neighbor relationships dynamically, which is crucial given the heterogeneous nature of corporate data.
  • Generalizability: The framework isn't just for "overall" scores; it showed strong performance when predicting individual Environmental, Social, and Governance components separately, with the Environmental score being the easiest to predict (87% accuracy).

Key Results

The researchers compared ESGPred against several baselines, including Vanilla FinBERT and a fine-tuned Transformer-only model:

  • Accuracy Boost: The best ESGPred configuration (Description-Based Graph + Graph Transformer) achieved 85% accuracy and an 0.84 F1-score.
  • Baseline Comparison: This significantly outperformed the Fine-Tuned FinBERT baseline (77% accuracy), proving that the graph structure adds substantial predictive value beyond the text itself.
  • Component Performance:
    • Environmental: 87% Accuracy
    • Social: 85% Accuracy
    • Governance: 79% Accuracy (noted as the most difficult to predict due to its less descriptive, more structural nature).

Limitations and Future Work

While promising, the authors acknowledge several constraints:

  • Single Ground Truth: The model currently learns to replicate LSEG’s specific methodology. Since agencies diverge, the model might inherit one agency's specific biases.
  • Static Graphs: The current model uses static snapshots. However, corporate relationships and ESG performance are dynamic and evolve over time.
  • Limited Features: The model primarily uses text. Future versions could integrate structured financial indicators, supply chain data, or ownership networks.
  • Explainability: While the model predicts well, identifying which specific relationships or report passages contributed most to a score remains a goal for future "Graph Explainability" research.

Summary updated

Abstract

Environmental, Social and Governance (ESG) scores are increasingly used to evaluate corporate sustainability, supporting investment decisions, ensuring regulatory compliance and assessing risk. However, current ESG evaluations largely rely on proprietary methodologies that often produce inconsistent ratings. This motivates the development of data-driven prediction frameworks. In this paper, we present ESGPred, a modular framework for predicting ESG scores by combining domain-specific language modeling with graph-based learning over a network that models the relationships among the companies being evaluated. In addition to presenting the rationale behind ESGPred and detailing its ESG score prediction process, we show that ESGPred can effectively predict both overall ESG ratings and their individual Environmental, Social, and Governance components. We evaluated ESGPred using a dataset of 5,266 sustainability reports published by 2,134 publicly listed companies between 2020 and 2023. In this evaluation, we used ESG ratings provided by LSEG Data & Analytics as ground truth. Our experimental results demonstrated that integrating graph-based relational learning improves prediction performance over purely text-based baselines.
BibTeX
@misc{baiocco2026modular,
  author = {Sofia Baiocco and Gianluca Bonifazi and Christopher Buratti and Michele Marchetti and Camilla Mazzoli and Federica Parlapiano and Giulia Quaglieri and Davide Traini and Domenico Ursino and Luca Virgili},
  title = {{A Modular Framework for ESG Prediction through Domain-Specific Language Modeling and Graph Learning}},
  year = {2026},
  eprint = {2611.16831},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16831},
}
Figure