Investigate The ESG Score Methodology

Zhi ChenarXiv 2023 · DOI · arXiv

Abstract

Whether the Refinitiv provide a reliable and trusted methodology in the process of aggregating 10 category scores to overall score?

Introduction: The "Black Box" of ESG Ratings

Environmental, Social, and Governance (ESG) ratings have become a cornerstone of modern investing, yet they are notoriously inconsistent. Different agencies often provide vastly different scores for the same company—a phenomenon known as "aggregate confusion." This lack of transparency creates a "black box" where the final rating's construction remains opaque to investors.

This study investigates the aggregation process used by Refinitiv (ASSET4), specifically examining whether turning 10 distinct category scores into one final ESG score introduces noise or loses critical information. By decoding the underlying weights assigned to different sectors, we can determine if these scores truly reflect a company's sustainability or if the aggregation process itself dilutes the predictive power of the raw data.

Methodology: Reverse-Engineering the Score

To peer into this black box, we employ a reverse-engineering approach using linear regression analysis. We treat the overall ESG score as the dependent variable and the 10 sub-category scores (e.g., Emissions, Human Rights, Management) as independent variables.

The hierarchical structure of Refinitiv's ESG scoring methodology, from 750+ data points to the final ESG score.

The analysis was performed across three dimensions:

  1. By Sector: To identify industry-specific weighting schemes.
  2. By Year: To track the evolution of the methodology over time (2002–2020).
  3. By Individual Company: To verify consistency within peer groups.

This approach allows us to identify the specific weights Refinitiv assigns to each category without access to proprietary internal formulas.

Key Results: A "Perfect Fit"

The most striking finding is the near-perfect linear relationship between category scores and the final ESG rating. Across various sectors and nearly two decades of data, the adjusted R-squared values for the regression models are consistently 1.0 (or extremely close to it).

Adjusted R-squared values across sectors and years (2002-2020), demonstrating the deterministic linear nature of the aggregation.

This proves that the overall ESG score is not the result of a complex, non-linear AI model, but a straightforward, deterministic weighted sum.

Sector-Specific Weighting and Governance

The weights are "industry-aware." For example, in the Energy sector, the highest weights are typically assigned to the Emissions Score and Management Score. In contrast, for Banking, the Management Score and Workforce Score dominate. Notably, the "Management Score" often carries the highest weight across many sectors (e.g., ~0.30 for Software & IT Services), suggesting that governance structures are frequently the most significant driver of the final ESG rating.

Discussion and Limitations

A critical observation is Refinitiv's practice of historical recalculation. When the methodology is updated, all previous years' scores are recalculated using the new weights. While this ensures longitudinal comparability, it can be misleading if investors are unaware that "historical" scores have been retroactively altered.

While this paper successfully decodes the aggregation of category scores, it does not fully explore the predictive power of raw data versus aggregated scores regarding stock returns. Some individual company regressions yielded "abnormal coefficients" or "NA" results, likely due to data gaps. Future research should investigate whether the 750+ raw data points, if aggregated through non-linear models, could better predict financial volatility than the current rigid weighting system.

Conclusion

The Refinitiv ESG scoring methodology is a transparent, linear aggregation of category scores. While the "black box" is actually a simple weighted sum, the rigidity of this system and the practice of retroactive score adjustments are factors investors must carefully consider when integrating these ratings into their decision-making processes.

Summary updated

Abstract

Whether the Refinitiv provide a reliable and trusted methodology in the process of aggregating 10 category scores to overall score?
BibTeX
@misc{chen2023investigate,
  author = {Zhi Chen},
  title = {{Investigate The ESG Score Methodology}},
  year = {2023},
  eprint = {2312.00202},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.00202},
}
Figure