ReportParse: A Unified NLP Tool for Extracting Document Structure and Semantics of Corporate Sustainability Reporting

Gaku Morio, Soh Young In, Jungah Yoon, Harri Rowlands, Christopher D. Manning

Introduction

Corporate sustainability reports have become a primary medium for companies to communicate environmental commitments. However, these documents—often lengthy, unstructured PDFs—present significant challenges for systematic analysis. Researchers and ESG (Environmental, Social, and Governance) analysts frequently encounter "greenwashing," where broad rhetorical commitments lack specific actionable data. At IJCAI-24, researchers introduced ReportParse, an open-source Python tool designed to standardize the extraction of document structure and semantic content from these complex reports.

Problem Statement: The Challenge of PDF Parsing

The explosion of mandatory climate disclosure has led to a vast volume of reports. These documents typically feature complex layouts, including multi-column text and embedded graphics, which are difficult for standard text extractors to process accurately. A critical issue identified by the authors is "reproducibility debt": different parsing tools (e.g., PyMuPDF vs. deepdoctection) yield different text extractions, leading to inconsistent downstream NLP analysis. ReportParse addresses this by providing a unified, reproducible pipeline.

Methodology: The Reader-Annotator Architecture

ReportParse utilizes a modular "Reader-Annotator" architecture, ensuring that semantic labels are directly linked to the physical document structure.

The ReportParse pipeline architecture, illustrating the flow from input PDF through layout analysis and annotation models to user interfaces.
  1. The Reader: This module identifies the physical layout using deepdoctection to recognize titles, text blocks, and lists. Text is then tokenized into sentences using spaCy.
  2. The Annotator: The tool applies specialized NLP models (e.g., ClimateBERT) to the extracted text. It is model-agnostic, allowing integration with various Hugging Face models for tasks like sentiment analysis or policy stance detection.
  3. Interfaces: The system includes a Python CLI for large-scale processing and a Gradio-based web interface for human-in-the-loop verification.

Key Results and Insights

The authors validated ReportParse through a pilot study of 2,480 reports from the energy and basic materials sectors (2014–2020).

The "Parser Effect"

A significant finding was that the choice of PDF reader significantly impacts final analysis. For instance, different parsers can lead to varying sentiment ratios for the same document, emphasizing the need for a standardized tool like ReportParse to ensure scientific reproducibility.

The tool identified a clear upward trend in corporate support for general IPCC and UN climate policies. However, a discrepancy emerged when analyzing specific energy transition policies.

Trends in corporate stances toward (a) general climate policies and (b) specific energy transition policies, showing a gap between broad rhetoric and specific action.

While supportive language for broad climate goals increased, support for specific transition mechanisms (e.g., carbon capture) did not show equivalent growth. This suggests a potential gap between high-level corporate rhetoric and concrete policy backing.

Conclusion and Future Work

ReportParse serves as a powerful "accelerator" for human experts, significantly reducing the time required to audit sustainability reports. While the tool improves efficiency, the authors note that error propagation from layout analysis remains a challenge. Future work will focus on quantifying reporting consistency over time and mitigating biases in underlying NLP models.

Project Website: https://github.com/climate-nlp/reportparse

Summary updated

BibTeX
@inproceedings{morio2024reportparse,
  author = {Gaku Morio and Soh Young In and Jungah Yoon and Harri Rowlands and Christopher D. Manning},
  title = {{ReportParse: A Unified NLP Tool for Extracting Document Structure and Semantics of Corporate Sustainability Reporting}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure