Deep Semantic Compliance Advisor for Unstructured Document Compliance Checking

Honglei Guo, Bang An, Zhili Guo, Zhong SuIJCAI 2020 · DOI

Introduction

In the highly regulated banking sector, compliance monitoring is a multi-billion dollar challenge. Financial institutions must continuously verify that thousands of unstructured documents, such as complex legal contracts, align with evolving regulatory frameworks. Historically, this has been a manual, high-cost process prone to inconsistency. Traditional automated solutions often rely on keyword matching or rigid rule-based systems, which fail to capture the "deep semantics" of legal language—where two clauses may use different terminology but share the same legal intent (entailment), or appear similar while containing critical contradictions.

To address these challenges, researchers from IBM Research - China have developed the Deep Semantic Compliance Advisor (DSCA). This platform automates the review process by leveraging Graph Neural Networks (GNNs) and attention mechanisms to capture both syntactic structure and thematic relatedness. The system effectively reduces a four-hour manual review task to approximately one minute while outperforming state-of-the-art natural language processing (NLP) models.

Methodology

The DSCA platform operates at two distinct semantic levels to ensure comprehensive document analysis.

Statement-Level: GNN-based Syntactic Encoding

For individual sentence comparison, the researchers proposed a novel GNN encoder. Unlike sequential models that treat sentences as simple word chains, the DSCA converts sentences into labeled directed graphs. These graphs incorporate grammatical dependencies, opposite dependencies, and sequential order.

Architecture of the GNN-based syntactic encoder for sentence-level representation.

Information flows along these structural edges through aggregation layers, allowing the network to understand a word's context based on its specific grammatical role. A stochastic readout mechanism, inspired by Variational Autoencoders, is then used to produce a generalized sentence vector, reducing redundancy and improving the model's ability to reason across different linguistic structures.

Clause-Level: Attention-based Relatedness

Legal clauses are frequently lengthy and verbose, with relevant information often scattered. To identify corresponding requirements between contracts and regulations, the DSCA employs an attention-based neural model. This component utilizes "soft-alignment" to link semantically related terms (e.g., "liability" and "responsibility") even when they are not lexical matches. A multi-dimensional comparison function—incorporating cosine distance, Euclidean distance, and element-wise operations—determines the final semantic relatedness between paragraphs.

Experimental Evaluation

The DSCA was evaluated using two primary datasets:

  1. Stanford Natural Language Inference (SNLI): A benchmark of 570,000 sentence pairs used to test general linguistic reasoning.
  2. Proprietary Banking Dataset: A real-world dataset of English contracts containing over 2,900 manually labeled pairs provided by legal professionals.

Key Results and Discussion

The DSCA demonstrated superior performance across all metrics. On the SNLI benchmark, the GNN encoder achieved 86.6% accuracy, surpassing existing models like Gumble TreeLSTM. Crucially, the GNN approach proved significantly more robust when processing long sentences (16+ words), where standard sequential models typically fail to maintain distant dependencies.

Performance comparison showing the DSCA model's robustness to increasing sentence length compared to PSAN.

On the real-world banking dataset, the DSCA reached 91.1% accuracy in clause matching, notably outperforming Support Vector Machines (80.6%) and standard LSTMs (89.1%). Beyond accuracy, the system's operational impact is profound, reducing the time required for a single contract check from over four hours to just one minute.

Conclusion and Future Work

The Deep Semantic Compliance Advisor represents a significant advancement in automated legal tech, proving that incorporating explicit syntactic priors via GNNs enhances semantic understanding in complex domains. Future research will focus on reducing the dependency on initial parsers, scaling the model using semi-supervised data labeling, and improving explainability to provide users with natural language justifications for non-compliance findings.

Summary updated

BibTeX
@inproceedings{guo2020deep,
  author = {Honglei Guo and Bang An and Zhili Guo and Zhong Su},
  title = {{Deep Semantic Compliance Advisor for Unstructured Document Compliance Checking}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/613},
}
Figure