LicenseGPT: A Fine-tuned Foundation Model for Publicly Available Dataset License Compliance

Jingwen Tan, Gopi Krishnan Rajbahadur, Zi Li, Xiangfu Song, Jianshan Lin, Dan Li, Zibin Zheng, Ahmed E. Hassan

Abstract

Dataset license compliance is a critical yet complex aspect of developing commercial AI products, particularly with the increasing use of publicly available datasets. Ambiguities in dataset licenses pose significant legal risks, making it challenging even for software IP lawyers to accurately interpret rights and obligations. In this paper, we introduce LicenseGPT, a fine-tuned foundation model (FM) specifically designed for dataset license compliance analysis. We first evaluate existing legal FMs and find that the best-performing model achieves a Prediction Agreement (PA) of only 43.75%. LicenseGPT, fine-tuned on a curated dataset of 500 licenses annotated by legal experts, significantly improves PA to 64.30%, outperforming both legal and general-purpose FMs. Through an A/B test and user study with software IP lawyers, we demonstrate that LicenseGPT reduces analysis time by 94.44%, from 108 seconds to 6 seconds per license, without compromising accuracy. Software IP lawyers perceive LicenseGPT as a valuable supplementary tool that enhances efficiency while acknowledging the need for human oversight in complex cases.

Introduction

The rapid expansion of AI development has turned dataset licensing into a "legal wild west." Unlike standardized Open Source Software (OSS) licenses, dataset licenses are frequently ambiguous, non-standardized, or obscured within complex "Terms of Use" pages. Research indicates that over 66% of dataset licenses are misrepresented on public platforms, often appearing more permissive than their actual legal terms allow. For software IP lawyers, determining commercial viability is a slow, manual process prone to error. While general Large Language Models (LLMs) excel at text summarization, they often struggle with the nuanced logic required for legal compliance, providing professional-sounding but legally inaccurate responses.

Methodology

To address these challenges, we developed LicenseGPT, a specialized foundation model fine-tuned for dataset license compliance. The development process followed a structured pipeline involving data collection, model selection, and expert validation.

Overview of the LicenseGPT research methodology and experimental design.

Data Collection and Annotation

We introduced the DL (Dataset Licenses) dataset, comprising 500 valid licenses from 10 major platforms including Hugging Face, GitHub, and Kaggle. These were categorized into General Licenses, Customized Licenses, and Official Terms of Use. Four software IP lawyers with 8–20 years of experience manually labeled each license for commercial use permissions and the underlying legal reasoning.

Model Development

We selected LawGPT (a 7B parameter model based on ChatGLM) as the base model due to its existing legal knowledge and local hosting capabilities. Fine-tuning was performed using LoRA (Low-Rank Adaptation) to efficiently adapt the model to specific dataset license terminology. We also developed a modular prompt design, separating system personas from specific user queries to optimize performance.

Results and Analysis

LicenseGPT significantly outperformed both general-purpose and existing legal foundation models in accuracy and efficiency.

Performance Comparison and the "Fluency Trap"

LicenseGPT achieved a Prediction Agreement (PA) of 64.30%, a substantial improvement over the base LawGPT (43.75%) and general models like Qwen-1.5 (59.72%) and GPT-4 (18.06%).

Performance metrics across various models, highlighting the discrepancy between semantic similarity and prediction agreement.

A key insight from our evaluation is the "Fluency vs. Accuracy" trap. As shown in the performance table, general models like GPT-4 (CGPT4) achieved very high Semantic Similarity (SS) scores (~94.80%) but very low Prediction Agreement (PA) (~18.06%). This indicates that while GPT-4 produces text that sounds like a lawyer, its legal conclusions are frequently incorrect. LicenseGPT prioritizes logical accuracy, reversing this trend to provide more reliable legal judgments.

Efficiency and Expert Perception

In a user study with professional IP lawyers, LicenseGPT reduced the average analysis time per license from 108 seconds to just 6 seconds—a 94.44% increase in efficiency. Lawyers perceived the tool as a "high-speed filter" or a digital "intern" capable of handling initial passes, allowing them to focus on complex, ambiguous cases.

Conclusion

LicenseGPT demonstrates the power of domain-specific fine-tuning in the legal sector. While its 64.3% accuracy rate indicates it is not a standalone replacement for human expertise, it serves as a powerful supplementary tool for increasing efficiency in dataset compliance workflows. Future work will explore scaling to larger base models (70B+) and advocating for machine-readable license standards like SPDX 3.0 to further reduce legal ambiguity in AI development.

Summary updated

Abstract

Dataset license compliance is a critical yet complex aspect of developing commercial AI products, particularly with the increasing use of publicly available datasets. Ambiguities in dataset licenses pose significant legal risks, making it challenging even for software IP lawyers to accurately interpret rights and obligations. In this paper, we introduce LicenseGPT, a fine-tuned foundation model (FM) specifically designed for dataset license compliance analysis. We first evaluate existing legal FMs (i.e., FMs specialized in understanding and processing legal texts) and find that the best-performing model achieves a Prediction Agreement (PA) of only 43.75%. LicenseGPT, fine-tuned on a curated dataset of 500 licenses annotated by legal experts, significantly improves PA to 64.30%, outperforming both legal and general-purpose FMs. Through an A/B test and user study with software IP lawyers, we demonstrate that LicenseGPT reduces analysis time by 94.44%, from 108 seconds to 6 seconds per license, without compromising accuracy. Software IP lawyers perceive LicenseGPT as a valuable supplementary tool that enhances efficiency while acknowledging the need for human oversight in complex cases. Our work underscores the potential of specialized AI tools in legal practice and offers a publicly available resource for practitioners and researchers.
BibTeX
@inproceedings{tan2025licensegpt,
  author = {Jingwen Tan and Gopi Krishnan Rajbahadur and Zi Li and Xiangfu Song and Jianshan Lin and Dan Li and Zibin Zheng and Ahmed E. Hassan},
  title = {{LicenseGPT: A Fine-tuned Foundation Model for Publicly Available Dataset License Compliance}},
  booktitle = {SIGSOFT FSE Companion},
  year = {2025},
  doi = {10.1145/3696630.3728530},
}
Figure