ModelGo: A Tool for Machine Learning License Analysis

Moming Duan, Qinbin Li, Bingsheng HeWWW 2024 · OpenReview

Abstract

Productionizing machine learning projects is inherently complex, involving a multitude of interconnected components that are assembled like LEGO blocks and evolve throughout the development lifecycle. These components encompass software, databases, and models, each subject to various licenses governing their reuse and redistribution. However, existing license analysis approaches for Open Source Software (OSS) are not well-suited for this context. For instance, some projects are licensed without explicitly granting sublicensing rights, or the granted rights can be revoked, potentially exposing their derivatives to legal risks. Indeed, the analysis of licenses in machine learning projects grows significantly more intricate as it involves interactions among diverse types of licenses and licensed materials. To the best of our knowledge, no prior research has delved into the exploration of license conflicts within this domain. In this paper, we introduce ModelGo, a practical tool for auditing potential legal risks in machine learning projects to enhance compliance and fairness. With ModelGo, we present license assessment reports based on 5 use cases with diverse model-reusing scenarios, rendered by real-world machine learning components. Finally, we summarize the reasons behind license conflicts and provide guidelines for minimizing them.

Introduction

Building a production-ready machine learning (ML) model is rarely a "from scratch" endeavor. Modern developers assemble projects using a mix of open-source code, massive third-party datasets, and pre-trained model weights. Each of these components comes with its own license, and when they are integrated through processes like fine-tuning, ensembling, or distillation, the legal implications become incredibly murky.

Traditional Open Source Software (OSS) tools are designed to detect code replication or linking, but they are largely blind to ML-specific "reuse" mechanisms. This creates a "licensing disorganization" in the field, where many projects have ambiguous or even invalid license declarations, exposing developers to significant legal risks. ModelGo is the first automated auditing tool specifically designed to navigate these legal "LEGO blocks" of machine learning development.

The Challenge of ML Licensing

The core issue lies in the gap between technical AI activities and traditional legal language. Standard OSS tools cannot answer critical questions such as:

  • If a model is fine-tuned on a dataset with a "Non-Commercial" license, can the resulting weights be sold?
  • Does creating a vector embedding of a copyrighted corpus count as a "derivative work"?
  • How do "Responsible AI Licenses" (RAIL), which restrict behavioral use, interact with standard permissive or copyleft licenses?

Methodology: The ModelGo Framework

To bridge this gap, ModelGo utilizes two novel frameworks to categorize and analyze ML projects.

1. A Taxonomy for AI Activities

Since legal terms like "modify" or "adapt" are vague in the context of neural networks, ModelGo categorizes AI activities into four buckets based on their output:

  • Combination: Strong separation (e.g., ensembling different models).
  • Amalgamation: Weak separation where components are unrecoverably mixed (e.g., fine-tuning or model fusion).
  • Distillation: Derivatives from concepts (e.g., a "student" model learning from a "teacher" model).
  • Generation: Derivatives from data (e.g., synthetic data created by a generative model).

2. Ternary Dependency Trees

Unlike simple software dependency lists, ModelGo uses a tree structure to distinguish how a component is used within the project lifecycle:

  • Mix-works: Components actually embedded in the final product (like model weights).
  • Sub-works: Components released alongside the project but not embedded (like the training dataset).
  • Aux-works: Tools used to build the project but not released (like the original model used for distillation).

By traversing this tree, ModelGo can determine if a "copyleft" license on a dataset "proliferates" (infects) the final model weights.

Ecosystem Analysis and Key Insights

To evaluate ModelGo, the researchers profiled the Hugging Face ecosystem, specifically looking at projects with over 1,000 likes. They manually transcribed the licenses of 15 common data sources (including Wikipedia and Common Crawl) and 11 popular models (including Llama 2, BERT, and Stable Diffusion). The tool currently supports licenses covering over 96% of the licensed content on Hugging Face.

Key Findings

  • The "Generation" Loophole: Traditional licenses (like Creative Commons "NoDerivs") often fail to govern the outputs of generative models. Developers can sometimes bypass restrictive data licenses by using that data to generate synthetic data, which remains largely "ungoverned" under current frameworks—unless a RAIL license is involved.
  • RAIL Incompatibility: Responsible AI Licenses (like those for Llama 2 or BLOOM) often include "behavioral restrictions" (e.g., prohibiting use for medical advice). These are technically "non-permissive" terms that can trigger conflicts when mixed with standard OSS licenses like GPL-3.0.
  • Copyleft Proliferation: The study confirms that fine-tuning or "amalgamating" models can force a project to inherit restrictive licenses, potentially requiring a company to open-source their proprietary weights.

Case Studies

The authors demonstrated ModelGo through five real-world case studies, highlighting specific risks:

  • Corpus Combination: Translating a dataset triggers "ShareAlike" conditions, forcing the new dataset to remain under the original license.
  • Mixture of Experts: Identified a direct conflict between the GPL-3.0 license and the BLOOM-RAIL license, warning that the final work might be legally un-releasable.
  • Knowledge Transfer: Proved that "model fusion" (averaging weights) triggers "remix" and "adapt" terms, even though no code was explicitly copied.

Limitations and Conclusion

While ModelGo provides a significant step forward, it has limitations. License terms currently require manual transcription into YAML format, as automated extraction via LLMs proved unreliable for legal precision. Furthermore, ModelGo is a risk-assessment tool, not a "robot lawyer"; it identifies potential conflicts, but final legal interpretations require human counsel.

ModelGo serves as a vital tool for ML engineers to enhance compliance and fairness, providing a structured way to audit the complex legal dependencies inherent in modern AI development.

Summary updated

Abstract

Productionizing machine learning projects is inherently complex, involving a multitude of interconnected components that are assembled like LEGO blocks and evolve throughout development lifecycle. These components encompass software, databases, and models, each subject to various licenses governing their reuse and redistribution. However, existing license analysis approaches for Open Source Software (OSS) are not well-suited for this context. For instance, some projects are licensed without explicitly granting sublicensing rights, or the granted rights can be revoked, potentially exposing their derivatives to legal risks. Indeed, the analysis of licenses in machine learning projects grows significantly more intricate as it involves interactions among diverse types of licenses and licensed materials. To the best of our knowledge, no prior research has delved into the exploration of license conflicts within this domain. In this paper, we introduce ModelGo, a practical tool for auditing potential legal risks in machine learning projects to enhance compliance and fairness. With ModelGo, we present license assessment reports based on 5 use cases with diverse model-reusing scenarios, rendered by real-world machine learning components. Finally, we summarize the reasons behind license conflicts and provide guidelines for minimizing them.
BibTeX
@inproceedings{duan2024modelgo,
  author = {Moming Duan and Qinbin Li and Bingsheng He},
  title = {{ModelGo: A Tool for Machine Learning License Analysis}},
  booktitle = {WWW},
  year = {2024},
  url = {https://openreview.net/forum?id=Et9rHdWGAZ},
}

Loading related content...

Figure