Assisted Learning: A Framework for Multi-Organization Learning

Xun Xian, Xinran Wang, Jie Ding, Reza Ghanadan

Abstract

In an increasing number of AI scenarios, collaborations among different organizations or agents (e.g., human and robots, mobile units) are often essential to accomplish an organization-specific mission. However, to avoid leaking useful and possibly proprietary information, organizations typically enforce stringent security constraints on sharing modeling algorithms and data, which significantly limits collaborations. In this work, we introduce the Assisted Learning framework for organizations to assist each other in supervised learning tasks without revealing any organization's algorithm, data, or even task. An organization seeks assistance by broadcasting task-specific but nonsensitive statistics and incorporating others' feedback in one or more iterations to eventually improve its predictive performance. Theoretical and experimental studies, including real-world medical benchmarks, show that Assisted Learning can often achieve near-oracle learning performance as if data and training processes were centralized.

Introduction

In the era of big data, valuable information is often siloed across different organizations. For example, a hospital might want to predict a patient's length of stay, but a separate clinical lab holds the specific bloodwork features that would make that prediction much more accurate. Traditional solutions involving data centralization are often precluded by privacy constraints (e.g., HIPAA, GDPR), intellectual property concerns regarding proprietary algorithms, and the risks associated with central controllers in Federated Learning (FL).

The Assisted Learning framework facilitates collaboration among diverse agents and organizations, such as mobile units and monitoring systems.

Assisted Learning (AL) addresses these challenges by providing a decentralized framework that allows organizations to collaborate on machine learning tasks without ever sharing raw data, proprietary models, or even their specific objectives. By iteratively exchanging non-sensitive statistics, participants can achieve "near-oracle" performance—matching the accuracy of a system where all data is centralized in a single database.

The Assisted Learning Protocol

The core of the framework is an iterative protocol operating in two primary stages:

Stage I: Iterative Learning

Instead of sharing data, the "learner" (Alice) sends a query to the "assistant" (Bob). This query consists of task-specific statistics—specifically, the residuals (the errors) from Alice's current model. Bob treats these residuals as "labels" and fits his own private model to them using his private features. He then sends the fitted values back to Alice. Alice incorporates this feedback to improve her model and repeats the process.

Stage II: Prediction

When a new patient arrives, Alice queries Bob for his prediction based on his features. She then combines his output with her own to produce a final result.

Key Innovations: * Model-Agnostic: Alice and Bob can use entirely different model architectures (e.g., Random Forest vs. Deep Neural Network). The protocol is indifferent to internal architectures. * Decentralized: Organizations communicate directly and autonomously without a mandatory master server. * Task Privacy: The assistant does not necessarily need to know the learner's specific prediction objective; they simply help minimize the provided error signal.

Experimental Evaluation

The authors evaluated Assisted Learning using synthetic regression (Friedman1) and real-world clinical data from the MIMIC-III database. The MIMIC-III benchmark involved over 38,000 patients, focusing on the Length of Stay (LOS) prediction task. In this setup, different data tables (e.g., Lab results, ICU charts) were treated as separate organizations.

Assisted Learning applied to MIMIC-III data tables. (a) shows data distribution across tasks, and (b) illustrates how Lab1 receives assistance from other divisions for the Length of Stay task.

Key Results

  • Near-Oracle Performance: In both synthetic and real-world tests, AL converged to "oracle" performance—the level achievable if all data were centralized. On the Friedman1 dataset, the error rate dropped significantly as soon as assistance began, quickly hitting the theoretical limit.
  • Synergy over Selection: AL outperforms simple model selection or averaging because it captures interactions between disparate feature sets, creating a synergy that utilizes the union of information.
  • Efficiency: For linear models, prediction loss decays exponentially with the number of assistance rounds, enabling rapid performance gains.

Discussion and Future Work

Assisted Learning demonstrates that iterative residual-passing can reach centralized performance limits while preserving privacy. However, the number of assistance rounds must be carefully managed, as it acts similarly to model complexity; too few rounds lead to underfitting, while too many can lead to overfitting.

Future research will address adversarial risks, such as "model extraction" by malicious learners, and extend the framework from regression to classification and clustering tasks. Additionally, developing statistical methods to help organizations identify high-utility partners before starting the learning process remains an open challenge.

Summary updated

Abstract

In an increasing number of AI scenarios, collaborations among different organizations or agents (e.g., human and robots, mobile units) are often essential to accomplish an organization-specific mission. However, to avoid leaking useful and possibly proprietary information, organizations typically enforce stringent security constraints on sharing modeling algorithms and data, which significantly limits collaborations. In this work, we introduce the Assisted Learning framework for organizations to assist each other in supervised learning tasks without revealing any organization's algorithm, data, or even task. An organization seeks assistance by broadcasting task-specific but nonsensitive statistics and incorporating others' feedback in one or more iterations to eventually improve its predictive performance. Theoretical and experimental studies, including real-world medical benchmarks, show that Assisted Learning can often achieve near-oracle learning performance as if data and training processes were centralized.
BibTeX
@inproceedings{xian2020assisted,
  author = {Xun Xian and Xinran Wang and Jie Ding and Reza Ghanadan},
  title = {{Assisted Learning: A Framework for Multi-Organization Learning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=2qAmyDhKjkU},
}
Figure