Inflow, Outflow, and Reciprocity in Machine Learning

Mukund Sundararajan, Walid KricheneICML 2023 · OpenReview

Machine learning models are fundamentally built on pooled data. While extensive research has focused on data valuation—measuring how much a specific data point contributes to a model—we rarely ask if the resulting "deal" is fair for the individual contributor. In systems like Federated Learning, Recommender Systems, or healthcare predictive apps, users contribute their personal data and receive a service in return.

However, existing research often remains "one-sided," focusing either strictly on user contribution or strictly on user privacy. This paper introduces a formal framework to measure the "value exchange" in machine learning: how much an individual’s data helps others versus how much they benefit from the model. Understanding this balance is critical for preventing "free-rider" problems and creating sustainable, fair incentives in data ecosystems.

A Framework for Value Exchange

To quantify this balance, the authors propose a framework built on two core metrics:

  1. Inflow ($I_u$): The aggregate influence of everyone else’s data on the predictions made for user $u$. (How much the user benefits).
  2. Outflow ($O_u$): The aggregate influence of user $u$’s data on the predictions made for everyone else. (How much the user contributes).

From these, we derive $\alpha$-Reciprocity. A system is considered reciprocal if, for most individuals, the ratio of inflow to outflow is close to 1.

To measure this "influence," the researchers compared two techniques:

  • Marginal Influence: A "leave-one-out" approach that retrains the model without a specific user's data.
  • TracIn: A gradient-based method that tracks how much a specific training example reduces the loss of a prediction during the Stochastic Gradient Descent (SGD) process.

A major technical contribution of this work is an efficiency optimization for TracIn. Normally, calculating pairwise influence between all training and inference points is quadratically expensive ($O(N^2)$). The authors demonstrate that by summing gradients first, they can reduce this to linear complexity ($O(N)$), making influence mapping feasible for large-scale datasets.

Key Insights: SGD and Inherent Reciprocity

The paper’s most striking theoretical result is that under standard independent and identically distributed (IID) assumptions, models trained with SGD are perfectly reciprocal in expectation. This occurs because the gradient "dot product" used to measure influence in TracIn is symmetric.

Time evolution of reciprocity during training

However, the authors emphasize that reciprocity shouldn't necessarily be the primary goal of a model (like accuracy), but rather a guardrail (like privacy or fairness). It serves as a monitoring metric to ensure the system isn't becoming exploitative over time.

Furthermore, while the math suggests reciprocity in expectation, individual heterogeneity matters. "Outliers"—such as users with highly unique tastes or rare medical conditions—might experience lower reciprocity because their data doesn't generalize well to help others, and the bulk of others' data doesn't help them.

Empirical Results

The researchers evaluated their framework on three distinct datasets: MovieLens 100K (collaborative filtering), a Diabetes dataset, and a Breast Cancer dataset.

In experiments, TracIn proved significantly more stable than Marginal Influence. Removing one user in a large dataset often results in a nearly identical model, leading to a low signal-to-noise ratio for Marginal Influence.

Distribution of reciprocity across users

The empirical results strongly supported the theoretical claims:

  • High Correlation: In the MovieLens experiments, the correlation between a user's inflow and outflow was remarkably high (0.89 for a single run and 0.98 when averaged across multiple splits).
  • Broad Reciprocity: On the MovieLens data, 75% of individuals had a reciprocity ratio within a reasonable range, even though the theoretical "single-pass" assumption of the proof was violated (the model was trained for 1000 steps).
  • Healthcare Stability: For the Diabetes and Breast Cancer datasets, the models were found to be highly reciprocal; for instance, the Breast Cancer model achieved a TracIn-reciprocity score of 0.93.

Limitations and Future Directions

While this framework provides a strong foundation for measuring fairness in data ecosystems, it has limitations. It relies on a "Utility Assumption," where a user’s benefit is strictly tied to the model's loss function. In reality, a user might value novelty in a recommender system more than strict accuracy.

Additionally, the theory assumes stationarity—that users are present for both training and inference. In the real world, new users might join a system late, generating high inflow but zero outflow, which temporarily breaks the balance. Finally, the framework does not account for non-ML value, such as the intrinsic value provided by content creators on a platform.

A promising future direction is the development of "active balancing" algorithms: training methods that enforce reciprocity by dynamically weighting examples during SGD, ensuring no participant is over-contributing without receiving commensurate benefit.

Summary updated

Abstract

Data is pooled across entities (individuals or enterprises) to create machine learning models, and sometimes, the entities that contribute the data also benefit from the models. Consider for instance a recommender system (e.g. Spotify, Instagram or YouTube), a health care app that predicts the risk for some disease, or a service built by pooling data across enterprises. In this work we propose a framework to study this value exchange, i.e., we model and measure contributions (outflows), benefits (inflows) and the balance between contributions and benefits (the degree of reciprocity). We show theoretically, and via experiments that under certain distributional assumptions, some classes of models are approximately reciprocal. These results only scratch the surface; we conclude with several open directions.
BibTeX
@inproceedings{sundararajan2023inflow,
  author = {Mukund Sundararajan and Walid Krichene},
  title = {{Inflow, Outflow, and Reciprocity in Machine Learning}},
  booktitle = {International Conference on Machine Learning},
  year = {2023},
  url = {https://openreview.net/forum?id=9PJ2V6qvQL},
}
Figure