Abstract In distributed machine learning, centralizing data is often impractical due to privacy, bandwidth, and storage constraints. This paper introduces Argumentation-based Distributed Induction (ADI), a framework where multiple agents collaboratively refine a shared model by exchanging specific counterexamples rather than entire datasets. By treating model refinement as a logical debate, agents can converge on a hypothesis that matches the accuracy of centralized models while sharing significantly less data. Experimental results across propositional and relational datasets demonstrate that this approach not only improves communication efficiency but can also enhance model generalization through a natural regularization effect.
Introduction
In the ideal machine learning scenario, all relevant data is accessible in a single, centralized repository. However, real-world data is frequently distributed across different servers, organizations, or devices. Centralizing this data is often prohibited by privacy concerns or limited by massive file sizes. The challenge of Distributed Induction is to enable these separate entities to learn a single, cohesive model without the overhead or risk of raw data movement.
Previous approaches to distributed learning often relied on simple model merging or voting, which can lead to inconsistencies. This paper proposes a different path: Can agents use the logic of human-style argumentation to refine their models collaboratively?
Methodology: Knowledge as Rules
The ADI framework requires agents to represent their knowledge as a disjunction of rules (e.g., "If it has feathers and flies, it's a bird"). This representation is highly flexible and compatible with standard induction algorithms like ID3 (decision trees), CN2, or relational learners like INDIE.
As shown in Figure 2, a standard decision tree can be decomposed into a set of independent rules. This allows agents to propose specific components of their hypothesis for "peer review" by other agents.
The ADI and RADI Protocols
The core of the framework lies in two communication protocols designed to minimize data leakage while maximizing model consistency:
- ADI (Argumentation-based Distributed Induction): Agents take turns proposing rules. If Agent B finds a "defeating counterexample" in its local data that proves Agent A's rule is incorrect, it sends only that specific example back. Agent A then updates its model and tries again.
- RADI (Reduced ADI): This is an optimized version where Agent B evaluates all of Agent A's proposed rules simultaneously, identifying the minimum set of counterexamples needed to invalidate all incorrect rules. This significantly reduces the number of communication rounds.
Figure 4 illustrates this process: rules are tested against local data to find specific instances ($e_1, e_2, e_3$) that contradict the proposed hypothesis. This "argumentation as a filter" ensures that agents only share the data that is strictly necessary to correct misconceptions.
Key Results and Insights
The researchers evaluated the framework using standard UCI benchmarks (Soybean, Zoology, Cars) and a complex relational dataset (Sponges). Several key insights emerged:
- Accuracy Parity: In almost every test, ADI and RADI achieved training set accuracy indistinguishable from a centralized approach where one agent has all the data.
- Data Efficiency: Agents only had to share between 19% and 21% of their local examples to reach a consensus, representing a massive win for privacy and bandwidth.
- The "Peer Review" Effect: Interestingly, models produced through argumentation often performed better on test data than centralized models. Because rules had to survive "attacks" from another agent's dataset, they were less likely to overfit to local noise, acting as a natural form of regularization.
- Computational Speed: While the basic ADI protocol took approximately 155 seconds in tests, the optimized RADI protocol reduced that time to about 18 seconds, making it viable for real-world distributed systems.
Conclusion and Future Work
While the current framework is limited to two-agent scenarios and rule-based models, it demonstrates a powerful proof-of-concept for privacy-preserving distributed learning. By focusing on counterexamples—the data that disproves current assumptions—agents can ignore the "easy" data they already agree on and focus their communication on resolving conflicts. Future work aims to scale this to multi-agent environments and explore "rule-based arguments" that could potentially eliminate the need to share raw examples entirely.
Summary updated