Abstract: Lifelong machine learning (LML) is a paradigm to design adaptive agents that can learn in dynamic environments. Current LML algorithms consider a single agent that has centralized access to all data. However, given privacy and security constraints, data might be distributed among multiple agents that can collaborate and learn from collective experience. Our goal is to extend LML from a single agent to a network of multiple agents that collectively learn a series of tasks.
Introduction
Traditional Lifelong Machine Learning (LML) focuses on a single agent that learns a sequence of tasks, building up a knowledge base over time to improve its performance on future tasks. However, this "lone wolf" approach is often impractical in real-world scenarios where data is siloed due to privacy concerns, bandwidth constraints, or physical distribution—such as in fleets of robots or distributed wearable devices.
To address these challenges, this paper introduces the Collective Lifelong Learning Algorithm (CoLLA). CoLLA shifts the LML paradigm from a single-agent setting to a collaborative network, allowing multiple agents to share and refine a collective knowledge base without compromising data privacy or incurring high communication costs.
The CoLLA Framework
The CoLLA framework is built upon several technical pillars designed for distributed, online learning:
- Parametric Task Modeling: Following the GO-MTL framework, each task's parameters are modeled as a sparse linear combination of "atoms" from a local dictionary (the knowledge base).
- Graph-Based Communication: Agents operate as nodes in an undirected graph, communicating only with their immediate neighbors to maintain scalability.
- Distributed Optimization: The framework employs the Alternating Direction Method of Multipliers (ADMM) to ensure agents reach a consensus on a global knowledge base without a central controller.
- Online Learning: The algorithm is designed to update knowledge bases incrementally as new tasks arrive, avoiding the need for full re-processing of historical data.
By "gossiping" with neighbors, agents nudge their local knowledge bases toward a shared consensus, effectively benefiting from the collective experience of the entire network.
Experimental Evaluation
The researchers evaluated CoLLA using the Computer Survey Dataset, which involves predicting the likelihood of a subject purchasing one of 20 different computers. The experiment treated 190 subjects as individual tasks, distributed across 19 agents (10 tasks per agent). The primary metric for evaluation was "jumpstart" performance—the percentage improvement in initial performance on a new task due to transferred knowledge from previous tasks.
Results and Discussion
The experimental results demonstrate that collaboration significantly enhances learning effectiveness. As shown in the table below, CoLLA consistently outperforms the single-agent ELLA baseline across all tested network topologies.
Key observations from the results include:
- Topology Impact: The structure of the network directly influences performance. The Complete (CP) graph, where all agents communicate, achieved the highest distributed performance (46.05%). Even a simple Linear Tree (LT) configuration provided a ~5% boost over the single-agent ELLA (37.99%).
- Closing the Centralization Gap: While centralized methods like GO-MTL (61.81%) and Offline CoLLA (61.71%) provide an upper bound on performance, CoLLA demonstrates that distributed learning can achieve substantial gains. The authors note that CoLLA performs competitively with these centralized standards while maintaining data privacy.
- Privacy by Design: By sharing only knowledge base dictionaries rather than raw data or task-specific coefficients, CoLLA provides a layer of inherent privacy, making it suitable for sensitive applications.
Conclusion and Future Work
CoLLA represents a significant step toward practical, distributed lifelong learning. By enabling agents to learn collectively, the framework overcomes the limitations of siloed data and isolated learning. Future research will focus on extending CoLLA to dynamic networks with changing topologies and asynchronous agents to better reflect the complexities of real-world deployments.
Summary updated