Abstract
With the rapid growth of Internet technologies, cloud computing and social networks have become ubiquitous. An increasing number of people participate in social networks and massive online social data are obtained. In order to exploit knowledge from copious amounts of data obtained and predict social behavior of users, we urge to realize data mining in social networks. Almost all online websites use cloud services to effectively process the large scale of social data, which are gathered from distributed data centers. These data are so large-scale, high-dimension and widely distributed that we propose a distributed sparse online algorithm to handle them. Additionally, privacy-protection is an important point in social networks. We should not compromise the privacy of individuals in networks, while these social data are being learned for data mining. Thus we also consider the privacy problem in this article. Our simulations shows that the appropriate sparsity of data would enhance the performance of our algorithm and the privacy-preserving method does not significantly hurt the performance of the proposed algorithm.
Introduction
Modern social media platforms generate hundreds of terabytes of data daily across global data centers. This "Social Big Data" presents a unique challenge characterized by three "H's": it is Huge in volume, High-dimensional (thousands of features per user), and Highly distributed across worldwide infrastructure.
Centralized machine learning is often impractical due to the prohibitive costs of data movement and significant privacy risks. Moreover, while user profiles contain thousands of data points, only a small subset is typically relevant to specific prediction tasks. This paper proposes a distributed online learning framework that leverages Lasso-based sparsity to manage high-dimensional data while incorporating differential privacy to safeguard user information.
Proposed Framework: Private Distributed Online Learning
The authors introduce a distributed sparse online algorithm designed for real-time processing across decentralized data centers. The framework integrates several key components:
- Distributed Optimization: Instead of relying on a central server, each data center maintains its own model parameter vector ($w$). Centers synchronize by exchanging these parameters with immediate neighbors in a communication graph.
- Online Mirror Descent (OMD): This serves as the core learning mechanism, allowing models to update incrementally as new data arrives.
- Lasso ($L_1$ Regularization): To address high dimensionality, the algorithm applies an $L_1$ penalty. This encourages sparsity by pushing irrelevant feature weights to zero, resulting in a model that is both computationally efficient and robust against overfitting.
- Differential Privacy (DP): To protect individual identities, the algorithm injects Laplace noise into model updates before they are broadcast to neighbors. This ensures that the inclusion of any single user's data does not significantly alter the shared model, mitigating the risk of data reconstruction attacks.
Experimental Evaluation
The algorithm was evaluated using a real-world social dataset comprising 100,000 entries with 10,000 dimensions. Simulations were conducted in a distributed environment using Hadoop on a Linux cluster.
Key Insights
- Sparsity as a Performance Driver: A critical finding was that inducing sparsity actually improved model accuracy. By filtering out irrelevant high-dimensional features, the model focuses on the most predictive signals. As shown in Figure 4, accuracy peaks when an optimal level of sparsity is achieved, outperforming non-sparse methods by nearly 18%.
- Manageable Privacy-Utility Trade-off: While increasing privacy (reducing $\epsilon$) naturally leads to higher "regret" (error), both mathematical proofs and empirical results demonstrate that the performance impact remains within acceptable bounds. The algorithm maintains a standard $O(\sqrt{T})$ regret bound, the gold standard for online learning.
- Scalability and Robustness: The system scales effectively; increasing the number of data centers resulted in only a marginal decline in accuracy. Furthermore, the method proved resilient to different network topologies, suggesting it is well-suited for diverse global infrastructure layouts.
Conclusion and Future Work
This research demonstrates that privacy, efficiency, and high-dimensional data mining can coexist in large-scale social networks. By combining distributed optimization with Lasso regularization and differential privacy, the proposed framework offers a scalable solution for modern social big data.
Future work will address real-world constraints such as network latency and communication delays between data centers. Additionally, the authors plan to investigate dynamic data scenarios where shifting social trends might require adaptive feature selection mechanisms.
Summary updated