Abstract
In this paper, we investigate the distributed parallel Support Vector Machine training strategy and propose a BSP-Based Support Vector Regression Machine Parallel Framework. This framework is capable of implementing most distributed Support Vector Regression (SVR) algorithms by abstracting the network topology among distributed nodes. We adopt the Bulk Synchronous Parallel (BSP) model to manage the exchange of support vectors across strongly connected graphs. Furthermore, we introduce dynamic algorithms that allow the communication graph to evolve during each BSP super-step. Evaluation on the KDD99 dataset demonstrates that the framework preserves the performance of original algorithms while providing significant computational speedups.
Introduction: The Challenge of Scaling SVR
Support Vector Regression (SVR) is a robust method for function fitting and pattern recognition. However, its primary drawback is computational complexity, typically scaling at $O(N^3)$ relative to the dataset size $N$. This makes traditional serial training impractical for large-scale datasets. While various Distributed Parallel SVR (DPSVR) algorithms have been proposed, they are often tied to specific, rigid network topologies. There is a clear need for a unified, programmable framework that can adapt to different communication strategies and handle dynamic node interactions.
The BSP Parallel Model
The proposed framework utilizes the Bulk Synchronous Parallel (BSP) model to structure the distributed training process. In this model, computation is divided into a sequence of "super-steps."
Each super-step consists of three distinct phases:
- Concurrent Computation: Each node performs SVR training on its local data subset.
- Communication: Nodes exchange their identified Support Vectors (SVs) with other nodes according to a defined topology.
- Barrier Synchronization: The system ensures all nodes have completed their communication before the next super-step begins.
Topology and Convergence
A critical requirement for distributed SVR to reach a global optimum is that the communication network must form a strongly connected graph. By treating the network topology as a programmable adjacency matrix, the framework becomes algorithm-agnostic, allowing researchers to implement Ring, Full Connected, Gather, or Cascade structures with ease.
Dynamic Topology and Efficiency
One of the framework's key innovations is the support for Dynamic Topologies. This allows the communication graph to change between super-steps. For instance, a "Dynamic Cascade" can achieve the same results as a static 15-node cascade using only 8 physical nodes by reassigning node roles across different layers of the hierarchy.
Experimental Evaluation
The framework was evaluated using the KDD99 network intrusion detection dataset (190,578 records). The goal was to predict network attack patterns through regression.
Key Results
- Accuracy Preservation: The parallel framework maintained a detection precision of approximately 98%, matching the performance of serial SVR. This confirms that the distributed approach does not sacrifice model quality.
- Speedup: Training on a 12-node Cascade topology reduced the processing time from 667.75 seconds (serial) to 196.04 seconds, representing a ~3.4x speedup.
- Convergence Efficiency: Topologies like "Full Connected" and "Cascade" converged much faster (2–5 super-steps) than the "Ring" topology, which required one super-step per node to propagate information.
Conclusion
The BSP-based framework provides a flexible and efficient environment for parallelizing SVR. By abstracting communication into super-steps and supporting dynamic topologies, it allows for significant reductions in training time without sacrificing model accuracy. Future work will focus on improving load balancing to prevent bottlenecks caused by uneven distributions of support vectors across nodes.
Summary updated