Abstract
Multistream classification involves labeling an unlabeled target data stream using knowledge from a separate, potentially biased, labeled source stream. This paper presents MSCRDR (MultiStream Classification with Relative Density Ratio), a framework designed to address the computational inefficiencies and asynchronous drift challenges inherent in existing methods. By employing an automated Expectation-Maximization (EM) approach for relative density ratio estimation, MSCRDR achieves significant improvements in both processing speed and classification accuracy.
1. Introduction
In modern data-driven applications like social media sentiment analysis, information arrives as continuous streams. A common challenge is the "multistream" scenario, where a model must classify an unlabeled "target" stream using a "source" stream of labeled data. Often, the source stream is biased, meaning its distribution does not perfectly match the target. Existing solutions utilize transfer learning and drift detection but suffer from high computational overhead due to quadratic-time optimization for bias correction and difficulties in managing asynchronous concept drift between streams.
2. The MSCRDR Framework
The proposed MSCRDR framework introduces three primary innovations to overcome these hurdles.
2.1 $\alpha$-Relative Density Ratio Estimation (RDRE)
Instead of modeling source and target distributions independently, MSCRDR directly estimates the ratio of their densities. This ratio serves as an "importance weight" for labeled data, allowing the classifier to prioritize source examples that are most representative of the current target distribution.
2.2 Automated Parameter Learning via EM
A critical challenge in RDRE is selecting the mixture parameter $\alpha$, which governs the smoothness of the density ratio. MSCRDR utilizes an Expectation-Maximization (EM) procedure to automatically learn the optimal $\alpha$ and model weights ($\theta$) from data windows. The authors provide theoretical proof that the underlying loss function is convex, ensuring convergence to a global optimum.
2.3 Drift Detection and Ensemble Update
The framework monitors distribution shifts using log-likelihood ratios of the density estimates. When a drift is detected in the target stream, the system updates its weighted ensemble of Support Vector Machines (SVMs) by replacing the weakest model with one trained on the most recent bias-corrected data.
3. Experimental Setup
The framework was evaluated against several baselines, including MSC and FUSION, using both real-world and synthetic datasets:
- Real-World: Forest Cover (geospatial), Yelp (Hotel/Restaurant reviews), and Amazon (Music/DVD/Electronics reviews).
- Synthetic: SynRBF@X, simulating various drift speeds.
4. Results and Analysis
MSCRDR demonstrated superior performance across all benchmarks.
4.1 Accuracy and Robustness
As shown in Figure 3, MSCRDR consistently maintains higher accuracy than MSC and FUSION. On the Forest Cover dataset, it reached 91.04% accuracy, significantly outperforming MSC (85.98%) and FUSION (82.73%). The poor performance of the "EnsSVM" baseline, which ignores bias, underscores the necessity of covariate shift correction.
4.2 Computational Efficiency
A key advantage of MSCRDR is its reduced execution time. By replacing quadratic optimization techniques (like KMM or KLIEP) with direct density ratio estimation and efficient EM-based parameter learning, the framework scales linearly with data volume, making it suitable for high-speed streaming environments.
5. Conclusion and Future Work
MSCRDR provides a robust and efficient solution for multistream classification by integrating automated bias correction with proactive drift detection. While highly effective, the system's performance is sensitive to window size ($N$), and future work could explore the integration of deep learning base learners or memory-optimized kernel storage for edge computing applications.
Summary updated