Dual-Regularized Multi-View Outlier Detection

Handong Zhao, Yun Fu

Abstract

Detecting outliers in multi-view data is a complex task due to the presence of different types of anomalies. Traditional methods often focus on either class-outliers (inconsistencies across views) or attribute-outliers (abnormalities within views), but rarely both. This paper introduces DMOD (Dual-regularized Multi-view Outlier Detection), a novel framework that simultaneously identifies both types of anomalies by decomposing data into latent cluster assignments and sample-specific errors. Experimental results across multiple benchmark datasets demonstrate that DMOD significantly outperforms state-of-the-art methods, achieving substantial gains in detection accuracy.

Introduction

In many real-world applications, data is collected from multiple sources or "views." For instance, a person's identity can be represented by facial images and voice recordings. Outlier detection in such multi-view contexts is challenging because an object might appear normal in one view but reveal its anomalous nature when compared across views.

Existing research typically addresses two distinct categories of outliers:

  1. Class-outliers: Samples that exhibit inconsistent cluster memberships across different views (e.g., appearing as "Class A" in View 1 but "Class B" in View 2).
  2. Attribute-outliers: Samples that are inherently abnormal within individual views, such as corrupted data points.
Illustration of outliers in View 2: the red circle represents an attribute-outlier, while the blue triangle represents a class-outlier that shifted clusters relative to View 1.

Most current multi-view methods prioritize class-outliers, while single-view methods handle attribute-outliers. DMOD bridges this gap by providing a unified framework to detect both simultaneously.

Methodology: The DMOD Framework

The DMOD framework utilizes matrix factorization to represent data from each view $i$ as: $$X^{(i)} = H^{(i)}G^{(i)} + S^{(i)}$$

Detecting Class-Outliers

The term $G^{(i)}$ represents latent coefficients corresponding to cluster memberships. DMOD employs a dual-regularization term to align these memberships across views using an alignment matrix $M_{ij}$. This approach accounts for the "orderless" nature of clusters (where Cluster 1 in one view might correspond to Cluster 3 in another). Samples that cannot be consistently aligned across views are flagged as class-outliers.

Detecting Attribute-Outliers

The term $S^{(i)}$ captures sample-specific errors or "weirdness." By applying an $\ell_{2,1}$-norm to this error matrix, the model encourages row-sparsity, forcing most errors to zero. Non-zero rows effectively highlight attribute-outliers that deviate from the expected cluster structure in the original feature space.

Outlier Measurement Criterion

The authors propose a unified Outlier Measurement Criterion ($\phi$) that integrates inconsistency in latent cluster assignments and the magnitude of error terms into a single outlier score for each sample.

Experimental Evaluation

DMOD was evaluated on five benchmark datasets, including UCI datasets (iris, breast cancer, ionosphere, letter) and the BUAA VisNir Face Database. Outliers were manually injected to test the model's robustness: class-outliers were created by swapping feature subsets, and attribute-outliers were generated using random values.

Performance Results

DMOD consistently outperformed state-of-the-art baselines, including HOAD (multi-view) and LRR (single-view).

ROC curve comparison on the letter dataset, showing DMOD (Ours) achieving superior hit rates at lower false alarm rates compared to baselines.

On the letter dataset, DMOD achieved a remarkable 26.13% improvement in Area Under the Curve (AUC) over the next best method. Unlike specialized methods that excel only in specific outlier scenarios, DMOD maintained dominant performance regardless of the mix of class and attribute outliers.

Discussion and Future Work

While DMOD is highly effective and scales linearly with the number of samples, it does require pre-setting the number of clusters ($K$). Sensitivity analysis shows that setting $K$ too low can lead to information loss and decreased performance. Future research could focus on automating the selection of $K$ and refining the thresholding process for binary outlier classification.

Conclusion

DMOD represents a significant advancement in multi-view outlier detection by treating class and attribute anomalies as distinct but related phenomena. By operating in both latent and original feature spaces, it provides a robust and comprehensive solution for complex real-world data analysis.

Summary updated

Abstract

Multi-view outlier detection is a challenging problem due to the inconsistent behaviors and complicated distributions of samples across different views. The existing approaches are designed to identify the outlier exhibiting inconsistent characteristics across different views. However, due to the inevitable system errors caused by data-captured sensors or others, there always exists another type of outlier, which consistently behaves abnormally in individual view. Unfortunately, this kind of outlier is neglected by all the existing multi-view outlier detection methods, consequently their outlier detection performances are dramatically harmed. In this paper, we propose a novel Dual-regularized Multi-view Outlier Detection method (DMOD) to detect both kinds of anomalies simultaneously. By representing the multi-view data with latent coefficients and sample-specific errors, we characterize each kind of outlier explicitly. Moreover, an outlier measurement criterion is well-designed to quantify the inconsistency. To solve the proposed non-smooth model, a novel optimization algorithm is proposed in an iterative manner. We evaluate our method on five datasets with different outlier settings. The consistent superior results to other state-of-the-art methods demonstrate the effectiveness of our approach.
BibTeX
@inproceedings{zhao2015dual,
  author = {Handong Zhao and Yun Fu},
  title = {{Dual-Regularized Multi-View Outlier Detection}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2015},
}
Figure