Joint Feature Selection and Outlier Detection

22 concepts14 papers1 proposal

In high-dimensional data analysis, researchers face a fundamental circular dependency: to identify outliers, one needs a clean set of relevant features; however, to identify which features are truly relevant, one must first account for the presence of outliers that distort statistical measures. Traditional pipelines attempt to solve this sequentially—performing feature selection as a preprocessing step before applying an anomaly detection algorithm.

However, as noted by Guansong Pang et al. in their work on HOUR, this decoupled approach often fails because the initial selection process is "unaware" of the detection objective. Noisy or irrelevant features can mask outliers, while the outliers themselves can "poison" the feature selection process, leading to a suboptimal subspace where anomalies remain hidden.

To overcome this, a growing body of research proposes joint optimization, where feature selection and outlier detection are treated as a single, unified task.

The Failure of Task-Agnostic Selection

Most standard feature selection methods are task-agnostic, focusing on general data characteristics like variance. For example, Principal Component Analysis (PCA) typically retains components with the highest variance. Yet, Tetiana Gula and João P. C. Bertoldo demonstrate that in image anomaly detection, the most discriminative information often resides in low-variance eigencomponents that standard methods would discard.

Similarly, Mattia Carletti et al. argue in their work on DIFFI that unsupervised feature selection should be model-specific. By evaluating features based on how effectively an Isolation Forest can partition them, they ensure the selected subset is optimized for isolating rare instances rather than describing the majority of the data.

Mechanisms of Joint Optimization

The core intuition behind joint optimization is to create a feedback loop between the "what" (relevant features) and the "who" (anomalous points). This is typically achieved through three main strategies:

1. Unified Objective Functions

Instead of two separate algorithms, researchers define a single mathematical objective that rewards both feature sparsity and outlier contrast. Li cheng et al. propose the ODEFS framework, which uses a pairwise ranking loss to simultaneously learn feature weights and outlier scores. By ranking candidate outliers against unlabeled data points within the optimization loop, the model identifies the specific subspace where the "gap" between normal and abnormal is widest.

2. Iterative Refinement and Pseudo-labeling

When a single objective is too complex, models use an iterative approach. Guansong Pang et al. developed a sequential ensemble learning method where the outlier scores from one iteration serve as "pseudo-labels" to guide sparse modeling in the next. This mutual refinement ensures that the feature space specializes for the specific detection algorithm being used.

3. Alternating Optimization in Streams

In dynamic environments where data flows in streams, the challenge is even greater. Shuo Lei and Xuchao Zhang introduced RoOFS, which handles adversarial corruption by alternating between identifying uncorrupted data points and selecting relevant features on the fly. This prevents error propagation; the model uses its current best guess of "clean" data to pick features, which in turn makes the next round of outlier detection more accurate.

Applications Across Domains

This integrated philosophy has proven effective across diverse data types:

  • Attributed Networks: Minnan Luo et al. developed ANOMALOUS, which uses CUR decomposition to jointly filter noisy attributes and identify nodes that deviate from both structural and attribute patterns.
  • Computer Vision: Yumin Suh et al. showed that integrating outlier rejection directly into subgraph matching objectives allows algorithms to ignore "clutter" that would otherwise force incorrect correspondences.
  • Hyperspectral Imaging: Dandan Ma et al. utilize task-driven learning to select spectral bands that maximize the distinction between background noise and anomalies.
  • Explainable AI: Khushnaseeb Roshan demonstrated that using SHAP values to identify the primary drivers of reconstruction error in autoencoders allows for a targeted feature selection strategy that significantly improves network anomaly detection.

Why It Works: The Power of Contrast

The common thread across these papers is the shift from representation to discrimination. Traditional feature selection asks, "Which features represent the data best?" Joint optimization asks, "Which features make the outliers stand out most?" By focusing on the margin between the typical and the atypical—whether through homophily couplings (Pang et al.) or Dirichlet Mixture Models (Ziou and Boutemedjet)—these methods ensure that the "signal" of an anomaly is never drowned out by the "noise" of irrelevant dimensions.

Key Foundational Work

Simultaneous Non-gaussian Data Clustering, Feature Selection and Outliers Rejection (2011) — contributed 1 concept elements to this direction.
Subgraph matching using compactness prior for robust feature correspondence (2015) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 12 papers
  2. Version 1 · 2026-10-04 · 10 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_joint_feature_outlier_optimization_9b42195a,
  title = {joint feature-outlier optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/joint-feature-outlier-optimization/rdr_9b42195ad68f45979f75b1f3a97bb01e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_joint_feature_outlier_optimization_9b42195a_v2,
  title = {joint feature-outlier optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/joint-feature-outlier-optimization/rdr_9b42195ad68f45979f75b1f3a97bb01e/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake