Open-world Reconstruction-based Classifier with Concept Drift Adaptation

Joanna KomorniczakarXiv 2026 · arXiv

In the rapidly shifting landscape of real-time data—from network traffic to sensor feeds—machine learning models often hit a wall. Most classifiers operate in a "closed world," assuming they will only ever see the categories they were trained on. But what happens when a new type of cyber-attack emerges, or a sensor's environment changes?

A recent paper by Joanna Komorniczak from Wrocław University of Science and Technology introduces ORCCA (Open-world Reconstruction-based Classifier with Concept Drift Adaptation), a method designed to thrive in these "open-world" environments.

TL;DR

ORCCA is a novel framework for data stream classification that uses an ensemble of autoencoders to simultaneously detect new classes and adapt to shifting data distributions (concept drift). By analyzing reconstruction errors instead of raw features, it achieves state-of-the-art performance in non-stationary environments while remaining computationally efficient.

What Problem Does This Paper Address?

Modern machine learning systems face two primary "non-stationary" challenges in data streams:

  1. Concept Drift: The underlying distribution of known classes changes over time (e.g., seasonal changes in user behavior).
  2. Novelty Detection: Entirely new, previously unseen categories of data appear (e.g., a new species in a wildlife camera trap).

Most existing methods tackle one or the other. "Closed-world" models fail when novelties appear, often misclassifying them as known classes with high confidence. Conversely, many novelty detectors struggle to adapt when the "known" classes themselves start to drift. ORCCA aims to bridge this gap, operating in a fully unsupervised manner after an initial short offline training phase.

Fig. 1. An example of a non-stationary data stream, where, over 100 chunks, four novel classes appear (beige markers). The known class distribution (blue and red markers) shifts due to concept drifts. The bottom plot shows the proportions of classes at a given timestamp.

Methods

The core innovation of ORCCA is shifting the analysis from high-dimensional feature space to a one-dimensional proxy: reconstruction error.

  • Per-Class Autoencoders: ORCCA maintains an ensemble where each known class has its own dedicated autoencoder.
  • Reconstruction Error & KDE: For every incoming sample, ORCCA calculates how well each autoencoder can "reconstruct" it. It uses Kernel Density Estimation (KDE) to model the distribution of these errors. If a sample's error is "typical" for a known class, it's classified; if it's an outlier for all models, it's flagged as unknown.
  • Unsupervised Adaptation: When a sample is recognized as a known class, ORCCA fine-tunes that class's autoencoder weights. This allows the model to "follow" the class as it drifts without needing new labels.
  • Novelty Grouping: Unknown samples are stored in a buffer. ORCCA uses Sarle’s bimodality coefficient to analyze the error distributions in the buffer. This helps the system decide if the unknowns represent a single new class or multiple distinct novelties that need their own new autoencoders.
Fig. 2. The normalized distributions of reconstruction errors of known classes (black bars) and novel-class samples (blue bars). The central column shows the responses of KDE (red color indicates high density score) and the right column the label known/unknown produced by ORCCA. Scatterplots use two principal components.

Datasets

The authors conducted an extensive evaluation using:

  • Synthetic Streams: 330 different data streams generated using the Madelon distribution, varying the number of drifts and novelties.
  • Semi-synthetic Streams: Embeddings from the CIFAR-10 dataset (extracted via a pre-trained ResNet18) arranged into streams to simulate realistic high-dimensional data.
  • Real-world Benchmarks:
  • Covertype: Predicting forest cover types (581k samples).
  • KDD Cup ’99: Network intrusion detection (494k samples).
  • PAMAP2: Physical activity monitoring (194k samples).

Key Insights and Ideas

  • The Power of 1D Proxies: One of the most surprising findings is that reducing complex, high-dimensional features to a simple reconstruction error score is sufficient for both classification and novelty detection. This significantly lowers the computational burden.
  • Two-Step Recognition is Safer: The paper demonstrates that using per-class thresholds for known classes before applying KDE for unknowns is more robust than relying on density estimation alone.
  • Self-Correction via Fine-Tuning: The unsupervised adaptation mechanism effectively "re-calibrates" the models. By trusting its own high-confidence predictions to update weights, ORCCA can maintain high accuracy even as data distributions shift significantly.

Key Results

  • Superior Novelty Detection: ORCCA consistently achieved the best mean rank in binary "known vs. unknown" classification across nearly all test scenarios.
  • Resilience to Drift: In streams with frequent concept drifts, competitors like MINAS saw performance drop below 30%, whereas ORCCA remained competitive by successfully adapting its internal models.
  • Scalable Resource Use: While memory usage grows as new classes are discovered (since each needs a new autoencoder), the processing time per sample remains stable. Even in extreme scenarios with 5 novelties and 4 drifts, ORCCA's processing time growth was only 17%, far lower than some ensemble-based competitors that saw over 200% growth.
Comparison of processing time and memory growth across different methods.

Limitations and Future Work

The authors identify a few areas for improvement:

  • Memory Footprint: Storing full neural network weights for every discovered class can become expensive in very long-running streams with hundreds of novelties. Future work might explore "knowledge distillation" to compress these models.
  • Sudden vs. Smooth Drift: The current adaptation assumes relatively smooth transitions. Sudden, violent shifts in data might still be misidentified as novelties rather than drifts.
  • Imbalance Sensitivity: On highly imbalanced real-world data (like the Covertype set), the model's ability to distinguish between rare known classes and novelties can degrade.

Overall, ORCCA represents a significant step toward autonomous machine learning systems that can "learn to learn" in the wild, adapting to change without constant human intervention.

Summary updated

Abstract

Data stream processing has become central to modern machine learning applications, with concept drifts and novel class appearances posing the primary challenges faced by recognition methods. Adapting to concept drifts while also being able to detect novel classes and distinguish groups of unknown observations can be seen as a step towards effective method employment in an open-world setting. This work proposes a classification method that can detect and learn the distributions of new, unknown classes, as well as adapt to concept drifts once the distribution of known classes shifts. The proposed method uses the autoencoders' reconstruction errors and the KDE that models their distribution. The autoencoders incrementally adapt to the changing distributions of known classes via weight fine-tuning, resulting in continuous adjustment to evolving concepts. Conducted experiments used a diverse set of synthetic data streams, embeddings of a real-world static CIFAR dataset arranged into a stream to simulate concept drifts and the emergence of unknowns, as well as real-world datasets where novelty is observed. The methodology was compared with state-of-the-art approaches with three-criterion recognition quality assessment and resource consumption evaluation. The results indicate that the proposed method is competitive with reference approaches in stationary environments and performs best when concept drifts occur.
BibTeX
@misc{komorniczak2026open,
  author = {Joanna Komorniczak},
  title = {{Open-world Reconstruction-based Classifier with Concept Drift Adaptation}},
  year = {2026},
  eprint = {2611.09350},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.09350},
}
Figure