Learning with Labeled Sessions

Rong Jin, Huan Liu

Introduction

In traditional supervised learning, models are trained on individual instances, each associated with a ground-truth label. However, many real-world applications provide labels only for "sessions" or "objects"—collections of multiple instances where the overall label is determined by a majority vote. For example, in physiological monitoring, a 30-minute window might be labeled "Watching TV," even if the user briefly engages in other activities like getting a snack. Similarly, in speaker identification, a long audio clip is assigned a speaker label based on the majority of its constituent segments.

The primary challenge in these scenarios is label ambiguity. Naively assigning the session label to every constituent instance introduces significant noise, as some instances may not actually correspond to that label. This noise often leads standard classifiers to overfit or perform poorly.

Session-Based Learning vs. Multiple-Instance Learning

The authors define session-based learning as a distinct paradigm from Multiple-Instance Learning (MIL). While MIL considers a "bag" positive if at least one instance is positive, session-based learning relies on a majority rule. This shift fundamentally alters the optimization landscape, making the model more robust to single outliers but requiring more sophisticated methods to identify the "correct" subset of instances.

Comparison of traditional supervised learning, multiple-instance learning (MIL), and session-based learning.

The SBoost Algorithm

To address label ambiguity, the researchers proposed SBoost (Session-based Boosting). SBoost is an iterative boosting framework that incorporates several key innovations:

  • Dual-Error Loss Function: Unlike standard boosting algorithms that focus solely on instance-level accuracy, SBoost optimizes a loss function that balances instance-based and session-based errors.
  • Adaptive Instance Weighting: SBoost maintains weights for each instance. Instances that consistently disagree with the session's majority label are assigned lower weights, effectively filtering out noise during the training process.
  • Iterative Optimization: The algorithm trains a sequence of weak classifiers (e.g., small decision trees). In each iteration, instance weights are updated based on their contribution to the correct session-level classification.

A critical hyperparameter, $\gamma$, allows the model to balance the importance of individual point accuracy against the broader session-level goal.

Experimental Evaluation

The performance of SBoost was evaluated on two types of datasets:

  1. Synthesized UCI Datasets: Standard binary datasets (Spam, CMC, German) were grouped into sessions of 10, with minority instances from the opposing class mixed in to simulate real-world ambiguity.
  2. Physiological Data: Sensor data from the ICML 2004 workshop, where the goal was to predict user activity sessions despite internal behavioral variations.

Results and Discussion

SBoost consistently outperformed both standalone Decision Trees and the "Naive" AdaBoost approach (which treats every instance as having the session label). For the Spam dataset, SBoost achieved a classification error of 0.189, compared to 0.258 for AdaBoost and 0.232 for a single Decision Tree.

Classification error of SBoost vs. AdaBoost over iterations on physiological data.

As shown in the figure above, SBoost demonstrates superior overfitting resistance. While AdaBoost’s error rate often begins to increase after several iterations due to fitting the noise in ambiguous sessions, SBoost’s error rate continues to decline and eventually stabilizes. This confirms that SBoost effectively identifies and down-weights noisy instances that do not align with the majority session label.

Conclusion

SBoost provides a robust framework for learning from session-labeled data by explicitly handling label ambiguity through a dual-error loss function and adaptive weighting. While currently focused on binary classification, future work could extend this approach to multi-class problems and explore automated methods for hyperparameter tuning.

Summary updated

Abstract

Traditional supervised learning deals with labeled instances. In many applications such as physiological data modeling and speaker identification, however, training examples are often labeled objects and each of the labeled objects consists of multiple unlabeled instances. When classifying a new object, its class is determined by the majority of its instance classes. As a consequence of this decision rule, one challenge to learning with labeled objects (or sessions) is to determine during training which subset of the instances inside an object should belong to the class of the object. We call this type of learning 'session-based learning' to distinguish it from the traditional supervised learning. In this paper, we introduce session-based learning problems, give a formal description of session-based learning in the context of related work, and propose an approach that is particularly designed for session-based learning. Empirical studies with UCI datasets and real-world data show that the proposed approach is effective for session-based learning.
BibTeX
@inproceedings{jin2005learning,
  author = {Rong Jin and Huan Liu},
  title = {{Learning with Labeled Sessions}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2005},
}
Figure