Learning Precise Temporal Point Event Detection with Misaligned Labels

Julien Schroeter, Kirill A. Sidorov, A. David MarshallAAAI 2021 · DOI

Temporal point event detection involves identifying the exact timestamps of instantaneous events within time-series data, such as drum hits or medical signals. A significant challenge in this field is the prevalence of misaligned training labels due to human error or sensor latency. Traditional approaches like label smoothing often result in imprecise, "blurry" predictions. This paper introduces $L_{SoftLoc}$, a robust loss function that combines soft localization learning with a Poisson-Binomial counting constraint. This framework allows models to learn effectively from noisy labels while producing sharp, precise detections during inference.

The Challenge of Misaligned Labels

In many real-world applications, detecting the precise moment an event occurs is critical. However, obtaining perfectly aligned ground truth labels is often impractical. Annotators may mark events slightly early or late, leading to temporal misalignment in training data. Standard deep learning models trained on such data using Stepwise Cross-Entropy (SCE) often fail to generalize well, as they are penalized for predicting the true event location if it deviates from the noisy label.

Figure 1: Training on misaligned labels often leads to imprecise predictions. The goal is to learn from noisy data while maintaining sharp inference.

Limitations of Standard Label Smoothing

A common workaround for misaligned labels is label smoothing, which spreads the label's probability over a small window. While this aids convergence, it introduces three fundamental problems:

  1. Ambiguity: The model learns to output a probability "blob" rather than a point, necessitating complex post-processing like peak-picking.
  2. Entanglement: Closely spaced events may have their smoothed labels merged, making them indistinguishable.
  3. Temporal Cross-Influence: Smoothing forces causal models to rely on future information to predict the "tail" of an event, which is problematic for real-time systems.

The $L_{SoftLoc}$ Framework

To address these issues, the authors propose $L_{SoftLoc}$, which integrates two key components into a single loss function:

  • Soft Localization Learning ($L_{SLL}$): This component applies smoothing to both the ground truth labels and the model's predictions during the loss calculation. This symmetric approach makes the loss invariant to small temporal shifts without requiring the model to output blurry predictions during inference.
  • Counting-based Sparsity Constraint ($L_{PB}$): Using the Poisson-Binomial distribution, this constraint ensures that the total number of predicted events in a window matches the number of labels. This mathematically encourages the model to produce a single high-probability "spike" for each event, effectively enforcing sparsity.
Figure 5: Comparison of standard label smoothing issues (left) and the solutions provided by the proposed $L_{SoftLoc}$ method (right).

Experimental Evaluation

The framework was tested on four diverse domains with varying noise levels: Golf Swing Sequencing, Smoking Puff Detection, Piano Onset Detection, and Drum Detection.

In the Smoking Puff Detection task, $L_{SoftLoc}$ achieved an $F_1$ score of 79.0 in high-noise environments ($\sigma=4$), whereas standard Cross-Entropy performance plummeted to 25.8. Similarly, in Piano Onset Detection, the model remained robust even when only 26% of training labels were within the tolerance window, maintaining an $F_1$ score above 73% even under extreme noise.

Conclusion

$L_{SoftLoc}$ provides a powerful solution for temporal point event detection in the presence of misaligned labels. By decoupling the learning of temporal location from the requirement of sharp output through symmetric smoothing and counting constraints, the method achieves state-of-the-art precision and robustness. This approach is particularly valuable for real-time, causal systems where temporal accuracy is paramount.

Summary updated

BibTeX
@inproceedings{schroeter2021learning,
  author = {Julien Schroeter and Kirill A. Sidorov and A. David Marshall},
  title = {{Learning Precise Temporal Point Event Detection with Misaligned Labels}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i11.17145},
}
Figure