Temporal Label Denoising for Video Activity Localization

This direction is split into other directions: temporal label noise. Its last version stays readable and citable here.

Version 1 of 1124 papers154 conceptsfrozen 2026-10-04

In an ideal world, every video in a training set would come with frame-perfect timestamps. A label like "jumping" would begin exactly as the feet leave the ground and end the millisecond they land. In reality, video datasets are messy. Labels are often "weak" (assigned to an entire five-minute clip even if the action lasts five seconds), "fuzzy" (varying between different human annotators), or "hallucinated" (generated by AI models that miss the mark).

The research direction of Temporal Label Denoising focuses on bridging the gap between these imprecise, noisy annotations and the precise temporal boundaries required for high-quality video understanding.

The Video-Level Label Fallacy

The most common source of noise is the assumption that a global label applies to every frame. When we scrape YouTube for "soccer goals," we get clips where the actual goal occupies only 5% of the duration. If a model is trained to associate every frame of that clip with the "goal" label, it inadvertently learns that a green field or a cheering crowd is the goal.

Kumara Kahatapitiya and Zhou Ren highlight this "task disparity." They argue that assigning a single video-level label to all constituent frames introduces significant noise, as many frames contain preparatory or transitional states rather than the action itself. Similarly, in medical imaging, Gautam Rajendrakumar Gare and colleagues found that labeling an entire lung ultrasound as "diseased" is problematic because the sensor might pass over healthy tissue during the scan, leading the model to learn incorrect features.

Modeling Boundary Uncertainty

Even when humans try to provide exact timestamps, they rarely agree. Is the start of a "handshake" when the hands first move or when they first touch? Binjie Zhang and Yu Li note that standard loss functions fail to account for this inherent ambiguity, which leads to poor generalization.

A key intuition in this field is to stop treating boundaries as fixed points and start treating them as distributions. In Boundary Uncertainty in a Single-Stage Temporal Action Localization Network, researchers propose modeling start and end times as Gaussian distributions. By predicting both a mean and a variance, the network can express "uncertainty" about a boundary, effectively down-weighting noisy labels during training.

Mengmeng Xu takes this further with DenoiseLoc, which treats localization as a generative denoising problem. By intentionally "jittering" ground-truth boundaries and teaching the model to "fix" them, the system learns the underlying structure of an action rather than just memorizing inconsistent timestamps.

Refining Pseudo-Labels via Cross-Modal Guidance

When labels are missing or too coarse, researchers often generate "pseudo-labels"—best-guess annotations created by a teacher model. However, these are notoriously noisy. To clean them, recent work leverages the "semantic bridge" of language.

Yingying Fan and Yu Wu introduced Language-guided Segment-level Label Denoising (LSLD). Their intuition is that while visual boundaries are blurry, the linguistic description of an event (via models like CLIP) can act as a filter. If a video-level label says "dog barking," but the audio-visual features of a specific segment don't align with the linguistic concept of "barking," the segment-level label is discarded as noise.

Similarly, Kunpeng Li and Yun Fu developed Sub-Pseudo Labels (SPL). Instead of discarding "noisy" clips where the visual content doesn't perfectly match the search tag, they create "middle-ground" concepts. This allows the model to learn from the "noise" by acknowledging that a clip might contain a sub-concept or a related action rather than a perfect instance of the target class.

Adversarial Background Mining

A final pillar of this research is distinguishing the "action" from the "context." In weakly supervised settings, models often struggle with "class-specific context"—for example, always seeing a swimming pool when someone is "diving."

Yongxin Ge and Jiaruo Yu proposed an adversarial strategy that forces the model to treat the entire untrimmed video as background. This creates a "tug-of-war" that forces the model to find the most discriminative features, effectively denoising the temporal signal by suppressing the static context that plagues noisy, video-level annotations.

By treating temporal noise not as a flaw to be ignored, but as a signal to be modeled, these researchers are enabling AI to learn from the vast, uncurated, and "fuzzy" reality of web-scale video.

Papers in this version

Showing 60 of 124 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_temporal_label_denoising_for_video_activ_afaa2034,
  title = {Temporal Label Denoising for Video Activity Localization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-label-denoising-for-video-activity-localization/rdr_afaa2034e4ad446888bec53498783591},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_temporal_label_denoising_for_video_activ_afaa2034_v1,
  title = {Temporal Label Denoising for Video Activity Localization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-label-denoising-for-video-activity-localization/rdr_afaa2034e4ad446888bec53498783591/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}