Learning Fine-Grained Tasks from Coarse Labels

23 concepts20 papers1 proposal

In machine learning, the cost of data is often a function of its precision. Labeling an image as a "bird" is fast and requires little expertise, but identifying it as a "Yellow-bellied Sapsucker" requires an ornithologist. Similarly, marking a whole document as "positive" is trivial compared to identifying every specific sentence that contributes to that sentiment.

This creates a bottleneck: we have mountains of "coarse" data but only molehills of "fine-grained" data. The research direction of coarse-to-fine supervision explores how we can bridge this gap, using high-level, inexpensive labels to guide models toward expert-level precision.

The Intuition of Hierarchical Constraints

Human knowledge is rarely flat. We understand the world through taxonomies: a Golden Retriever is a Dog, which is a Mammal, which is an Animal. Coarse-to-fine supervision leverages this structure by treating coarse labels as a "search space reduction" for the model.

Jain et al. demonstrate this through a "sanity check" mechanism. By using a coarse classifier to amendment fine-grained predictions at test time, they ensure that even if a model misclassifies a specific species, it stays within the correct high-level category. This hierarchical anchoring prevents the model from making "nonsensical" errors, such as confusing a bird for a butterfly.

Similarly, Gall et al. show that integrating coarse-labeled data (e.g., "feline") allows models to achieve nearly 99% of the performance of fully expert-labeled systems while using only half the fine-grained annotations. The coarse labels provide a reliable foundation that stabilizes the learning of more difficult, low-level distinctions.

The Efficiency of "Good Enough" Data

A recurring theme in this research is that volume can often compensate for a lack of precision. In tasks like semantic segmentation, where "pixel-perfect" masks are prohibitively expensive, researchers have found that a two-tier strategy is highly effective.

Zlateski argues that training on a large volume of coarse, low-cost labels followed by fine-tuning on a tiny fraction of high-quality labels achieves performance parity with models trained entirely on expensive datasets. This is echoed by Cordts et al. in the context of urban scene understanding, where a massive set of rapidly produced coarse annotations helps the model learn the underlying structure of a scene before it ever sees a precise label.

Surprisingly, even "seemingly uninformative" labels can help. Matsoukas et al. found that adding coarse labels for background objects in medical imaging—like skin or muscle in a mammogram—boosts performance on the primary task (tumor detection) by 57%. The categorical separation of background features provides critical guidance that is more important than the exact spatial precision of the boundaries.

Decomposing Aggregate Signals

In natural language processing, the challenge is often to infer fine-grained attributes from document-level signals. This is frequently framed as a latent variable problem: if we know a review is "negative," can we identify which specific sentences (aspects) drive that rating?

Fang et al. address this by treating sentence-level aspect selection as a hidden variable within a structural learning framework. By using the document-level label as supervision, the model learns to categorize sentences into "Price" or "Service" without needing manual sentence-level tags.

This "top-down" supervision is also effective in complex social tasks. Chen et al. developed a model to identify persuasion strategies in online requests by leveraging document-level "success" labels (whether a request was granted). They found that these indirect, coarse signals capture nuances that direct, fine-grained supervision might miss. Similarly, Li et al. show that knowing the set of emotions in a conversation is sufficient for a model to learn to map specific emotions to individual utterances, reducing human labeling time by up to 90%.

Dataless Learning via Taxonomies

The most extreme version of this idea removes manual labeling entirely, relying instead on the inherent semantic information in category names and their hierarchical relationships.

Meng et al. and Shen et al. have proposed frameworks that perform hierarchical classification using only class names as supervision. By embedding the hierarchy and the text into a shared space, these models can identify "core classes" and retrieve representative terms for each category. These weakly-supervised methods can match the performance of models trained on thousands of labeled examples, suggesting that the structure of human knowledge itself—encoded in taxonomies—is a powerful supervisory signal.

Key Foundational Work

From categories to subcategories: Large-scale image classification with partial class label refinement (2015) — contributed 1 concept elements to this direction.
Fine Granular Aspect Analysis using Latent Structural Models (2012) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_coarse_to_fine_supervision_8c619780,
  title = {coarse-to-fine supervision},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/coarse-to-fine-supervision/rdr_8c619780dc494a1e8cb6cb8a75ab4cd9},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_coarse_to_fine_supervision_8c619780_v1,
  title = {coarse-to-fine supervision},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/coarse-to-fine-supervision/rdr_8c619780dc494a1e8cb6cb8a75ab4cd9/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake