Traditional classification tasks typically rely on "logical labels" (e.g., binary 0 or 1), which fail to capture the inherent ambiguity and spectrum-like nature of real-world data. For instance, a 25-year-old individual may share visual characteristics with both 24-year-olds and 26-year-olds. Label Distribution Learning (LDL) addresses this by assigning a "description degree" to every possible label, forming a probability-like distribution. However, most datasets only provide logical labels. While Label Enhancement (LE) techniques aim to recover these distributions, they often overlook domain-specific prior knowledge—such as the fact that neighboring ages should have similar description degrees.
The RLLE Framework
To bridge this gap, the authors propose RLLE (Reinforcement Learning based method for Label Enhancement). This framework treats the conversion of a single logical label into a rich distribution as a dynamic, sequential decision process.
The RL agent starts with a "blank" distribution and iteratively adjusts the description degrees. The state representation includes both the instance's features (e.g., image pixels) and the current label distribution. At each step, the agent selects an action from a space consisting of bigger, smaller, or constant for each label. To manage the complexity of simultaneous adjustments across multiple labels, the authors developed Multi-dueling Q-learning (MQ-learning), which enables the network to output a set of actions for all labels in a single step.
Prior Knowledge Integration
A core innovation of RLLE is using domain-specific prior knowledge to define the terminal state and reward signal, rather than relying on ground-truth distributions.
For Age Estimation, the prior knowledge dictates that the chronological age should have the highest description degree, with degrees decreasing monotonically as the distance from the target age increases. For Emotion Classification, the model leverages psychological frameworks like Plutchik’s wheel, where related emotions (e.g., "awe" and "excitement") are positioned closely, while contrasting ones (e.g., "anger" and "sadness") are distant. This logic allows the agent to "unfold" a single label into a natural distribution without requiring human-annotated distributions for training.
Experimental Results
The RLLE framework was evaluated on four benchmarks: FG-NET and MORPH for age estimation, and Flickr_LDL and Twitter_LDL for emotion recognition.
In age estimation tasks, RLLE consistently outperformed state-of-the-art methods like IIS, BFGS, and CPNN. Notably, on the large-scale MORPH dataset, RLLE combined with the BFGS algorithm achieved a Mean Absolute Error (MAE) of 3.94, a significant improvement over the standard Gaussian distribution baseline of 4.78. In emotion recognition, RLLE surpassed the "vote strategy"—the gold standard of human annotation—in accuracy. On Flickr_LDL, RLLE ranked first in 66.6% of cases across various metrics. Furthermore, the multi-dueling architecture proved highly efficient, typically reaching the terminal distribution within just five episodes.
Conclusion
RLLE demonstrates that sequential adjustment guided by prior knowledge is a powerful paradigm for label enhancement. By decoupling the process from the need for extra topological data or expensive human voting, it provides a scalable solution for LDL. While the method's performance is currently tied to the availability of well-defined prior knowledge models, it opens new avenues for applying RL to data preprocessing and label refinement in complex, ambiguous domains.
Summary updated