Emotion recognizers are trained in one place and used in another. How often people show fear, disgust or anger, and how often annotators call an expression "neutral," changes sharply across countries. In one cross-cultural study, German annotators labeled 87% of the same scenarios as neutral, while Mexican and Ethiopian annotators often chose anger or guilt (CULEMO). Practitioners handle the rarity of negative emotions with focal loss, a training objective that down-weights examples the model already gets right so that rare classes drive learning (Focal Loss for Dense Object Detection). Focal loss has a side effect: the model's output scores are no longer the true class probabilities. That matters in a new region. The standard way to adapt a classifier to a new region's class frequencies without labels is to re-estimate those frequencies from unlabeled data and reweight the outputs, and that method assumes the outputs are true probabilities.
The proposal is to keep focal loss for training and convert its outputs back into probabilities before deployment. The conversion has a closed form that follows from the loss itself. With proper probabilities in hand, we then estimate each region's emotion frequencies from unlabeled data and adjust predictions per region, with no target labels and no retraining. A second goal is diagnostic. The method fixes errors caused by shifted class frequencies only. Whatever error remains after correction therefore measures how much a region differs in how emotions are labeled, as opposed to how often they occur. Prior work treats these two kinds of shift as one problem.
Core Claim
Our hypothesis has three parts.
- Applying unsupervised frequency correction directly to focal-loss outputs is biased. Focal loss pulls confidences toward the middle, so the estimated frequency shift should come out too small.
- Applying the inverse mapping first removes most of that bias.
- The combination of focal training, the inverse mapping and regional frequency correction beats both halves alone on tail-class recall and per-region calibration. The halves are cross-entropy training plus correction, and focal training without correction.
Deriving the inverse mapping. For a model output $p$ on the true class, the per-sample focal loss is $\mathrm{FL}(p) = -(1-p)^\gamma \log p$, where $\gamma \ge 0$ controls how strongly easy examples are down-weighted. Write $h(p) = (1-p)^\gamma/p - \gamma (1-p)^{\gamma-1}\log p$ for the negative derivative of $\mathrm{FL}$.
Let $q_k$ be the true probability of class $k$ given the input, and $p_k$ the model output for class $k$. Minimizing the expected loss $\sum_k q_k \mathrm{FL}(p_k)$ over the probability simplex gives the condition $q_k h(p_k) = \lambda$ for every class, for some constant $\lambda$. Solving for the true probabilities:
$$\hat q_k(x) = \frac{1/h(p_k(x))}{\sum_j 1/h(p_j(x))},$$
When $\gamma = 0$ this reduces to $\hat q_k = p_k$, the familiar cross-entropy case. For the class-weighted variant with weights $\alpha_k$, $h$ is replaced by $\alpha_k h$.
The derivation holds only at the population optimum. Whether it survives finite data, regularization and label noise is the empirical question. Recent theory asks whether focal outputs can be mapped back to class probabilities at all (On Focal Loss for Class-Posterior Probability Estimation; Functional Properties of the Focal-Entropy). We use that question as the basis for a deployment procedure and test it under geographic shift.
Estimating regional frequencies. Given $\hat q(x)$ and the source class frequencies $\pi$, the target frequencies $\pi'$ are found by the standard expectation–maximization fixed point on the unlabeled target-region pool. Each sample's probabilities are reweighted by $\pi'_k/\pi_k$ and renormalized, and $\pi'$ is set to the average reweighted probability. The loop repeats until $\pi'$ stops changing.
Relation to Prior Work
Motivating work. CULEMO documents large culture-specific emotion distributions but proposes no correction. A Deeper Look at Facial Expression Dataset Bias separates two kinds of dataset bias. Capture bias comes from differences in cameras, settings and demographics. Category bias comes from annotators who disagree on emotion boundaries. That paper shows a classifier can identify a face's source dataset with 79% accuracy.
Closest competitor. Its method, ECAN, aligns source and target features and learns a class reweighting toward the target. ECAN differs from our approach in four ways:
- It retrains per target domain.
- It uses cross-entropy.
- It needs unlabeled target data during training.
- It never separates frequency shift from labeling shift.
Our approach is post-hoc, runs at deployment, and produces an explicit estimate of each region's frequencies.
Focal loss under imbalance or shift. Three lines of work use focal-style losses under imbalance or shift, and all treat the class distribution as fixed or learned once:
- MultiEMO adds focal weighting to a contrastive loss on MELD and lifts Fear F1 from about 11.7% to 29.67%. Its class weights are static.
- Multitask Learning for Emotionally Analyzing Sexual Abuse Disclosures uses class-balanced focal loss on English tweets only. - Threshold-adaptive Unsupervised Focal Loss keeps running per-class confidence thresholds, but for a single synthetic-to-real target.
None of them returns probabilities usable for regional reweighting.
Constraining work. AdaFocal shows that a single $\gamma$ leaves some confidence ranges over-confident and others under-confident. It needs a validation set drawn from the same distribution as deployment, which a held-out region lacks. Our inverse mapping needs no target validation data, but AdaFocal remains a strong baseline when run on source-region validation data.
Evaluation design. The geospatial literature fixes how evaluation must be done. Random splits overstate accuracy, and geographically held-out sets are mandatory (Position: Mission Critical; Deploying an AI Application to Detect Flood). It also shows that class make-up varies across space (STAR).
Research Program
The work separates three regimes. Each test bed is chosen so that one kind of shift dominates.
Pure frequency shift with objective labels. We use Sen1Floods11 flood mapping and hold out entire flood events. The share of flooded pixels varies by event, but water is labeled objectively. This test bed checks the mechanism without subjectivity confounding it.
Pure frequency shift with subjective labels. We train facial expression models on RAF-DB. We then resample the FERPlus and RAF-DB test sets to match the per-country emotion distributions reported in CuLEmo, keeping the emotion categories shared with the seven basic expressions and renormalizing. Because resampling leaves the appearance of each emotion unchanged, the true target frequencies are known exactly. Prior-estimation error can therefore be measured directly.
Pure labeling shift, and real mixtures. CuLEmo itself gives the same scenarios different labels by country. The inputs are identical while the labels shift, so no frequency correction should help much, and the diagnostic should flag this. Real cross-dataset transfer from RAF-DB to FERPlus, SFEW 2.0, CK+ and the multi-ethnic MMA set mixes both kinds of shift.
Backbones. On faces we use the frozen CLIP backbone with learned channel masks from CAFE. On floods we use Prithvi 2.0 adapted with LoRA, the parameter-efficient fine-tuning method that trains small low-rank weight updates and that Fine-tune Smarter, Not Harder found most robust geographically. The loss and the correction then sit on top of the strongest available representations rather than competing with them.
Sweeps and diagnostic. We sweep $\gamma \in \{0, 0.5, 1, 2, 3\}$, with and without class weights. For each region we report the gap between the corrected model and a model given the true target frequencies. We compare that gap with annotator disagreement, measured by FERPlus crowd label distributions and by CuLEmo's at least five annotators per country. If the residual reflects labeling shift, it should rise with disagreement.
Baselines
Naive reference. Cross-entropy training with no correction, evaluated on the held-out regions. This shows how large the problem is.
Strongest simple competitor. Cross-entropy training plus the same expectation–maximization frequency correction, with temperature scaling fit on spatially held-out source data. Cross-entropy outputs are already approximately calibrated, so this pipeline needs no inverse mapping. If it matches our method on tail recall, the focal ingredient is unnecessary. It will use the same backbone, data and tuning budget, with the same number of configurations as our $\gamma$ sweep.
Strongest focal methods. - Fixed-$\gamma$ focal loss with no correction. - AdaFocal, with its validation bins drawn from source regions, followed by correction. AdaFocal reports 1.36% expected calibration error on CIFAR-100 with ResNet-50. No reported result was found on Lacuna for AdaFocal under geographic or cross-dataset shift.
Strongest adaptation method. ECAN receives the same unlabeled target pools our correction sees. It reports 86.49% accuracy for RAF-DB 2.0 to CK+ and 54.34% for RAF-DB 2.0 to SFEW 2.0.
Strongest zero-shot facial baseline. CAFE reports a 63.48% mean zero-shot accuracy across target datasets when trained on RAF-DB, including 73.16% on FERPlus.
Strongest flood baseline. LoRA-adapted Prithvi 2.0 reports 87.57% mIoU on the Sen1Floods11 geographic hold-out set, against 82.07% for full fine-tuning (Fine-tune Smarter, Not Harder).
Key ablation. Focal training plus correction applied to the raw focal outputs, skipping the inverse mapping. This isolates the contribution of the inverse mapping.
What counts as beating these baselines. Our method must improve macro-F1 and Fear/Disgust recall over cross-entropy plus correction on the resampled and cross-dataset regions. It must not raise per-region calibration error. It must match or exceed ECAN without per-target retraining.
Why not just…?
Why not just train with cross-entropy and apply frequency correction? This is the real threat. Cross-entropy gives usable probabilities for free. The open question is whether focal training learns better tail features, enough to survive the conversion back to probabilities. The ablation grid answers this directly: if cross-entropy plus correction equals or beats our method, we report that focal loss is unnecessary for regional deployment. The evidence that focal-style weighting improves tail features on emotion data, such as Fear F1 roughly 11.7% to 29.67% on MELD in MultiEMO, is suggestive but comes bundled with other changes.
Why not just recalibrate focal outputs with temperature scaling? A single temperature rescales all outputs uniformly. The focal distortion is a nonlinear, class-wise function of each output, and AdaFocal's finding that different confidence ranges err in opposite directions suggests one parameter cannot undo it. We include "focal plus temperature scaling plus correction" as a baseline. If it closes the gap, the closed-form inverse adds nothing.
Why not just use domain adaptation such as ECAN, or a stronger foundation model? ECAN must be retrained for every new region and still conflates frequency shift with labeling shift. Stronger backbones address representation, not frequencies: CuLEmo shows large language models misjudge cultural emotion norms without explicit context. We run on top of CAFE and LoRA-adapted Prithvi, so any gain is additional to theirs.
Decisive Checks
Mechanism check. On the resampled facial sets with known frequencies, the estimate from inverted outputs should land closer to the true frequencies (measured by L1 distance) than the estimate from raw focal outputs. The raw estimate should show the predicted shrinkage, growing with $\gamma$.
Control check. On Sen1Floods11, the same ordering of methods should hold in mIoU and per-event calibration. Otherwise the effect is specific to faces.
Negative-control check. On CuLEmo, where the inputs are shared and only the labels shift, correction should give little gain, and the residual should be large.
Diagnostic check. On real cross-dataset transfer, the residual left after correction should correlate positively with annotator disagreement across target datasets and classes. That would give practitioners a label-free way to tell whether a new region needs better data collection or only reweighting.
References
- CULEMO: Cultural Lenses on Emotion - Benchmarking LLMs for Cross-Cultural Emotion Understanding
- Focal Loss for Dense Object Detection
- On Focal Loss for Class-Posterior Probability Estimation: A Theoretical Perspective
- Functional Properties of the Focal-Entropy
- A Deeper Look at Facial Expression Dataset Bias
- MultiEMO: An Attention-Based Correlation-Aware Multimodal Fusion Framework for Emotion Recognition in Conversations
- Multitask Learning for Emotionally Analyzing Sexual Abuse Disclosures
- Threshold-adaptive Unsupervised Focal Loss for Domain Adaptation of Semantic Segmentation
- AdaFocal: Calibration-aware Adaptive Focal Loss
- Position: Mission Critical – Satellite Data is a Distinct Modality in Machine Learning
- Deploying an Artificial Intelligence Application to Detect Flood from Sentinel 1 Data
- Statistically-Guided Deep Network Transformation to Harness Heterogeneity in Space (Extended Abstract)
- Generalizable Facial Expression Recognition
- Fine-Tune Smarter, Not Harder: Parameter-Efficient Fine-Tuning for Geospatial Foundation Models