Shortfall and Surplus: Asymmetric Aggregation of Signed Fuzzy Distances, Tested at Recommender Scale

Many ways of comparing fuzzy quantities, such as a hotel's spread of cleanliness ratings against what a particular traveller expects, discard the direction of the difference. They report that two things are "0.4 apart" without saying which one is ahead. A small line of work recovers direction with signed fuzzy distances: a positive value means one set lies to the right of the other, a negative value means it lies to the left. No one has said how to combine many signed distances, one per decision criterion, into a single judgment.

The usual tunable combiners are power means, a family whose exponent slides the behaviour from "the weakest criterion dominates" to "the strongest criterion dominates." They are only defined for non-negative inputs. As a result, the field either throws the sign away or averages it linearly. We propose an aggregation operator that keeps the sign and treats it asymmetrically. Shortfalls (criteria where an option falls below the reference) and surpluses (criteria where it exceeds it) are pooled by separate power means with separate, learnable exponents. We then test it, along with the discrimination-oriented fuzzy measures it competes with, on multi-criteria recommendation datasets with millions of ratings, not on the five-to-ten-alternative case studies where such measures are normally shown.

Core Claim

Fuzzy decision-making papers usually fix a weighted average and evaluate on a toy example. We claim that two of their assumptions are wrong in a way real data can detect. The first is that positive and negative evidence across criteria should be combined symmetrically. The second is that one fixed linear rule suffices.

Concretely, let $d_1,\dots,d_n$ be signed directional distances between an item and a user-specific reference on each of $n$ criteria, with positive values meaning surplus. Write $d_i^{+}=\max(d_i,0)$ and $d_i^{-}=\max(-d_i,0)$. The score is

$$S_{p,q,\lambda}(d)=\Big(\sum_i w_i (d_i^{+})^{p}\Big)^{1/p}-\lambda\Big(\sum_i w_i (d_i^{-})^{q}\Big)^{1/q},$$

where $w_i$ are criterion weights summing to one, $p,q\ge 1$ are the surplus and shortfall exponents, and $\lambda>0$ weighs shortfall against surplus. Large $q$ makes the worst shortfall dominate, which is conjunctive or veto-like. Small $p$ makes surpluses count only when they are broad. Setting $p=q$ and $\lambda=1$ recovers a symmetric, sign-respecting operator, which serves as our symmetric ablation.

The empirical claim is threefold. On held-out preferences, the fitted $q$ will be reliably larger than $p$, and $\lambda$ reliably larger than one. This asymmetry will improve ranking over symmetric and linear aggregation. Finally, the gain should be largest for users with little history, where a three-parameter operator can be fit and a flexible learner cannot.

Where This Sits in Prior Work

Directional fuzzy distance supplies the input. Measuring the directional distance between fuzzy sets drops the absolute value from the modified Hausdorff distance on interval endpoints, integrates it over horizontal α-cuts, and extends it to non-convex and non-normal sets. A Fuzzy Directional Distance Measure instead treats fuzzy sets as mass assignments over crisp sets, yielding a distance that is itself a fuzzy set over signed values. Both combine pieces by plain averaging and validate on synthetic sets and qualitative MovieLens 100k examples. We reuse their distance unchanged and replace the averaging step across criteria.

Analysing Fuzzy Sets Through Combining Measures of Similarity and Distance is the closest existing combination. It fuses a signed distance with one minus a similarity using an ordered weighted average with hand-set weights {0.7, 0.3}. That work fuses two measures for a single pair of sets with a fixed rule. We aggregate many signed criterion distances with a learned, sign-asymmetric rule, and we evaluate the result quantitatively.

Parameterized aggregation supplies the operator family and also constrains it. FALCON: Feedback Adaptive Loop for Content-Based Retrieval uses a power mean of distances to query exemplars, finding α = −5 best for disjunctive queries. Power mean pyramid scores for summarization evaluation shows that tuning the power-mean exponent beats linear pooling in agreement with human judgments. Both apply the power mean only to non-negative quantities, where a negative input to a fractional power is undefined. Splitting by sign is the minimal fix that keeps their tunability. Richer dependency-modelling operators, such as Heronian means, Muirhead means and Choquet integrals, model interactions between criteria but likewise assume unsigned inputs. We deliberately do not adopt them, because parameter count and interpretability are exactly what the low-data regime punishes.

Alternative discrimination supplies the competitors. Analysis on the Hesitation and its Application to Decision Making maps intuitionistic fuzzy values into the Poincaré half-plane to obtain a strictly monotone score that never ties. Novel Distance Measures of q-Rung Orthopair Fuzzy Sets and Their Applications builds Jensen–Shannon distances that include hesitancy. Both output unsigned scalars with equal treatment of components, and both are validated on single small cases: a 10-city site selection reporting ρ = 0.803 agreement with other methods, and a house-selection example. We do not dispute their theory. We test whether the ties they prevent actually occur often on real data, and whether preventing them matters for ranking more than knowing the direction of failure.

The empirical-validation direction supplies the evaluation setting. Multi-criteria recommendation datasets are already standard in graph-based recommendation, as in Criteria-Aware Graph Filtering and Criteria Tell You More than Ratings. The fuzzy measures above have never been run on them.

Research Program

The first component is a conversion protocol from multi-criteria rating data to fuzzy decision instances. For each item and criterion, the histogram of ratings (1–5 or the dataset's scale) becomes a membership function. We test two normalizations. Dividing by the maximum count gives normal sets. Dividing by the total count gives non-normal sets, which exercises the non-normal extension of the signed Hausdorff distance. For each user and criterion, a reference set is built from that user's criterion ratings on items they rated highly overall in the training split. The signed distance between item set and reference set gives one $d_i$ per criterion. For the intuitionistic and orthopair competitors, the same histograms are mapped to membership as the mass above the reference median and non-membership as the mass below it, with hesitancy as the remainder. This puts every method on identical information. The protocol, splits and code will be released, so fuzzy-measure papers have a shared benchmark with significance testing, not a single illustrative matrix.

The second component is fitting the operator. The parameters $p$, $q$ and $\lambda$ are fit by maximizing a pairwise ranking likelihood on training interactions, using a grid followed by local refinement, since there are only three continuous parameters. We fit at three granularities: global, per dataset, and per user with a global prior through shrinkage. Criterion weights $w_i$ are either uniform or fit globally. The per-user fits are what make the cold-user hypothesis testable.

The third component is a diagnostic study of discrimination. On real data, we measure how often linear scores, the Poincaré score, the Jensen–Shannon distances, unsigned power means and our operator produce exact or near ties (within 1e-6) among a user's top-50 candidates. We also measure how often two items tie in unsigned magnitude while failing in opposite directions on different criteria. If such opposite-direction ties are rare, the motivation for directional aggregation is weak, and we will report that.

Baselines

The naive reference point is ranking by an item's mean overall rating. A second naive point is ranking by the unweighted arithmetic mean of signed per-criterion distances. Together they show what any aggregation must beat. The fixed-rule fuzzy baselines run on the same converted data. These are the {0.7, 0.3} combined measure of Analysing Fuzzy Sets Through Combining Measures of Similarity and Distance, the Poincaré score of Analysis on the Hesitation, and the 2D and 3D Jensen–Shannon distances of Novel Distance Measures of q-Rung Orthopair Fuzzy Sets. For each of these, criterion scores are pooled by a weighted average, as in the originals. The tunable unsigned baseline is a FALCON-style power mean of absolute distances, with its exponent fit on the same validation split and grid size as ours. Beating it isolates the value of the sign as such.

The strongest learned competitor on the question "does structure beat flexibility" is gradient-boosted trees, or a logistic ranking model, over the same signed per-criterion features. These models can in principle learn asymmetry themselves. They are given the same features, the same validation split, and a tuning budget matched in configurations evaluated.

The strongest existing recommender is CA-GF from Criteria-Aware Graph Filtering. It is training-free and fast, and it is reported to gain up to 24% in NDCG@5 over the best prior multi-criteria methods on TripAdvisor. It ran in 0.2 seconds on BeerAdvocate, against about 10,020 seconds for CPA-LGC from Criteria Tell You More than Ratings. CPA-LGC is itself reported to beat non-graph multi-criteria methods by up to 141.20% in Precision@5. The file gives only relative gains. No reported absolute NDCG@5 or Precision@5 value for CA-GF or CPA-LGC was found on Lacuna, so we will rerun both from their released configurations on our splits.

We do not expect a three-parameter operator alone to beat CA-GF, which exploits collaborative signal we ignore. The fair test is a reranker: CA-GF retrieves the top 100 candidates, and our operator, or each baseline aggregator, reranks them by a convex mix with CA-GF's score. The mixing weight is tuned identically for all aggregators. Beating CA-GF here means a statistically significant NDCG@5 gain under a paired test over users, with Holm correction across datasets, alongside a failure of the unsigned power-mean reranker to achieve the same gain.

Two ablations isolate the key ingredients. The symmetric case $p=q$, $\lambda=1$ isolates asymmetry. Replacing fuzzy signed distances with plain differences of criterion means isolates the fuzzy machinery.

Why not just…?

Why not just use signed differences of mean criterion ratings? This is the most serious threat. Directional Hausdorff distance on α-cuts of a unimodal rating histogram may be nearly a monotone function of the mean difference. If so, the fuzzy layer adds cost without value. The crisp-mean ablation runs our operator unchanged on mean differences. If it matches the fuzzy version, we will report the contribution as the asymmetric operator alone. We expect the fuzzy version to help mainly on items with polarized, bimodal rating distributions, which are non-convex fuzzy sets. We will stratify results by a bimodality statistic so the difference, or its absence, is visible.

Why not just let a gradient-boosted model learn the asymmetry from signed features? With enough data it probably can. That is why the claim is restricted to a regime rather than made globally. We will stratify users by history length and expect the boosted model to overtake the operator above some threshold. Showing where that crossover lies is a result in itself. If the boosted model wins even for users with fewer than ten interactions, the low-data argument fails. In that case we would retain the operator only for its interpretable parameters, such as a user's fitted veto strength $q$.

Why not just use a tie-free score like the Poincaré mapping? Analysis on the Hesitation proves its score never ties. Removing ties does not tell a ranker that one option fails badly on location while another fails mildly on everything. The tie diagnostic and the ranking comparison on identical inputs settle whether strict monotonicity or directional asymmetry is what improves held-out ranking.

Why not just use a standard weighted average? Power-mean results on summarization in Power mean pyramid scores (Spearman 0.2417 against 0.169 for linear pyramid pooling on the TAC-08 test set) and convexity failures in FALCON suggest it will lose. The arithmetic-mean baseline measures by how much on this task.

Proposed Experiments

We use TripAdvisor (7 criteria), BeerAdvocate (4 criteria, over 3 million ratings), Yahoo!Movie (4 criteria) and Yelp-2022, with the user-level temporal splits of the graph-filtering papers. Metrics are NDCG@5, Precision@5 and Recall@10 for top-N ranking. We also report Kendall's τ between predicted scores and held-out overall ratings, tie rates, and wall-clock time per million scored pairs, to address the runtime question that theoretical fuzzy papers leave open.

Four checks are decisive. First, the fitted global $q$ exceeds $p$, and $\lambda$ exceeds one, with bootstrap confidence intervals excluding equality on at least three of the four datasets. Second, the asymmetric operator beats its symmetric ablation and the unsigned power mean significantly. Third, the operator beats the boosted model on the shortest-history user stratum. Fourth, the operator, used as a reranker, adds a significant NDCG@5 gain on top of CA-GF.

Failure of the first check falsifies the central claim. Failure of the fuzzy-versus-crisp comparison narrows it to the operator. The tie study is reported regardless of outcome, because it shows whether the discrimination problem these measures target occurs on real data at all.