CONCORD-Rec: Catalog-Conditioned Modality Utility for Semantic-ID Diffusion Recommendation

Haopeng Jin, Bingkang Shi, Tao Yu, Yifei Ma, Hao Liu, Hongzhu Yi, Jinwen LuoarXiv 2026 · arXiv

TL;DR

CONCORD-Rec is a multimodal recommender that uses discrete diffusion and a novel "reliability gate" to handle messy real-world data. By training this gate to recognize when a modality (like a video's audio or a product's title) actually helps narrow down the catalog, it achieves significant gains in accuracy and robustness, especially when data is missing or conflicting.

What Problem Does This Paper Address?

Modern recommendation systems often use "generative" approaches, where an item is represented by a short sequence of codes called a Semantic Identifier (SID). To make better predictions, these models try to fuse multiple "modalities"—text descriptions, images, audio, and video.

However, more data isn't always better. In the real world, multimodal data is often:

  • Missing: A video might not have a soundtrack.
  • Degraded: An image might be low-resolution or blurry.
  • Conflicting: A product title might say "Red Shirt" while the image shows a blue one.

Standard models often use "uniform fusion," treating all available data as equally useful. This can lead to "garbage in, garbage out," where a noisy or conflicting modality misleads the model. The authors of CONCORD-Rec set out to build a system that can selectively ignore unhelpful information and focus on what truly reduces ambiguity in the product catalog.

Methods

CONCORD-Rec introduces a reliability-conditioned discrete diffusion framework. Its core innovation is how it supervises the fusion of different modalities:

  1. Catalog-Conditioned Utility: Instead of just training the model to predict the next code, the authors use a "teacher" model to calculate counterfactual utility. They compare how much the prediction loss improves when a specific modality is added versus when it is absent, all while looking at the actual "feasible catalog" (the set of items that match the codes already predicted).
  2. Tokenwise Allocation Gate: Not all modalities are useful for every part of an item's ID. For example, text might help identify a broad category (the first code), while an image helps identify a specific style (a later code). CONCORD-Rec uses a gate that decides, for each specific position in the SID, how much weight to give to each modality.
  3. Bounded Modality Residuals: To prevent any single noisy modality from completely hijacking the prediction, the model uses a mathematical "bound" (a Btanh function). This ensures that modalities can only refine the "behavioral" prediction (based on user history) within a safe range.
  4. Feasible Catalog Decoding: During the recommendation process, the model constantly checks its predictions against the actual catalog to ensure it only generates valid item IDs.
Figure 1: CONCORD-Rec architecture. Historical content and SID states condition a gate over bounded modality corrections. The triangle depicts convex mixtures in a two-coordinate slice; catalog support constrains joint commitments. Offline, a frozen out-of-fold teacher compares target-group losses on identical states and support. Signed utilities supervise detached gate targets. Availability varies by input; geometry, weights and codes are schematic.

Datasets

The researchers evaluated CONCORD-Rec across a wide range of benchmarks:

  • MicroLens-50K & MicroLens-100K: Large-scale audiovisual datasets for short-video recommendation.
  • TikTok, Amazon Baby, Sports, and Clothing: Datasets originally used in the DiffMM paper, featuring text, image, and audio features.
  • Amazon Scientific, Instrument, and Game: Datasets from the ANR-DiffRec paper.
  • MovieLens & Steam: Standard recommendation benchmarks.

Crucially, they introduced controlled intervention protocols to test robustness. They systematically removed modalities, degraded signal quality, and swapped in "conflicting" content (e.g., pairing a video with a caption from a different category) to see how well the models held up.

Key Insights and Ideas

  • Availability $\neq$ Usefulness: Just because a modality is present doesn't mean it should be used. The "aha moment" here is supervising the gate itself with a measure of how much a modality actually helps narrow down the catalog.
  • The Catalog is a Constraint: Generative recommendation isn't just about predicting tokens; it's about finding a real item. By conditioning the utility on the "feasible catalog support," the model learns to prioritize information that distinguishes between similar items.
  • Granularity Matters: Moving from "whole-item" gating to "tokenwise" gating allows the model to be much more surgical, using different evidence for different levels of the item's semantic hierarchy.
  • Stability Bounds: The authors provide a theoretical proof showing that their bounded interface makes the ranking stable even if the gate makes small errors.

Key Results

CONCORD-Rec consistently outperformed state-of-the-art baselines:

  • Overall Accuracy: It improved NDCG (a standard ranking metric) by about 4% across 20 different dataset/protocol combinations.
  • Robustness to Conflict: In the most challenging "semantic conflict" tests (where modalities contradict each other), CONCORD-Rec showed a massive 14.63% improvement over uniform fusion.
  • Efficiency: The model is highly efficient. A 4-step CONCORD-Rec process achieved better quality than a standard 16-step uniform decoding process while reducing p95 latency by over 80%.
Figure 3: Observation robustness on MicroLens-100K. Within-family severity sweeps compare uniform fusion and CONCORD-Rec under the same corruption draws and common-encoder protocol. Panels separate missing observations, semantic conflict, and signal degradation; every marker is an author-confirmed five-pair mean. At severity 0.3, relative NDCG@20 gains are 7.95%, 14.63%, and 8.46%. Lines connect sampled settings. The plotted means have no marginal-SD bands; paired-difference inference is reported in Appendix A.26.

Limitations and Future Work

  • State-Specific Bounds: The current theoretical stability bounds are specific to a single state; they don't yet provide a global guarantee for the entire multi-step diffusion trajectory.
  • Offline vs. Online: Like most research papers, the evaluation is offline. The authors note that while their "catalog utility" signal is a strong proxy, it doesn't perfectly capture real-world user satisfaction, which would require online A/B testing.
  • Future Directions: The authors suggest that future work could involve "adaptive-step" policies, where the model automatically decides how many denoising steps are needed based on the observed utility and token entropy.

Summary updated

Abstract

Multimodal generative recommendation with discrete semantic identifiers (SIDs) requires distinguishing historical modalities that merely alter token predictions from those that genuinely reduce catalog-level ambiguity. When audiovisual, visual, or textual streams are missing, degraded, or semantically conflicting, uniform fusion and unconstrained token gates can propagate misleading content across iterative denoising steps. We propose CONCORD-Rec, a reliability-conditioned discrete diffusion recommender supervised by counterfactual catalog-partition utility. Given a fixed tokenizer and a partially masked SID state, CONCORD-Rec evaluates out-of-fold teacher coalitions over the exact feasible catalog support and aggregates item posteriors across the target code partition at each unresolved position. Rather than distilling raw token probabilities directly, the resulting counterfactual loss differences supervise an availability-masked tokenwise gate that allocates bounded modality residuals around a behavioral denoiser while retaining an exact behavioral fallback slot. We establish theoretical fixed-support ranking stability bounds under gate perturbations and evaluate position-level and code-conditional allocation across audiovisual and image-text recommendation benchmarks under controlled missingness, signal degradation, and semantic conflict. Across 20 source-compatible protocol-dataset rows, CONCORD-Rec improves NDCG by 4%; under matched 30% conflict it improves NDCG by 14.63%. The full-vs-uniform-T=16 sweep improves quality by 2.00-3.58% while reducing p95 latency by up to 86.89%.
BibTeX
@misc{jin2026concord,
  author = {Haopeng Jin and Bingkang Shi and Tao Yu and Yifei Ma and Hao Liu and Hongzhu Yi and Jinwen Luo},
  title = {{CONCORD-Rec: Catalog-Conditioned Modality Utility for Semantic-ID Diffusion Recommendation}},
  year = {2026},
  eprint = {2611.23710},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23710},
}

Loading related content...

Figure