Xing USP: A Cross-Family Semantic Contract for Information Extraction and Intent-Slot Parsing

Zhongqiu Li, Boyang Wang, Ruiyu Fang, Shiquan Wang, Di Wu, Jiayi Liang, Feilong Zhang, Hongfeng Du, Xiaomeng Huang, Mengjiao Bao, Luwen Pu, Xuewei He, Yu Zhao, Chao Wang, Kaidong Yu, Bingkai Yang, Qingjun Zhu, Shaoyi Du, Shuangyong Song, Yongxiang LiarXiv 2026 · arXiv

TL;DR

The authors introduce USP, a unified "semantic contract" that bridges the gap between Information Extraction (IE) and Intent-Slot parsing. By normalizing 61 datasets into a shared representation and introducing five diagnostic "views," they demonstrate that standard evaluation metrics often mask significant structural failures in large language models (LLMs).

What Problem Does This Paper Address?

Structured semantic parsing—tasks like identifying entities (NER), relations (RE), events (EE), or user intents and slots (SLU)—is traditionally evaluated in silos. More importantly, evaluation is often shallow, focusing on whether a model picks the right category label while ignoring whether it correctly binds arguments to specific instances.

For example, a model might correctly identify a "Transaction" event but link the "Buyer" to the wrong "Amount." Current benchmarks often fail to penalize these structural misalignments, leading to an overestimation of model utility in real-world applications where data integrity and instance-level associations are critical.

Methods

The core of the paper is the USP Semantic Contract. This is a shared interface that represents categories, fields, values, and instance associations across all four task families.

The USP Contract

Instead of task-specific formats, every output follows a Category -> List of Argument Groups structure. This preserves the "conceptual identity" of an instance (e.g., a specific event or a specific tool call) without requiring complex, non-standardized identifiers.

Diagnostic Evaluation

Rather than relying on a single aggregate score, the authors propose five "views" that ask increasingly difficult questions about the model's output:

  1. Category Agreement: Did you pick the right label?
  2. Field Recognition: Did you identify the right slots?
  3. Field-Value Pairing: Did you get the right data in those slots?
  4. Instance-Bound Assignment: Did you group the right slots together for a specific instance?
  5. Complete-Instance Recovery: Did you get the entire structure perfectly right?

The authors fine-tuned a 29B parameter Mixture-of-Experts (MoE) model, Xing4.0-29B-Intent, using a unified autoregressive objective to handle these diverse tasks through a single schema-conditioned prompt.

Datasets

The researchers normalized 61 public datasets into the USP format, creating a massive, unified corpus for training and evaluation. This includes:

  • 29 NER datasets (e.g., OntoNotes, MSRA)
  • 16 RE datasets (e.g., DocRED, FewRel)
  • 9 EE datasets (e.g., DuEE, CASIE)
  • 7 Intent-Slot sources (e.g., ATIS, SNIPS, MultiWOZ)

They also introduced USP-Core-6K, a metadata-stratified subset designed for cost-efficient model screening without sacrificing representativeness.

Key Insights and Ideas

  • The "Widening Gap": The most striking finding is that as the evaluation criteria become stricter (moving from Category to Complete-Instance matching), the performance gap between the best model and its competitors grows dramatically.
  • Category Agreement is Deceptive: High scores in category matching often hide "structural deficits." A model might look competent at labeling but fail miserably at actually structuring the data it extracts.
  • Unification without Dilution: The paper proves that heterogeneous tasks like NER and complex Event Extraction can share a single representation and learning objective without losing the nuances required for each specific task.
Figure 2: Task-balanced F1 under five structural matching views. Each view evaluates the same outputs using its own matching units; the profiles do not represent sequential processing stages.

Key Results

  • State-of-the-Art Performance: The fine-tuned Xing4.0-29B-Intent model achieved a task-balanced Field-Value Micro-F1 of 57.70, ranking first across all four task families and all five diagnostic views.
  • Structural Superiority: While its lead over the next best system was only 2.93 points for simple category matching, that margin exploded to 20.73 points for complete-instance recovery.
  • Robustness across Datasets: As shown in the per-dataset margin analysis, the model maintained a positive margin over the best alternatives across the vast majority of the 61 datasets.
Figure 3: Per-dataset Field-Value Micro-F1 margins of Xing4.0-29B-Intent over the best alternative. Vertical offsets separate overlapping points within each task fam- ily.

Limitations and Future Work

  • Conversion Fidelity: The authors acknowledge that while their converted records satisfy the USP contract, a formal "round-trip audit" (converting USP back to native formats) is needed to ensure no source semantics were lost during normalization.
  • Schema Sensitivity: The current study uses fixed schemas. Future work is needed to explore how sensitive models are to changes in schema wording or the inclusion of detailed descriptions.
  • Inference Budgets: The comparison includes proprietary APIs and open-weight models with different parameter counts; therefore, the results reflect performance under specific execution conditions rather than a strictly cost-matched benchmark.

Summary updated

Abstract

Structured semantic parsing is typically judged by whether the correct category is assigned, which leaves value recovery and instance-level associations unexamined: a system may recognize an event yet bind an argument to the wrong instance, or identify an intent yet supply an incorrect slot value. We present USP, a cross-family semantic contract that represents categories, fields, values, and instance associations through a common interface for named-entity recognition, relation extraction, event extraction, and intent-slot parsing. The contract is paired with five diagnostic matching views---category agreement, field recognition, field-value pairing, instance-bound assignment, and complete-instance recovery---that ask distinct questions about the same output. Using this contract, we normalize 61 datasets into a unified corpus with shared schemas, prompts, and matching rules, and compare six models on a fixed test subset. Our fine-tuned Xing4.0-29B-Intent attains 57.70 task-balanced Field-Value Micro-F1 (equally weighted across the four task families) and ranks first in every family and on all five views; its margin over the strongest alternative widens from 2.93 points for category matching to 20.73 points for complete-instance matching. This widening gap shows that category agreement alone understates structural deficits. USP thus offers the community a shared representation, a normalized 61-dataset corpus, and a diagnostic evaluation protocol that make structural agreement measurable and comparable across extraction and intent-slot tasks.
BibTeX
@misc{li2026xing,
  author = {Zhongqiu Li and Boyang Wang and Ruiyu Fang and Shiquan Wang and Di Wu and Jiayi Liang and Feilong Zhang and Hongfeng Du and Xiaomeng Huang and Mengjiao Bao and Luwen Pu and Xuewei He and Yu Zhao and Chao Wang and Kaidong Yu and Bingkai Yang and Qingjun Zhu and Shaoyi Du and Shuangyong Song and Yongxiang Li},
  title = {{Xing USP: A Cross-Family Semantic Contract for Information Extraction and Intent-Slot Parsing}},
  year = {2026},
  eprint = {2610.34343},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.34343},
}
Figure