Syntax-Aware Role Definitions for Zero-Shot Cross-Lingual Semantic Role Labeling

Semantic role labeling (SRL) asks who did what to whom in a sentence. It finds the phrases attached to a verb and labels them with roles such as "the eater" or "the thing eaten." Most systems treat these roles as opaque IDs (A0, A1). Work on English has shown that swapping the IDs for their dictionary definitions from PropBank, the standard English role lexicon, helps a lot when training data is scarce. Low-resource languages are where data is scarcest. Moving that idea across languages hits two problems. First, the definitions are English and the target sentence is not, so the model must match a Swahili or Tamil phrase against an English gloss. Second, the one non-English attempt failed because the available definitions were too generic.

We propose to describe each role in two ways at once. One part is its English definition. The other is a profile of how the role is grammatically realized, for example "usually the subject of an active verb, a noun phrase, appears before the predicate." That profile is written in a language-universal grammar, so it can be read off any language that has a dependency parser. Candidate argument phrases are built from a hybrid syntactic structure that combines word-to-word grammatical links with phrase boundaries. Each candidate is matched against these two-part role embeddings with a fast dot product. The bet is that when the English gloss cannot reach a target-language phrase, the grammatical profile still can.

Core Claim

In zero-shot and few-shot cross-lingual SRL, the benefit of natural-language label definitions depends on attaching a syntactic realization profile to each label. Without it, gloss-only matching should give smaller gains cross-lingually than in English. With it, the label side carries a signal that transfers through Universal Dependencies (UD), the dependency annotation standard shared by more than 100 languages. The claim has a falsifiable corollary. Profiles should rescue cases where glosses are uninformative, such as the generic Chinese PropBank frames. They should also matter more as the target language moves further from English.

Relation to Prior Work

Three papers supply the components, and all three stop short of this question. Label Definitions Improve Semantic Role Labeling supplies the label-definition idea. It recasts role assignment as matching a [sentence] [SEP] [definition] pair, and reports +3.2 F1 with 1,000 training examples and +4.4 F1 on unseen predicates. The same paper also constrains us. Its method ran 3.1 times slower than a classification head, and it gave no gain on Chinese because the definitions there were generic. It uses no syntax and is evaluated essentially in English. We keep the definitions, add syntax to the label side, and replace the pairwise cross-encoder with precomputed label vectors.

Better Combine Them Together! (HeSyFu) supplies the hybrid encoder. It fuses phrase-structure (constituency) and dependency trees with label-aware graph convolutions and reports 89.04 F1 on CoNLL-2005 WSJ and 92.83 on CoNLL-2009 English. It also finds that processing phrases before dependencies works better. Its roles stay symbolic and it is English-only. Its sibling Encoder-Decoder Based Unified Semantic Role Labeling with Label-Aware Syntax embeds dependency labels on the input words and still predicts roles as discrete targets. We move syntactic labels onto the output role representations.

The cross-lingual papers motivate the work. On the Benefit of Syntactic Supervision for Cross-lingual Transfer in Semantic Role Labeling shows that UD supervision lifts mBERT zero-shot Argument-F1 from 51.91 to 58.52. It also introduces a two-step method that finds argument heads and then expands them into spans, but its labels stay symbolic. Transferability of Syntax-Aware Graph Neural Networks in Zero-Shot Cross-Lingual Semantic Role Labeling finds that graph networks degrade as predicate–argument distance grows, with Tamil as a hard case. SHINE shows that phrase-level structure closes a "span gap" in cross-lingual entity and relation extraction, but it uses symbolic heads and no argument structure.

Universal Proposition Bank 2.0 makes the whole proposal possible. It annotates 23 languages with English PropBank rolesets, so English definitions are the correct label vocabulary for every target language. Its benchmarks nonetheless never use those definitions.

Research Program

Span encoder. A multilingual encoder (XLM-R) feeds a graph that has two kinds of nodes and edges. Word nodes are joined by UD dependency edges, each carrying its relation type. Span nodes sit above the words they cover.

Where the phrases come from. Most low-resource languages have no constituency treebank, so phrase nodes are derived from the dependency tree itself: each word's projective subtree becomes a candidate phrase, labeled by the part of speech of its head. Where a trained constituency parser exists, we use it instead. The multilingual parser of Multilingual Constituency Parsing with Self-Attention and Pre-Training covers the SPMRL languages. Comparing the two phrase sources tells us whether real constituency adds anything beyond the dependency-derived phrases.

Distance hypothesis. Phrase nodes collapse long stretches of text into a single hop. We therefore expect them to soften the distance degradation that Devianti and Miyao report.

Scoring. For predicate sense $p$, candidate span $s$ with encoder representation $h_s$, and role $r$ in the roleset of $p$, the score is

$$\text{score}(s, r) = h_s^\top \big( f(d_{p,r}) + W \pi_{p,r} \big),$$

where:

  • $f$ is a text encoder applied to the English PropBank definition $d_{p,r}$, such as "eater."
  • $\pi_{p,r}$ is the role's realization profile: a smoothed distribution over syntactic configurations for that role, estimated from English training data. The configurations are the UD relation path from predicate to argument head, the phrase category, linear side relative to the predicate, and voice.
  • $W$ is a learned projection.

Profiles for rare rolesets back off toward the profile of the generic role (all A0s, say). All label vectors are computed once in advance. Inference therefore costs one span encoding plus dot products, which addresses the 3.1× slowdown of the cross-encoder approach. Training uses English (CoNLL-2009/2012 PropBank), optionally plus a small amount of target-language data.

Two analyses. The first decomposes gains by whether the target language's word order and case marking match English. The second tests whether the profile can substitute for a bad gloss. For that test we retrain on Chinese CoNLL-2009 with Chinese-estimated profiles attached to the generic Chinese definitions. This directly revisits the failure case reported by Zhang, Jindal and Li.

Baselines

Naive reference. XLM-R with a standard symbolic span classifier, trained on English and applied zero-shot. It shows how much any structure or label semantics buys.

Strongest syntax-only competitor. UD multitask supervision with symbolic labels (Zhang, Strubell and Hovy). Its reported mBERT zero-shot Argument-F1 on Universal PropBank is 58.52. On the 23-language UP2.0 setting, the strongest reported average is TAGAT at 52.78 F1 (Devianti and Miyao). We report it alongside SAN-RPR, which won on 18 of the 23 languages. Those numbers come from different encoders and splits. We will therefore re-run both methods with our XLM-R encoder, the same UD parses, the same English training data and the same hyperparameter search budget. Beating them means a higher macro-average F1 over the 23 languages and wins on a majority of individual languages.

Strongest label-only competitor. The definition cross-encoder of Label Definitions Improve Semantic Role Labeling, run with XLM-R. No cross-lingual result for it exists on Lacuna. The closest is its English few-shot gain of 3.2 F1.

Hybrid symbolic competitor. HeSyFu with dependency-derived phrases and symbolic roles. It separates the effect of the hybrid encoder from the effect of the label side.

Ablations of the key ingredient. We drop $\pi$ (gloss only), drop $f(d)$ (profile only), replace phrase nodes with dependency-only graphs, and swap derived phrases for parser constituents. Each is trained under identical compute.

Why not just…?

Why not just use the definition cross-encoder with a multilingual encoder and skip syntax? This is the most serious threat. It is the label-only baseline above. If gloss-only cross-encoding matches the full model on distant languages, our central claim fails. The profile-only ablation and the per-language breakdown settle the question. The evidence that glosses alone transfer poorly is so far indirect: they failed on Chinese, a different language, even monolingually.

Why not just add UD supervision to symbolic heads? That already gives +6.6 F1 (Zhang, Strubell and Hovy). It cannot help unseen or rare rolesets, however, because a symbolic head has no representation for a label it never saw. We will evaluate separately on target-language predicates absent from English training. That is where the English unseen-predicate gain of +4.4 F1 suggests the label side matters.

Why not just translate the definitions into the target language? This is cheap and could close the gloss gap without any syntax. We include machine-translated definitions as an extra arm. If it equals the profile arm, the honest conclusion is that the profile's value lies only in low-gloss-quality settings such as Chinese, and we will report it that way.

Why not translate-train with label projection, as in X-SRL? Projection worked well for German predicates in X-SRL, at 84.9 F1. But it needs good machine translation and suffers from shifts introduced by translation. We compare on the X-SRL languages (German, French, Spanish) and expect projection to stay competitive there. Our method's case rests on the UP2.0 languages where translation quality is poor, and we will say so.

Proposed Experiments and Decisive Checks

The main benchmark is UP2.0 zero-shot transfer from English to all 23 languages, scored with argument span F1 (macro and per-language). It is complemented by few-shot curves at 100, 1,000 and full target sentences. The Finnish 100-sentence point gives a direct reference: 44.76 → 67.22 F1 with syntax in Zhang et al.

Three checks decide the outcome.

  1. Interaction. The combined gloss-plus-profile model must beat both single-source variants. The margin should grow with typological distance from English. A gain that stays flat across languages would mean the profile is just extra capacity.
  2. Distance. Binning F1 by predicate–argument distance should show phrase nodes shrinking the long-distance gap, particularly on Tamil versus Telugu.
  3. Gloss rescue. On Chinese CoNLL-2009, profiles should produce gains where generic definitions previously gave none.

We also report inference throughput against the cross-encoder, expecting near-classifier speed.