The Redundancy of Explicit Syntactic Features

25 concepts21 papers1 proposal

When designing natural language processing systems, it is intuitive to assume that layering explicit linguistic knowledge—such as part-of-speech (POS) tags, dependency trees, or semantic roles—will inherently improve a model's performance. After all, human language relies on these structural rules to convey meaning.

However, a recurring pattern across decades of NLP research demonstrates a surprising counter-intuitive reality: explicit syntactic and semantic features are frequently redundant. When models have access to raw lexical data, sequential context, or dense representations, they implicitly capture these structural relationships, rendering highly engineered linguistic annotations largely unnecessary.

Lexical Subsumption: How Words Encode Their Own Syntax

Why do explicit grammatical labels often fail to provide incremental value? The primary reason is that the identity and local context of words themselves already carry their structural behavior.

In an early information-theoretic analysis, Wu et al. demonstrated in An Information-Theoretic Empirical Analysis of Dependency-Based Feature Types for Word Prediction Models that specific lexical identities are vastly more informative for word prediction than abstract grammatical labels like POS tags or dependency types. The words themselves implicitly signal their grammatical relationships. This finding is mirrored in modern parsing challenges; Müller et al. showed in Dependency parsing with latent refinements of part-of-speech tags that parsers frequently struggle with highly granular linguistic tags because those distinctions merely replicate patterns already captured by lexicalized models.

This lexical sufficiency is evident across diverse tasks:

The Overlap Between Syntactic and Semantic Layers

Another major source of redundancy occurs at the intersection of syntax and semantics. Because semantic roles (e.g., agent or patient) heavily correlate with specific syntactic constituents (e.g., subject or object), explicitly modeling both layers often results in overlapping signals.

For example, Li et al. observed in A Unified Model for Soft Linguistic Reordering Constraints in Statistical Machine Translation that integrating semantic role information into translation models provides diminishing returns when robust syntactic constraints are already active. The syntactic model effectively captures the signal intended for the semantic model. Conversely, local contextual features like POS tags can provide enough structural scaffolding for semantic tasks without full syntactic trees; Martins et al. showed in Priberam: A Turbo Semantic Parser with Second Order Features that high-quality semantic graphs can be generated with minimal reliance on external syntactic parsers.

This mutual information means that higher-level semantic features can also absorb lower-level syntactic needs, and vice versa:

Sequential Models and the Obsolescence of Explicit Trees

With the advent of powerful sequential and graph-based architectures, the need for engineered structural features has decreased even further. Neural networks excel at extracting hierarchical context directly from sequential or positional representations.

Consider dependency parsing itself. de Lhoneux et al. investigated in Recursive Subtree Composition in LSTM-Based Dependency Parsing whether explicitly modeling tree structures via recursive composition adds value to BiLSTM-based parsers. The authors discovered that recursive modeling is largely redundant because the sequential BiLSTM already implicitly captures the necessary hierarchical context.

Explicit Tree Features (Recursive Composition, Sibling Relations)
       │
       ▼ (Overlapping Context / Redundant Signal)
┌──────────────────────────────────────────────────────────┐
│  Sequential Context (BiLSTMs, Relative Positions, Spans) │
└──────────────────────────────────────────────────────────┘
       ▲
       │ (Implicitly Subsumes)
Word Identities & Local Lexical Context

Furthermore, complex graph features often duplicate simpler positional or span-based information:

Why It Works: The Principle of Correlated Signals

The fundamental reason explicit syntactic features exhibit such high redundancy is that language is a system of highly correlated signals. A single structural relationship is mirrored across multiple levels simultaneously: in the specific choice of words (lexical), the order of those words (sequential), the grammatical categories (syntactic), and the underlying meaning (semantic).

When a model is trained on dense, high-dimensional representations—whether they are lexical n-grams or neural hidden states—it naturally exploits these correlations to optimize its objective. As a result, explicitly injecting a feature like a POS tag or a dependency path rarely provides a unique discriminative signal that the model has not already derived from simpler inputs. Recognizing this redundancy allows practitioners to build faster, leaner, and less complex models without sacrificing predictive accuracy.

Key Foundational Work

An Information-Theoretic Empirical Analysis of Dependency-Based Feature Types for Word Prediction Models (1999) — contributed 1 concept elements to this direction.
Extracting Clauses for Spoken Language Understanding in Conversational Systems (2002) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_syntactic_feature_redundancy_41f75df4,
  title = {syntactic feature redundancy},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/syntactic-feature-redundancy/rdr_41f75df418dc409c8336dfec80b7f849},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_syntactic_feature_redundancy_41f75df4_v1,
  title = {syntactic feature redundancy},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/syntactic-feature-redundancy/rdr_41f75df418dc409c8336dfec80b7f849/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake