Differential Use of Implicit Negative Evidence in Generative and Discriminative Language Learning

Anne S. Hsu, Thomas L. GriffithsNeurIPS 2009 · OpenReview

A classic puzzle in cognitive science is the "poverty of the stimulus." Children learn complex linguistic rules—such as restrictions on verb alternations—almost entirely from positive evidence (hearing adults speak correctly). They rarely receive negative evidence (explicit corrections like, "No, you can't say 'I disappeared the rabbit'").

If children only hear what is right, how do they ever learn that certain novel generalizations are wrong? Historically, some researchers argued this proves language-specific knowledge must be innate. Others countered that statistical learning is sufficient. This paper suggests the answer depends fundamentally on the type of learning approach used: generative or discriminative.

Two Models of Language Learning

The researchers contrast two computational models of learning to understand how they handle missing data:

  1. The Generative Model (Hierarchical Bayesian Model): This approach treats language learning as inferring the parameters of an underlying probability distribution. Crucially, it leverages implicit negative evidence. If a verb could have appeared in a certain structure but didn't, the model treats this absence as meaningful—a "suspicious coincidence"—and lowers the probability of that combination being grammatical.
Generative Model Structure
  1. The Discriminative Model (Logistic Regression): This model learns a boundary to classify sentences as grammatical or ungrammatical based on feature weights (verbs and structures). Without explicit "No" labels for specific combinations, it tends to generalize based on the features it has seen, making it robust to the specific distribution of the input but blind to implicit negative evidence.

The Artificial Language Experiment

To test these models and compare them to human learning, the authors designed an artificial language with 3 nouns and 4 verbs, capable of appearing in 3 sentence structures (S1, S2, S3).

The training data was carefully manipulated:

  • Most verbs appeared in only one structure.
  • One "alternating" verb appeared in two structures.
  • The Critical Test (V4): Verb 4 was shown frequently in structure S1 but never in S2.

The core question: Would learners think V4 used in S2 is grammatical?

Key Findings: The Power of "Not Hearing"

The computational models yielded starkly different predictions for the critical V4-S2 combination.

Model Predictions
  • Model Predictions: The generative model (HBM) used the "suspicious coincidence" of V4 never appearing in S2 to predict that the combination is ungrammatical (see chart d). In contrast, the discriminative model (Logistic Regression), lacking a negative label, generalized from the fact that S2 is a common structure and V4 is a common verb, predicting the combination is grammatical.
  • Human Judgments: The researchers then tested 22 human participants, splitting them into two conditions designed to induce either a generative or discriminative mindset.
  • Generative Group: Participants listened to an "adult" speaker (all grammatical) and a "child" speaker (ungrammatical). Mirroring the HBM, they overwhelmingly rejected the unseen V4-S2 combination.
  • Discriminative Group: Participants performed a classification task with immediate "Correct/Incorrect" feedback. Mirroring the Logistic Regression model, they deemed the unseen V4-S2 combination grammatical ($p = 0.007$ difference between groups).
  • Production vs. Judgment: Interestingly, even though discriminative learners thought the unseen structure was grammatical, they rarely produced it (only 20% of the time), showing a gap between what we think is "legal" and what we choose to say.

Conclusion

The study demonstrates that humans are not hard-wired into a single learning mode; context dictates strategy. By simply changing the presentation of the task, researchers could flip how people treated missing data.

However, the generative condition more closely mirrors real-world child language acquisition, where children listen to a distribution of "correct" speech rather than receiving constant binary feedback. This supports the idea that children can learn linguistic exceptions through statistical inference—specifically by leveraging implicit negative evidence—without needing innate "grammar modules."

Summary updated

Abstract

A classic debate in cognitive science revolves around understanding how children learn complex linguistic rules, such as those governing restrictions on verb alternations, without negative evidence. Traditionally, formal learnability arguments have been used to claim that such learning is impossible without the aid of innate language-specific knowledge. However, recently, researchers have shown that statistical models are capable of learning complex rules from only positive evidence. These two kinds of learnability analyses differ in their assumptions about the role of the distribution from which linguistic input is generated. The former analyses assume that learners seek to identify grammatical sentences in a way that is robust to the distribution from which the sentences are generated, analogous to discriminative approaches in machine learning. The latter assume that learners are trying to estimate a generative model, with sentences being sampled from that model. We show that these two learning approaches differ in their use of implicit negative evidence -- the absence of a sentence -- when learning verb alternations, and demonstrate that human learners can produce results consistent with the predictions of both approaches, depending on the context in which the learning problem is presented.
BibTeX
@inproceedings{hsu2009differential,
  author = {Anne S. Hsu and Thomas L. Griffiths},
  title = {{Differential Use of Implicit Negative Evidence in Generative and Discriminative Language Learning}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2009},
  url = {https://openreview.net/forum?id=e0NiTNkNXNL},
}
Figure