Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks

Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, Yoav Goldberg

Introduction

Modern Natural Language Processing (NLP) relies heavily on sentence embeddings—fixed-length vector representations of variable-length sentences. While these embeddings are foundational for downstream tasks like machine translation and sentiment analysis, they often function as "black boxes." Standard evaluation metrics, which focus on final task performance, fail to diagnose why a model succeeds or fails. This paper introduces a "probing" framework to decode the specific linguistic information captured within these embeddings by using simple auxiliary tasks: predicting sentence length, word content, and word order.

Probing Methodology

The authors propose three Auxiliary Prediction Tasks to measure the "surface" properties preserved in an embedding. The core premise is that if a simple classifier can accurately predict a property from an embedding, that information must be encoded within it.

  1. Length Task: A multiclass classification task predicting sentence length (binned into 8 categories).
  2. Word Content Task: A binary task determining if a specific word appears in the sentence.
  3. Word Order Task: A binary task determining the relative order of two words known to be present in the sentence.

These tasks were evaluated across two primary architectures: Continuous Bag-of-Words (CBOW), which averages word vectors and is theoretically order-oblivious, and an LSTM Auto-encoder (Encoder-Decoder), which uses a recurrent neural network to compress and reconstruct sentences.

Key Insights: Beyond Surface Statistics

The probing tasks revealed several non-intuitive properties of common embedding models.

The Mechanism of Length Prediction

A surprising finding was CBOW's ability to predict sentence length with up to 65% accuracy, far exceeding the 20% baseline. This is attributed to the mathematical property of vector norms: as more random vectors are averaged, the resulting vector's norm tends to stabilize and decrease. Consequently, the embedding's magnitude effectively serves as a proxy for word count.

Comparison of length prediction accuracy on natural vs. synthetic sentences for CBOW models.

Order without Sequential Modeling

Despite its architecture, CBOW achieved approximately 70% accuracy on the word order task. This performance stems from "natural languageness"—the model exploits statistical regularities in language where certain functional words (e.g., "the", "a") consistently precede others.

Dimensionality and Information Bottlenecks

For LSTM models, increasing representation dimensions does not always yield better embeddings. While order prediction improves with dimensionality, word content accuracy was found to peak at 750 dimensions before declining. This suggests that at higher dimensions, the decoder (which is discarded after training) may assume more of the reconstruction burden, allowing the encoder to produce less informative representations.

Experimental Results

The results across the three auxiliary tasks highlight the strengths and weaknesses of different architectures.

Performance comparison of ED (LSTM) and CBOW models across Length, Content, and Order prediction tasks.
  • Length: LSTMs demonstrated superior performance (82–87% accuracy), though CBOW remained surprisingly effective.
  • Word Content: At lower dimensions (100–300), CBOW actually outperformed the more complex LSTM models, indicating that simple averaging is highly effective for capturing lexical presence.
  • Word Order: LSTMs were the clear leaders, reaching 91% accuracy at 1000 dimensions, confirming their ability to capture sequential structure.
  • Reconstruction vs. Probing: The study found that high BLEU scores (reconstruction accuracy) do not perfectly correlate with the capture of specific linguistic features, suggesting that BLEU is an insufficient metric for evaluating encoder quality in isolation.

Conclusion

This research demonstrates that fine-grained probing tasks provide a much clearer picture of what sentence embeddings "know" than downstream task performance alone. While current tasks focus on surface properties like length and order, they lay the groundwork for future evaluations of deeper semantic and syntactic generalizations. Understanding these internal representations is crucial for developing more robust and interpretable NLP models.

Summary updated

BibTeX
@inproceedings{adi2017fine,
  author = {Yossi Adi and Einat Kermany and Yonatan Belinkov and Ofer Lavi and Yoav Goldberg},
  title = {{Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks}},
  booktitle = {International Conference on Learning Representations},
  year = {2017},
  url = {https://openreview.net/forum?id=BJh6Ztuxl},
}
Figure