Can probing classifiers reveal the learning by contact center large language models?: No, it doesn’t!

Varun Nathan, Ayush Kumar, Digvijay IngleNAACL 2024

TL;DR

This paper investigates whether probing classifiers can effectively reveal the internal learning of large language models (LLMs) fine-tuned for contact center (CC) tasks. The study finds that while CC-fine-tuned LLMs significantly outperform their general-purpose counterparts on downstream CC tasks, probing classifiers show surprisingly similar performance across both types of models, questioning their efficacy in capturing domain-specific learning.

What Problem Does This Paper Address?

Large Language Models (LLMs) are increasingly being adapted for specialized domains through fine-tuning. While this improves performance on domain-specific tasks, understanding what these models actually learn during this process remains a challenge. Probing classifiers are a common technique to analyze LLM representations and infer their learned capabilities. However, it's unclear if these probes can truly capture the nuanced learning that occurs when LLMs are fine-tuned for complex, real-world domains like contact centers, which involve noisy, conversational data. This paper aims to bridge this gap by examining whether probing tasks can reveal the specific adaptations LLMs make for contact center interactions.

Methods

The researchers fine-tuned both encoder-decoder (Flan-T5) and decoder-only (Llama) LLMs of various sizes on a proprietary dataset of contact center ASR transcripts. This fine-tuning process, termed "instruction fine-tuning," involved adapting the LLMs to a mixture of tasks expressed through natural language instructions relevant to contact center use-cases.

To evaluate the learned capabilities, they employed two main approaches:

  1. Downstream Task Evaluation: They assessed the quality of responses generated by both fine-tuned (CC-LLMs) and out-of-the-box (OOB) LLMs on contact center specific tasks, using a qualitative assessment scale.
  2. Probing Tasks: They utilized probing classifiers to analyze the internal representations of the LLMs. These probes were designed to assess conversational, channel, and ASR properties specific to the contact center domain, as well as general linguistic properties using the SentEval benchmark. A linear MLP classifier was used on extracted hidden states.

Datasets

  • Training Data: A proprietary dataset of ASR transcripts from English conversations between agents and customers across various sectors (e-commerce, ed-tech, logistics). This dataset was used for instruction fine-tuning.
  • Evaluation Data:
    • Downstream Tasks: Not explicitly detailed, but implied to be derived from the contact center domain and used for qualitative response assessment.
  • Contact Center Probing Tasks: Tasks designed to assess conversational, channel, and ASR properties, based on prior work (Kumar et al., 2021).
  • SentEval Benchmark: A standard suite of probing tasks for evaluating general linguistic properties (surface, syntactic, and semantic).

Key Insights and Ideas

  • Fine-tuning Matters for Downstream Performance: Instruction fine-tuning on contact center data dramatically improves LLM performance on relevant downstream tasks, leading to significantly higher quality responses compared to general-purpose LLMs.
  • Probing Classifiers Fall Short: Despite the clear performance gains on downstream tasks, probing classifiers show remarkably similar performance across both fine-tuned and general-purpose LLMs. This suggests that current probing methods may not be sensitive enough to capture the specific, domain-adapted learning that occurs during fine-tuning.
  • Decoding vs. Representations: The study hints that the observed differences in downstream performance might be more attributable to the LLM's decoding strategy (how it generates output from its internal representations) rather than fundamental differences in the learned representations themselves.
  • Architecture vs. Size: The T5 encoder-decoder architecture consistently outperformed Llama's decoder-only architecture, even when the T5 model was smaller, suggesting architecture plays a crucial role in comprehending conversational contexts.

Key Results

  • Downstream Task Improvement: CC-LLMs achieved over a 48% improvement in response acceptability (>=Acceptable) compared to OOB-LLMs. Specifically, 91% of CC-Flan-T5 and 87% of CC-Llama responses were rated as acceptable or better, compared to 22% and 39% for their OOB counterparts.
  • Probing Task Performance: Table 1 and Table 2 show that the average scores for probing tasks (both contact center specific and SentEval) were largely similar between CC-LLMs and OOB LLMs, with no consistent or significant advantage for the fine-tuned models. For instance, on contact center probing tasks, the average scores were in the range of 66-75 for OOB models and 66-75 for CC models.
  • SentEval Performance: CC-LLMs showed a slight decrease in performance on SentEval tasks compared to OOB models, suggesting a potential trade-off or shift in focus away from general linguistic properties towards domain-specific ones after fine-tuning.

Limitations and Future Work

  • Limited Architectures: The study only explored two model architectures (encoder-decoder and decoder-only) and a limited range of sizes. Future work could investigate other architectures and sizes.
  • Linear Probing: The use of a linear MLP probe might be too simplistic. Exploring more complex probing methods, such as attention-based probes, could yield different insights.
  • Probing Task Coverage: The existing set of probing tasks may not fully capture the dynamic and context-dependent nature of conversations. Developing more specific and comprehensive probing tasks tailored to contact center interactions is suggested.
  • Proprietary Dataset: The use of a proprietary dataset limits reproducibility. The authors encourage the development of publicly available domain-specific datasets.
  • Decoding Strategy: The paper highlights the potential importance of decoding strategies, suggesting future research should investigate how these strategies interact with learned representations.
  • Generalization: The study prompts further research into the learning dynamics of different architectures and how to best achieve domain-specific adaptation without sacrificing general capabilities.

Summary updated

BibTeX
@inproceedings{nathan2024can,
  author = {Varun Nathan and Ayush Kumar and Digvijay Ingle},
  title = {{Can probing classifiers reveal the learning by contact center large language models?: No, it doesn’t!}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2024},
}
Figure