Combating Phone Scams with LLM-based Detection: Where Do We Stand? (Student Abstract)

Zitong Shen, Kangzhong Wang, Youqian Zhang, Grace Ngai, Eugene Yujun FuAAAI 2025 · DOI

Introduction

Phone scams have evolved into a multi-billion dollar global crisis. Modern scammers have moved beyond simple scripts, employing sophisticated psychological manipulation, deepfake audio, and number spoofing to deceive victims. Traditional defense mechanisms, such as blacklists or keyword-based machine learning classifiers, are increasingly failing because they are reactive. Once a specific set of fraudulent keywords is identified and blocked, scammers simply pivot their vocabulary to bypass detection.

This research investigates whether Large Language Models (LLMs), with their advanced natural language understanding, can break this "cat-and-mouse" cycle by identifying the underlying intent and tactics of a scam in real-time, rather than just flagging specific words.

The Keyword Trap: Why Traditional ML Fails

Traditional machine learning models, such as Random Forest (RF) and Support Vector Machines (SVM), often show deceptive success. When trained on standard datasets, these models achieve near-perfect scores. However, this performance is often built on a "keyword trap"—the models learn to associate specific terms with fraud rather than understanding the fraudulent context.

To test this, researchers created the Our-Synt (Adversarial) dataset, which includes conversations that intentionally incorporate common fraudulent keywords in legitimate contexts or avoid them in fraudulent ones. As shown in the table below, while the baseline model performs flawlessly on standard datasets (SC, SD, MASC), its performance collapses when faced with adversarial tactics.

Performance of traditional ML models across different datasets, highlighting the collapse on adversarial data (Our-Synt).

The Random Forest model's recall plummeted to 0.16 on the adversarial dataset, resulting in an F1-score of just 0.26. This proves that current non-LLM detectors are brittle and easily bypassed by strategic vocabulary shifts.

LLM Performance: Nuance and Reliability

Unlike traditional models, LLMs demonstrate a superior ability to identify scams based on "soft" signals like emotional manipulation and persuasive techniques. The researchers evaluated five prominent models: GPT-4, GPT-4o, GLM4, Doubao-Pro-32k, and ERNIE-3.5-8k.

Precision vs. Recall

In the context of security, a "false negative" (missing a scam) is significantly more dangerous than a "false positive" (flagging a legitimate call). Currently, LLMs lean heavily toward precision at the expense of recall. On real-world data ("Our-Real"), LLMs achieved high precision (over 0.98), meaning they rarely misidentify a legitimate call. However, recall for real-world scams was as low as 0.72 for some models, implying that nearly 30% of scams would still reach the victim undetected.

The Consistency Gap

A critical finding of this study is the "Consistency Gap." LLMs often suffer from hallucinations, which manifest as inconsistency when presented with the same data multiple times. The researchers used a Reliability Score based on discrete semantic entropy to quantify this.

Precision vs. Recall for various LLMs, with bubble size indicating the Reliability Score.

As illustrated in the scatter plot, GPT-4o emerged as the most robust model, achieving a perfect reliability score of 1.0 on real-world data. In contrast, models like GLM-4 and Doubao showed significantly more inconsistency, particularly on synthetic or adversarial data.

Conclusion and Future Outlook

While LLMs represent a significant leap forward in understanding the nuances of fraudulent communication, they are not yet a "silver bullet" for phone scam detection. The current recall rate of 0.72 is insufficient for high-stakes security applications.

Future development must focus on three key areas:

  1. Improving Recall: Minimizing false negatives is essential to ensure users are protected from sophisticated scammers.
  2. Mitigating Hallucinations: Reducing semantic entropy is vital for building user trust in automated detection systems.
  3. Addressing Practical Constraints: Beyond detection accuracy, future systems must address the latency of real-time analysis and the significant privacy concerns associated with processing private phone conversations through large-scale models.

Summary updated

BibTeX
@inproceedings{shen2025combating,
  author = {Zitong Shen and Kangzhong Wang and Youqian Zhang and Grace Ngai and Eugene Yujun Fu},
  title = {{Combating Phone Scams with LLM-based Detection: Where Do We Stand? (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i28.35298},
}
Figure