FEND: Far-End Detection in Outbound Voice AI Calls

Abhishek Kumar, Suraj Agrawal, Aashraya SachdevaarXiv 2026 · arXiv

TL;DR

FEND (Far-End Detection) moves beyond binary "human vs. machine" classification for outbound calls, introducing a four-class framework: HUMAN, VOICEMAIL, IVR, and AI CALLEE. By combining content-based LLM detectors with a novel speech-timing (cadence) classifier, FEND achieves a 0.979 F1-score in separating live people from automated systems, effectively identifying the growing trend of AI-powered call screeners.

What Problem Does This Paper Address?

In the world of conversational Voice AI, the "far end" of a call is no longer just a person or a mailbox. Traditional Answering Machine Detection (AMD) provides a single bit of information: is it a human or a machine?

For a modern AI agent, this is insufficient. A "machine" could be a voicemail (where the agent should leave a message), an IVR menu (where it should stay on the line to navigate), or a consumer AI call screener (where it needs to introduce itself to another bot). Treating all machines as "uninteresting" leads to expensive mistakes, like hanging up on a potential lead or reading a script to a mailbox. Furthermore, legacy AMD tools are notoriously inaccurate, often misclassifying 30% to 66% of live humans as machines because they speak for more than a few words.

Methods

The authors propose FEND, a multi-detector framework that processes both raw audio and streaming transcripts concurrently.

Figure 1: Detection architecture. Four detectors read the raw audio frames and the streaming transcript and run concurrently, each able to fire one class. Firings accumulate over a window of W seconds and the highest-priority class that has fired wins, with HUMAN the default when nothing fires. Section 5.4 sweeps W and the ordering.

FEND employs four specialized detectors:

  1. Tone Detector: Uses Short-Time Fourier Transform (STFT) to identify the "record-now" beep following a greeting.
  2. Keyphrase Detector: A multilingual list of common voicemail phrases (e.g., "after the tone").
  3. LLM Detectors: Two separate prompts for a Large Language Model (GPT-4o) to classify transcripts as either VOICEMAIL or IVR based on context.
  4. AI Callee Detector: This is the most novel component. Since AI screeners often use scripted or dynamic content that evades simple keyword matching, FEND looks at cadence. It measures articulation rates and inter-word gaps. Synthesized speech typically has much shorter gaps between words than humans, who need to pause to think and breathe.

An Arbitration module then resolves conflicts. It uses a priority-based system rather than a "first-to-fire" rule, ranking definitive evidence (like a beep) higher than probabilistic evidence (like speech timing).

Datasets

The study utilizes two primary datasets:

  • Telephony-AMD (Public): 400 labeled clips across English, Spanish, French, and German, covering all four classes.
  • In-house Voice AI calls: 1,000 randomly sampled outbound calls from a production environment. This dataset revealed a surprising distribution: while ~48% were humans and ~45% were voicemail, a significant 6.9% were AI callees (mostly consumer call screening services).

Key Insights and Ideas

  • Acoustic Features are Brittle: The authors found that standard acoustic "anti-spoofing" features (like spectral flatness or formant concentration) do not transfer well across different telephony recording conditions. They often track the "channel" rather than the "speaker."
  • Timing is the Signal: Speech timing (cadence) proved to be a robust discriminator for AI. Humans are "messy" speakers; AI is too efficient. By measuring the coefficient of variation in inter-word gaps, FEND separates AI from humans with high reliability.
  • Priority > Speed: The fastest detector isn't always the most accurate. The AI callee detector fires in ~2 seconds, while voicemail detection (waiting for a beep or a long greeting) takes ~9 seconds. Arbitration ensures that a late-arriving, high-certainty signal (like a beep) can override an earlier, lower-certainty one.

Key Results

FEND significantly outperforms industry standards like Asterisk’s AMD.

  • Voicemail Accuracy: On the public benchmark, FEND reached 100% accuracy for voicemail without a single false positive on humans. In contrast, Asterisk mislabeled nearly 30% of humans as machines even after tuning.
  • AI Detection: The cadence-based model achieved an ROC-AUC of 0.952 in separating AI screeners from humans.
Cadence Features for AI Detection
  • Production Performance: In real-world traffic, FEND achieved an F1-score of 0.979 for the "Human vs. Rest" decision. Crucially, only 2.9% of live people were mislabeled as automated—a vital metric for maintaining user experience.
Per-class Results

Limitations and Future Work

The authors acknowledge several hurdles:

  • Adversarial Pacing: If AI developers realize timing is being used for detection, they could easily program their bots to include "human-like" pauses and breaths.
  • ASR Dependency: The cadence model relies on the word-boundary timings provided by the ASR engine. Different ASR models might yield different timing signatures.
  • Class Imbalance: The IVR class was extremely rare in their specific outbound sample (0.4%), making it difficult to draw broad conclusions about that specific detector's performance in the wild.

Future work involves testing these models against more diverse "conversational" AI assistants (like AI receptionists) rather than just the scripted screeners observed in this study.

Summary updated

Abstract

When a conversational Voice AI agent places an outbound call, the far end may be a live person, a voicemail greeting, an interactive voice response (IVR) system, or increasingly another AI system answering on the callee's behalf. Standard Answering Machine Detection (AMD) reports only whether a human or a machine picked up, a design meant for human call centres, where one machine is as uninteresting as another. That single bit does not tell a Voice AI agent what to do, since leaving a voicemail message, navigating an IVR and holding a short exchange with another AI are three different actions and a two-valued label cannot distinguish them. In 1,000 randomly sampled calls from our own outbound deployment, a live person answered 48.1%, a voicemail system 44.6%, an AI 6.9% and an IVR 0.4%. Our proposed framework, FEND, reads the far end's audio and its streaming transcript and runs one detector for each automated class (voicemail, IVR, AI callee), with human the default when nothing fires, returning a single label within 20 seconds. On a public benchmark the voicemail components reach 100% accuracy without ever misclassifying a live human, where Asterisk's AMD misclassifies 30 to 66% of them. For the AI callee, acoustic features do not transfer across recording conditions, but speech timing does, separating AI from human at 0.952 ROC-AUC, because synthesised speech leaves far shorter gaps between words. On production traffic, separating a live person from any automated far end reaches F1 0.979, and only 2.9% of live people are given an automated label, the error that matters most for an agent acting on the result.
BibTeX
@misc{kumar2026fend,
  author = {Abhishek Kumar and Suraj Agrawal and Aashraya Sachdeva},
  title = {{FEND: Far-End Detection in Outbound Voice AI Calls}},
  year = {2026},
  eprint = {2611.18050},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.18050},
}
Figure