TL;DR
FEND (Far-End Detection) moves beyond binary "human vs. machine" classification for outbound calls, introducing a four-class framework: HUMAN, VOICEMAIL, IVR, and AI CALLEE. By combining content-based LLM detectors with a novel speech-timing (cadence) classifier, FEND achieves a 0.979 F1-score in separating live people from automated systems, effectively identifying the growing trend of AI-powered call screeners.
What Problem Does This Paper Address?
In the world of conversational Voice AI, the "far end" of a call is no longer just a person or a mailbox. Traditional Answering Machine Detection (AMD) provides a single bit of information: is it a human or a machine?
For a modern AI agent, this is insufficient. A "machine" could be a voicemail (where the agent should leave a message), an IVR menu (where it should stay on the line to navigate), or a consumer AI call screener (where it needs to introduce itself to another bot). Treating all machines as "uninteresting" leads to expensive mistakes, like hanging up on a potential lead or reading a script to a mailbox. Furthermore, legacy AMD tools are notoriously inaccurate, often misclassifying 30% to 66% of live humans as machines because they speak for more than a few words.
Methods
The authors propose FEND, a multi-detector framework that processes both raw audio and streaming transcripts concurrently.
FEND employs four specialized detectors:
- Tone Detector: Uses Short-Time Fourier Transform (STFT) to identify the "record-now" beep following a greeting.
- Keyphrase Detector: A multilingual list of common voicemail phrases (e.g., "after the tone").
- LLM Detectors: Two separate prompts for a Large Language Model (GPT-4o) to classify transcripts as either VOICEMAIL or IVR based on context.
- AI Callee Detector: This is the most novel component. Since AI screeners often use scripted or dynamic content that evades simple keyword matching, FEND looks at cadence. It measures articulation rates and inter-word gaps. Synthesized speech typically has much shorter gaps between words than humans, who need to pause to think and breathe.
An Arbitration module then resolves conflicts. It uses a priority-based system rather than a "first-to-fire" rule, ranking definitive evidence (like a beep) higher than probabilistic evidence (like speech timing).
Datasets
The study utilizes two primary datasets:
- Telephony-AMD (Public): 400 labeled clips across English, Spanish, French, and German, covering all four classes.
- In-house Voice AI calls: 1,000 randomly sampled outbound calls from a production environment. This dataset revealed a surprising distribution: while ~48% were humans and ~45% were voicemail, a significant 6.9% were AI callees (mostly consumer call screening services).
Key Insights and Ideas
- Acoustic Features are Brittle: The authors found that standard acoustic "anti-spoofing" features (like spectral flatness or formant concentration) do not transfer well across different telephony recording conditions. They often track the "channel" rather than the "speaker."
- Timing is the Signal: Speech timing (cadence) proved to be a robust discriminator for AI. Humans are "messy" speakers; AI is too efficient. By measuring the coefficient of variation in inter-word gaps, FEND separates AI from humans with high reliability.
- Priority > Speed: The fastest detector isn't always the most accurate. The AI callee detector fires in ~2 seconds, while voicemail detection (waiting for a beep or a long greeting) takes ~9 seconds. Arbitration ensures that a late-arriving, high-certainty signal (like a beep) can override an earlier, lower-certainty one.
Key Results
FEND significantly outperforms industry standards like Asterisk’s AMD.
- Voicemail Accuracy: On the public benchmark, FEND reached 100% accuracy for voicemail without a single false positive on humans. In contrast, Asterisk mislabeled nearly 30% of humans as machines even after tuning.
- AI Detection: The cadence-based model achieved an ROC-AUC of 0.952 in separating AI screeners from humans.
- Production Performance: In real-world traffic, FEND achieved an F1-score of 0.979 for the "Human vs. Rest" decision. Crucially, only 2.9% of live people were mislabeled as automated—a vital metric for maintaining user experience.
Limitations and Future Work
The authors acknowledge several hurdles:
- Adversarial Pacing: If AI developers realize timing is being used for detection, they could easily program their bots to include "human-like" pauses and breaths.
- ASR Dependency: The cadence model relies on the word-boundary timings provided by the ASR engine. Different ASR models might yield different timing signatures.
- Class Imbalance: The IVR class was extremely rare in their specific outbound sample (0.4%), making it difficult to draw broad conclusions about that specific detector's performance in the wild.
Future work involves testing these models against more diverse "conversational" AI assistants (like AI receptionists) rather than just the scripted screeners observed in this study.
Summary updated