Are You for Real? Detecting Identity Fraud via Dialogue Interactions

Weikang Wang, Jiajun Zhang, Qian Li, Chengqing Zong, Zhifei Li

Introduction

Identity fraud remains a critical challenge in the financial sector, where malicious actors use stolen or fabricated personal information to secure loans. Traditional verification methods typically rely on static checks—asking for information directly provided on an application, such as a graduation year or a specific employer. However, professional fraudsters often memorize these details perfectly. The core insight of this research is that while fraudsters can memorize nodes of information, they rarely possess the "peripheral" knowledge associated with those facts—the lived experience that connects a university to its nearest subway station or a workplace to a local park. This paper introduces an automated dialogue system designed to probe this peripheral knowledge through "derived questions."

Methodology: Hierarchical Dialogue Management

The proposed system leverages a Knowledge Graph (KG) and a hierarchical reinforcement learning (HRL) framework to conduct intelligent, multi-turn verification dialogues. The architecture consists of three primary layers:

  1. The Manager (High-level Agent): This agent oversees the dialogue strategy, deciding which high-level attribute (e.g., School, Company, or Residence) should be verified next based on the current state of the conversation.
  2. The Worker (Low-level Agent): Once a category is selected, the Worker identifies specific triplets within the KG to generate multiple-choice questions. For example, if the Manager selects "School," the Worker might query the KG for a nearby subway station to form a question.
  3. KG-based Dialogue State Tracker (KG-DST): This component uses a message-passing mechanism to update the system's belief about the applicant's legitimacy. If a user correctly identifies a nearby landmark, the system increases the probability that they also know other geographically related facts.
The hierarchical dialogue management system architecture, showing the interaction between the Manager, Worker, and Applicant.

To ensure the system could generate relevant questions, the researchers augmented a standard Chinese KG with geographic data from Amap, linking entities like universities to local infrastructure. Furthermore, the system incorporates a "Spread Degree" theory, which uses search engine frequency as a proxy for question difficulty; facts that are easily found online are considered less reliable for verification than obscure, localized details.

Experimental Setup and Datasets

Evaluating such a system requires a balance of real-world data and controlled simulation. The researchers utilized 906 anonymized loan applicant profiles and conducted a human calibration study with 31 volunteers. This study, involving over 1,500 questions, was used to build a Heuristic User Simulator that mimics the behavior of both legitimate applicants (who may occasionally forget details) and fraudsters (who may know common facts but fail on obscure ones).

Results and Performance

The full system (Full-S) was benchmarked against several baselines, including rule-based systems and flat neural models. The results demonstrate that the hierarchical approach significantly improves detection capabilities.

Performance comparison of the Full-S model against baselines in terms of recognition accuracy and average dialogue turns.

As shown in the performance metrics, the Full-S model achieved a recognition accuracy of 88.4%, a substantial improvement over the "Flat Rule" (74.8%) and "Hierarchical Rule" (78.7%) baselines. While the system requires slightly more turns (9.8 on average) than some simpler models, this extra dialogue allows the Manager agent to resolve ambiguities—for instance, giving a legitimate user a second chance if they miss a single difficult question, thereby reducing false positives.

Discussion and Future Work

The "derived question" strategy proves highly effective at distinguishing lived experience from memorized data. Key insights from the study include the fact that the HRL agent naturally learns to prioritize high-value targets like "School" and "Company," which are most frequently falsified in loan applications.

However, the system currently assumes a real-time interaction where applicants cannot use search engines. Future iterations could explore "trap" questions or cross-session consistency checks to further harden the system against sophisticated attackers. Additionally, expanding the KG to include more diverse personal attributes beyond geography remains a priority for increasing the system's robustness across different demographic profiles.

Summary updated

BibTeX
@inproceedings{wang2019are,
  author = {Weikang Wang and Jiajun Zhang and Qian Li and Chengqing Zong and Zhifei Li},
  title = {{Are You for Real? Detecting Identity Fraud via Dialogue Interactions}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  doi = {10.18653/v1/d19-1185},
}
Figure