Abstract
Traditional knowledge-based authentication (KBA) relies on a static set of questions that are often easily guessable or inapplicable to many users. This paper introduces a system that automatically generates personalized security questions from short, user-provided narratives about personal life events. By leveraging sentence simplification followed by either dependency parsing or semantic role labeling, the proposed method generates simple, fact-based questions. Evaluation against a human-generated gold standard shows that this approach significantly outperforms existing state-of-the-art question generation systems in both accuracy and conciseness.
Introduction
Security questions are a ubiquitous fallback for password recovery, yet the current implementation is fundamentally flawed. Most systems utilize a finite, static set of questions such as "What was the name of your first pet?" The authors identify three critical issues:
- Guessability: Answers are often discoverable via social media or public records.
- Applicability: Questions may not apply to all users (e.g., not everyone has a favorite uncle).
- Security Risks: Standardized questions allow for large-scale data aggregation and exploitation following breaches.
To address these, the researchers propose a system to create questions that are personalized, memorable, and unambiguous, ensuring a single clear answer for the user while remaining difficult for attackers to guess.
Methodology
The system employs a two-phase pipeline to transform free-form user text into crisp authentication questions.
Phase 1: Sentence Simplification
Unlike educational question generation (QG) which favors complexity, security QG requires simplicity. The authors developed a rule-based approach to break complex sentences into "one-fact" units. For instance, a sentence describing multiple actions is split so that each resulting unit contains only one subject and one main verb, reducing ambiguity.
Phase 2: Question Generation
Two distinct NLP techniques were explored for generating questions from simplified sentences:
- Dependency Parsing (DepPar): This method identifies grammatical structures (e.g., subject, object) and maps them to "WH*" question types. For example, a location tag triggers a "Where" question.
- Semantic Role Labeling (SRL): This focuses on the semantic roles of words, such as the "agent" (A0) or "theme" (A1). These roles are mapped to specific question types (e.g., mapping an agent to "Who").
Evaluation and Results
The system was evaluated using a custom dataset of 358 source sentences provided by 40 participants. A "gold standard" of 519 question-answer pairs was manually created for comparison. The researchers compared their DepPar and SRL approaches against two established QG systems: OA (Yao et al.) and H&S (Heilman and Smith).
Performance Metrics
The Dependency Parsing (DepPar) approach emerged as the superior method across all primary metrics.
Question Generation Performance: DepPar achieved an F1 score of 0.630, significantly outperforming the H&S baseline of 0.444. The system's focus on simplicity and the "one-fact" rule contributed to higher precision compared to systems optimized for linguistic fluency.
Answer Extraction Performance: For identifying the correct answer phrase within the source text, DepPar achieved an F1 of 0.670, nearly doubling the performance of the H&S system.
Key Insights
- Brevity and Security: The proposed system generated questions averaging 7.2 to 7.5 words, closely matching the human baseline. Longer questions from other systems were found to be less precise and more prone to user error.
- Personal Entropy: Questions derived from unique personal memories provide a much larger "search space" for attackers than standard questions, inherently increasing security.
Conclusion
This research demonstrates that automated, personalized question generation is a viable and superior alternative to static security questions. By prioritizing "shallow" but unambiguous questions, the system enhances both security and user experience. Future work will explore machine-learning-based simplification rules and conduct human-in-the-loop studies to further evaluate the usability and naturalness of the generated questions.
Summary updated