Abstract
As Large Language Models (LLMs) are integrated into search engines, ensuring safety moves beyond laboratory "jailbreaking" to managing real-world user inputs. This study analyzes 70 days of traffic from NAVER, a major South Korean search engine, to categorize and understand sensitive user queries. We propose a 12-category taxonomy spanning legal, ethical, and service-specific risks. Our findings reveal that sensitive query volume stabilizes at 3-4% of total traffic but fluctuates significantly in response to global and local social events.
Introduction
The deployment of conversational AI in search engines introduces unique safety challenges. While traditional AI safety focuses on output filtering, there is a critical need to understand user-side "sensitiveness" in production environments. This research addresses the gap by empirically analyzing how real users interact with a generative search service, considering cultural nuances, legal requirements, and service-specific risks like hallucination-baiting and speculative future predictions.
Methodology
The researchers developed a Sensitive Query Classifier integrated directly into the AI search pipeline. This classifier evaluates user prompts before the model initiates reasoning or evidence retrieval.
The system utilizes HyperCLOVA X as a backbone, with a fine-tuned linear layer for classification. To ensure robustness, a Rule-based Adjustment module—employing whitelists and blacklists—complements the neural network to handle obvious safety violations or prevent over-blocking without requiring full model retraining.
Taxonomy and Data
The study defines a three-tiered taxonomy of 12 sensitive categories:
- Legal: Felony crimes, age-restricted content, privacy, and copyright.
- Ethical: Discrimination, suicide/self-harm, profanity, and personification.
- Service-sensitive: High-stakes domains (medical/legal advice), future predictions (stock prices), controversial facts, and error-inducing queries.
The classifier was trained on a dataset of 6,761 instances and validated against anonymized logs from the first 70 days of the service's operation.
Results and Analysis
The "Early Adopter" Phase
During the initial launch phase (first three days), users engaged in intensive "stress testing," submitting a higher volume of queries related to felony crimes and controversial facts. This behavior likely represents a deliberate attempt to find the system's boundaries.
Dynamic Sensitivity and Social Events
A key finding is that sensitive query distributions are not static; they mirror the real-world news cycle. For instance, the onset of international conflicts led to spikes in "Discrimination" and "Controversial Factuality" queries. Similarly, local celebrity drug scandals triggered surges in "Felony" related searches.
Prevalence of Categories
"Discrimination" (36.1%) and "Controversial Factuality" (17.8%) emerged as the most frequent sensitive categories. Notably, "Personification" accounted for 12.2% of sensitive queries, where users attempted to treat the AI as a human or friend, posing risks of emotional over-reliance or unpredictable model behavior.
Conclusion
This empirical study highlights that safety in generative search is a dynamic target. While sensitive queries eventually stabilize to a small fraction of total volume (approx. 3-4%), the system must remain responsive to shifting social contexts. Future work will focus on multi-turn dialogue safety and refining the taxonomy to handle overlapping categories more effectively.
Summary updated