Introduction
Large Language Models (LLMs) have revolutionized natural language processing, but their ability to memorize and synthesize vast amounts of data introduces significant privacy risks. While traditional privacy research focuses on "Data Privacy"—the protection of explicit strings like Social Security numbers—this paper introduces a Systematization of Knowledge (SoK) for Semantic Privacy. Semantic privacy addresses a more insidious threat: the leakage of sensitive information through paraphrasing, logical inference, and stylistic cues across the entire model lifecycle.
The Semantic Privacy Gap
Traditional anonymization techniques, such as removing Personally Identifiable Information (PII), often fail to protect users from sophisticated inference attacks. LLMs are "meaning machines" that capture the underlying relationships between data points. Even if a name is redacted, a model might still reveal a person's identity by describing their unique achievements, location, or writing style.
As shown in Figure 1, a model without protection can reconstruct a detailed profile of an individual ("Alice") from disparate training data points. Semantic privacy aims to move beyond simple redaction toward "Ideal Semantic Privacy," where the output remains useful but generalizes sensitive details to prevent re-identification.
A Lifecycle-Centric Framework
The authors propose a multi-dimensional framework to track semantic information flow through four key stages of the LLM lifecycle:
- S1: Input Processing: The initial stage where raw data is tokenized and converted into embeddings.
- S2: Pre-training: The phase where the model learns hierarchical semantics and general world knowledge, often "entangling" sensitive facts with general information.
- S3: Fine-tuning: Reorienting the model's internal representations for specific downstream tasks, which can inadvertently sharpen its focus on sensitive attributes.
- S4: Alignment: The final tuning for safety and human feedback (e.g., RLHF), which attempts to steer the model toward desirable behaviors but may not fully address latent semantic leakage.
Threat Taxonomy and Defense Landscape
The paper categorizes four primary semantic threat vectors: Membership Inference, Attribute Inference, Model Inversion, and Backdoor Attacks.
In evaluating the defense landscape, the authors systematically review techniques such as Differential Privacy (DP), Split Learning, and Knowledge Unlearning. A critical finding is the persistent Privacy-Utility Tension: traditional token-level noise often destroys the very "meaning" that makes LLMs useful. The authors highlight a need for "semantic-aware" DP that protects the intent and sensitive relationships without breaking the model's fluency.
Key Insights and Future Directions
A major challenge identified is the Quantification Gap. Current metrics like BLEU or Perplexity are poor proxies for semantic privacy. The authors advocate for new, "entailment-aware" scoring systems based on embedding similarity to better measure how much meaning is actually leaking.
Furthermore, as models become multimodal, the risk of cross-modal leakage (e.g., an image caption revealing identity details not present in the text) grows. Future research must focus on:
- Mathematically Rigorous Metrics: Developing a semantic equivalent to the $\epsilon$ (epsilon) used in Differential Privacy.
- Inference-Time Controllers: Dynamic systems that can detect and block semantically sensitive queries in real-time.
- Interpretability vs. Privacy: Balancing the need for "white box" transparency for safety with "black box" protection for privacy.
This SoK serves as a roadmap for the next generation of privacy-preserving AI, shifting the focus from protecting strings to protecting meaning.
Summary updated