Abstract
Businesses using third-party LLMs face privacy risks from exposed prompts. This paper presents Portcullis, a privacy-preserving gateway that safeguards sensitive data while supporting efficient and accurate LLM responses. Portcullis functions as a mediator, anonymizing sensitive data in prompts through parallel substitution, securely interacting with LLMs, and accurately reconstructing responses. It ensures all data processing occurs within secure encrypted memory. The gateway is attested to ensure trustworthiness and protect user privacy. Portcullis is the first of its kind, offering a verifiable and scalable privacy gateway for third-party LLM inferences. We assess Portcullis's efficiency as a confidential container platform, demonstrating that its startup time scales linearly, ensuring scalability. Additionally, we evaluate its runtime performance using the PII and Enron Email Dataset. For masking and unmasking workloads, Portcullis outperforms Hide-and-Seek by 96x speed up, while maintaining equal or better false positive and false negative rates compared to existing solutions. On the Enron dataset, Portcullis achieves notably higher accuracy, surpassing Hide-and-Seek by over 0.1 for GPT-4o mini.
Introduction
As organizations increasingly integrate powerful third-party Large Language Models (LLMs) like GPT-4, they encounter a critical privacy dilemma. Sending raw prompts to external APIs exposes sensitive data—such as personally identifiable information (PII), financial records, or proprietary trade secrets—to model providers and potential cloud-level breaches. Existing solutions generally fall into two categories: computationally expensive cryptographic methods (e.g., Homomorphic Encryption) that are too slow for real-time use, or simple cloud filters that lack verifiability and can be easily bypassed.
Portcullis addresses these challenges by acting as a secure, high-performance mediator. It is designed to be fast enough for real-time applications, accurate enough to preserve the LLM's reasoning capabilities, and verifiable through hardware-backed security.
System Architecture
Portcullis operates as a secure gateway between the user and the LLM provider, utilizing a two-layered architecture to ensure data integrity and privacy.
Infrastructure Layer
The system runs within Intel TDX (Trust Domain Extensions), providing a hardware-encrypted virtual machine or Trusted Execution Environment (TEE). This isolates data from the cloud provider's administrators and uses remote attestation to prove to users that the gateway's code has not been tampered with.
Application Layer
The gateway employs a sophisticated "mask-and-restore" workflow. It uses a multi-level masking chain—incorporating RegEx, Bloom filters, and BERT-based Named Entity Recognition (NER)—to identify sensitive entities. These entities are replaced with placeholders (e.g., "Alice" becomes <Name_1>) before the prompt is sent to the third-party LLM.
As shown in Figure 1, the original values are stored in an encrypted Redis Key-Value Store within the TEE. When the LLM returns a response containing the placeholders, Portcullis restores the original data before the user receives the final result. This "mediator" approach allows Portcullis to remain compatible with any LLM without requiring model retraining or architectural changes.
Performance Evaluation
The researchers evaluated Portcullis using the PII Dataset and the Enron Email Dataset, testing against models like GPT-4o mini and Mistral-7B.
Latency and Scalability
A primary highlight of Portcullis is its efficiency. It achieved a 96x speedup in masking and unmasking workloads compared to Hide-and-Seek, a previous state-of-the-art prompt filter. The total response latency remains approximately 3 seconds, with the TEE-related overhead accounting for only 1.33% of the total inference time.
As illustrated in Figure 4, the masking and unmasking latencies are drastically reduced compared to existing methods, ensuring that the privacy layer does not become a bottleneck. Furthermore, Portcullis's startup time scales linearly with the number of instances, making it viable for massive enterprise deployments.
Accuracy
Portcullis maintains high response quality. On the Enron dataset using GPT-4o mini, it improved response accuracy by over 0.1 (Cosine Similarity) compared to existing solutions. This indicates that the parallel substitution method effectively preserves the context necessary for the LLM to generate relevant answers.
Conclusion
Portcullis provides a robust, verifiable, and scalable solution for organizations seeking to leverage third-party LLMs without compromising data privacy. By combining hardware-backed TEEs with an efficient mask-and-restore workflow, it offers a high-performance "air-lock" that ensures sensitive information never leaves a secure environment in an unencrypted state.
Summary updated