Abstract
Knowledge files have been widely used in large language model (LLM) agents, such as GPTs, to improve response quality. However, concerns about the potential leakage of knowledge files have grown significantly. Existing studies demonstrate that adversarial prompts can induce GPTs to leak knowledge file content. Yet, it remains uncertain whether additional leakage vectors exist, particularly given the complex data flows across clients, servers, and databases in GPTs. In this paper, we present a comprehensive risk assessment of knowledge file leakage, leveraging a novel workflow inspired by Data Security Posture Management (DSPM). Through the analysis of 651,022 GPT metadata, 11,820 flows, and 1,466 responses, we identify five leakage vectors: metadata, GPT initialization, retrieval, sandboxed execution environments, and prompts. These vectors enable adversaries to extract sensitive knowledge file data such as titles, content, types, and sizes. Notably, the activation of the built-in tool Code Interpreter leads to a privilege escalation vulnerability, enabling adversaries to directly download original knowledge files with a 95.95% success rate. Further analysis reveals that 28.80% of leaked files are copyrighted, including digital copies from major publishers and internal materials from a listed company. In the end, we provide actionable solutions for GPT builders and platform providers to secure the GPT data supply chain.
Introduction
Custom GPTs allow builders to upload "knowledge files"—ranging from textbooks and proprietary code to sensitive internal records—to provide the model with specialized context. While OpenAI implements basic safeguards to prevent users from downloading these files, security research has historically focused on a narrow "black-box" view: prompt injection. This approach attempts to trick the model into reciting file contents through dialogue.
However, GPTs function as modern web applications with a complex data supply chain involving clients, servers, and databases. This research argues that focusing solely on prompt-based leakage is insufficient. Even if a model refuses to "speak" the contents of a file, the underlying infrastructure may inadvertently leak data through metadata, background communication flows, or sandbox vulnerabilities.
Methodology
To systematically evaluate these risks, the researchers adopted a workflow inspired by Data Security Posture Management (DSPM), a framework typically used by enterprises to manage data risks. The approach followed four distinct phases:
- Data Discovery: Identifying where knowledge file data resides, specifically within Metadata, Web Socket "Flows," and Model Responses.
- Data Classification: Categorizing leaked data into dimensions such as File ID, type, size, title, and full content.
- Risk Assessment: Identifying specific leakage vectors and analyzing the "ground truth" to distinguish between real leaks and model hallucinations.
- Mitigation: Testing defense prompts and architectural changes to secure the supply chain.
To gather data at scale, the team utilized GPTracker to scrape the GPT Store and intercept "flows"—the structured JSON messages exchanged between the user's browser and OpenAI's servers via web sockets.
Empirical Dataset
The study is supported by a massive empirical dataset comprising:
- Metadata: 651,022 GPTs scraped from the GPT Store.
- Flows & Responses: 11,820 web socket flows and 1,466 model responses from interactions with unique GPTs.
- Original Files: 566 original PDF files successfully extracted via exploits to analyze for copyright infringement.
- Cross-Platform Validation: The workflow was also applied to the Poe and FlowGPT platforms to ensure findings were not limited to a single provider.
Key Insights and Results
The Infrastructure is the Leak
The study identified five primary leakage vectors: metadata, initialization flows, retrieval flows, sandboxed execution environments (SEE), and prompts. Crucially, data exposure often begins before a user even sends a prompt. The "GPT Initialization" flow, sent automatically upon opening a GPT, contains metadata fields with the IDs, titles, and sizes of all uploaded files.
The Code Interpreter Privilege Escalation
The most alarming finding is a privilege escalation vulnerability within the Code Interpreter. While OpenAI restricts users from downloading files from the /mnt/data/ directory, researchers found that asking the GPT to "copy [FILE] to a new file and send a download link" bypasses these controls. The system generates a new link for the copy, which the user can then download. This attack achieved a 95.95% success rate.
Retrieval-Augmented Generation (RAG) Leaks
The myfiles_browser tool used for RAG leaks full file content into background web socket flows. The study found that the system prioritizes smaller files; if the total token count exceeds approximately 100k, larger files are excluded from the flow, effectively hiding them from this specific vector but not from others.
Copyright and Impact
Analysis of the extracted files revealed that 28.80% were copyrighted materials. This includes digital copies of textbooks from major publishers like Springer and Elsevier, as well as internal corporate documents from a listed company.
Mitigation and Conclusion
While "defense prompts" (instructions telling the GPT not to share files) offer some protection, they are easily bypassed by sophisticated users. The most effective immediate mitigation for builders is disabling the Code Interpreter if it is not strictly necessary for the GPT's function.
Ultimately, platform-level changes are required. Redesigning APIs to filter sensitive metadata and hardening the sandboxed execution environment are essential steps to securing the GPT data supply chain against persistent leakage vectors.
Summary updated