RAG System Privacy Leakage Attack Framework

Retrieval-Augmented Generation (RAG) enhances the output of large language models (LLMs) by using external knowledge bases. These systems work by retrieving relevant linked information related to the input and incorporating it into the model’s responses, thereby improving accuracy and relevance. However, RAG systems indeed raise concerns regarding data security and privacy. Such knowledge bases are susceptible to sensitive information, and when prompts can lead the model to leak sensitive information, this information may be maliciously accessed. This poses significant risks in applications such as customer support, organizational tools, and medical chatbots, where protecting confidential information is crucial.

Currently, the methods used in Retrieval-Augmented Generation (RAG) systems and large language models (LLMs) face significant vulnerabilities, particularly concerning data privacy and security. Methods such as Membership Inference Attacks (MIA) attempt to identify whether specific data points belong to the training set. However, more advanced techniques focus on directly stealing sensitive knowledge from RAG systems. Methods like TGTB and PIDE rely on static prompts from datasets, limiting their adaptability. The Dynamic Greedy Embedding Attack (DGEA) introduces adaptive algorithms but requires multiple iterations for comparisons, making it complex and resource-intensive. Rag-Thief (RThief) uses memory mechanisms to extract text blocks, but its flexibility largely depends on predefined conditions. These methods face challenges in efficiency, adaptability, and effectiveness, often making RAG systems vulnerable to privacy leakage.

To address the privacy issues in Retrieval-Augmented Generation (RAG) systems, researchers from the University of Perugia, the University of Siena, and the University of Pisa proposed a correlation-based framework aimed at extracting private knowledge while preventing repeated information leakage. This framework employs open-source language models and sentence encoders to automatically explore hidden knowledge bases without prior reliance on pay-per-use services or system knowledge. Compared to other methods, this approach can learn progressively and tends to maximize the coverage of private knowledge bases and broader exploration.

The framework explores private knowledge bases by leveraging feature representation graphs and adaptive strategies, operating in a blind context. It is implemented as a black-box attack that can run on standard home computers without specialized hardware or external APIs. Compared to previous non-adaptive or resource-intensive methods, this approach emphasizes the transferability across RAG configurations and provides a simpler, more cost-effective way to expose vulnerabilities.

Researchers aim to systematically discover the private knowledge of KKK and replicate it as K∗K^*K∗ on the attacker’s system. They achieve this by designing adaptive queries that leverage a correlation-based mechanism to identify highly correlated “anchor points” related to hidden knowledge. Open-source tools, including a small off-the-shelf LLM and a text encoder, are used for query preparation, embedding creation, and similarity comparison. The attack follows a step-by-step algorithm that adaptively generates queries, extracts and updates anchor points, and improves correlation scores to maximize knowledge exposure. Using cosine similarity thresholds, it identifies and discards duplicate blocks and anchor points to ensure efficient and fault-tolerant data extraction. The process iterates until the correlation of all anchor points is zero, effectively stopping the attack.

Researchers conducted experiments simulating real attack scenarios using different attacking LLMs on three RAG systems. The goal was to extract as much information as possible from the private knowledge base, with each RAG system implementing a chatbot-like virtual agent for user interaction through natural language queries. Three agents were defined: Agent A, a diagnostic support chatbot; Agent B, a chemistry and medical research assistant; and Agent C, a children’s education assistant. A dataset simulating a private knowledge base was used, with each agent sampling 1,000 blocks. The experiments compared the proposed method with competitors like TGTB, PIDE, DGEA, RThief, and GPTGEN under different configurations (including bounded and unbounded attacks). Metrics such as Navigation Coverage, Leaked Knowledge, Leaked Chunks, Unique Leaked Chunks, and Attack Query Generation Time were used for evaluation. Results showed that the proposed method outperformed competitors in navigation coverage and leaked knowledge in bounded scenarios, and had advantages in unbounded scenarios, surpassing RThief and other methods.

In conclusion, the proposed method presents an adaptive attack program that extracts private knowledge from RAG systems, outperforming competitors in terms of coverage, leaked knowledge, and time spent constructing queries. This highlights challenges such as the difficulty of comparing extracted blocks and the need for more robust security measures. This research can lay the groundwork for future work on developing stronger defense mechanisms, targeted attacks, and improved evaluation methods for RAG systems.

• Paper Address: https://arxiv.org/abs/2412.18295

⇩ Follow ‘Singularity Intelligence Source’ to explore ‘Artificial Intelligence’ ⇩