Your RAG may be a data leak waiting to happen
An enterprise RAG connected to your internal document base is an information leakage vector if access controls are not implemented at the retriever level. An intern asking a question about salaries, a contractor accessing strategic documents, a user injecting instructions to bypass the guardrails: these scenarios are real and frequent. The three pillars of RAG security: access rights filtering at the vector store level, protection against prompt injection, and access auditing for AI governance. Without these controls, your AI assistant is an open door to your most sensitive data.
The problem
Most enterprise RAGs are deployed with a single vector index containing all documents: procedures, contracts, HR data, financial information, strategic documents. Vector search does not check access rights — it returns the chunks most similar to the query, regardless of their sensitivity.
The main attack vectors:
- Privilege escalation through the RAG — A user with limited rights uses the assistant to query confidential documents. A salesperson asks for the net margin of a project and receives an answer based on a confidential financial document. The RAG bypasses the existing access controls (SharePoint, Confluence, HRIS).
- Prompt injection — A malicious user manipulates their query to hijack the LLM. Indirect attacks are even more insidious: an attacker inserts malicious instructions into a document that will be indexed, and the LLM executes them when it reads the chunk.
- Exfiltration by inference — Through iterative questions ("Is the average salary above 50K? above 60K? above 65K?"), a user extracts sensitive information without ever directly accessing the documents.
- Residual data — The LLM may include in its answer fragments of confidential documents that were retrieved but not relevant to the question. The user receives information they should not have access to.
The AI solution
RAG security rests on three complementary layers of defense, from the query to the answer.
Access rights filtering (ACL)
Each chunk is tagged with the authorized user groups during indexing. The retriever filters by ACL before the vector search: only the chunks accessible to the current user are included in the top-K. Synchronize the ACLs with your directory (Active Directory, LDAP, Okta). Native implementation on Pinecone, Weaviate and Qdrant.
Protection against prompt injection
Three levels of defense: input sanitization (detection and blocking of malicious instructions), prompt hardening (system instructions resistant to injection with delimiters and refusal), and output filtering (verification that the answer does not contain out-of-scope data). Use Guardrails AI, NeMo Guardrails or Lakera Guard.
Audit and anomaly detection
Log each query and document access with the user metadata. Detect suspicious patterns: iterative queries on a sensitive topic, abnormal volume, repeated injection attempts. Automatic alerts for the security team. SIEM integration (Splunk, Sentinel, Elastic Security).
Implementation
Securing an existing RAG takes 4 to 6 weeks. Priority: access rights filtering, which covers 80% of the risk.
Rights mapping and ACL tagging (weeks 1-2)
Audit the current access rights of your document base (SharePoint, Confluence, DMS). Define the access groups by department, role and confidentiality level. Implement an indexing pipeline that automatically extracts the ACLs of each source document and associates them as metadata with each chunk. Test with 10 representative user profiles.
Protection against injection (weeks 3-4)
Implement three layers of defense. Input layer: a classifier detects injection attempts and blocks the query. Prompt layer: harden the system prompt with XML delimiters and an explicit refusal instruction. Output layer: verify that the answer does not mention out-of-ACL documents. Test with a red team of 5 standardized attack scenarios (OWASP LLM Top 10).
Security monitoring and audit (weeks 5-6)
Integrate the AI assistant's logs into your SIEM. Configure detection rules: more than 50 queries/hour per user, injection patterns, access to highly classified documents by a standard user. Schedule a quarterly pentest. Document the residual risks in your AI risk register.
Results
Results measured after securing enterprise RAGs in production.
FAQ
What is the risk of information leakage in a RAG?
A RAG indexes the entire document base in a single vector index. Without access control, a user can ask a question and receive an answer based on documents they normally don't have access to: salaries, strategic documents, client data. The retriever does not check rights — it returns the most similar chunks, regardless of their classification.
How do you implement access rights filtering in a RAG?
Two approaches: pre-retrieval filtering (each chunk's metadata includes the authorized groups, and the vector query filters by user_groups before the cosine similarity) and post-retrieval filtering (the retriever returns the top-K then a middleware filters). Pre-retrieval is more performant and safer. Natively supported by Pinecone, Weaviate, Qdrant and Milvus.
Are prompt injection attacks a real risk?
Yes. A malicious user can inject instructions to bypass the guardrails. Sophisticated attacks include indirect injection (instructions hidden in an indexed document) and system prompt extraction. The countermeasures: input sanitization, prompt hardening, and attack detection via a classifier (Lakera Guard, garak).
For technical profiles
RAG security architecture
RAG security is implemented in three layers: data layer (ACL on the chunks in the vector store), application layer (input validation, prompt hardening, output filtering), and monitoring layer (anomaly detection, SIEM integration).
ACL filtering — pre-retrieval implementation:
During indexing, each chunk receives a metadata allowed_groups: ["finance", "direction"] inherited from the source document (Active Directory / LDAP sync). On each query, the middleware extracts the user's groups from the JWT/session and adds a metadata filter to the vector store: filter={"allowed_groups": {"$in": user.groups}}. This filter is applied BEFORE the similarity computation.
Prompt hardening — best practices:
- XML delimiters to separate system instructions and user content.
- Explicit refusal instruction against prompt modification attempts.
- Scope restriction: answer only from the provided documents.
- Testing with standardized attack sets: OWASP LLM Top 10, garak.
Comparison of LLM security solutions
| Criterion | Guardrails AI + ACL | Lakera Guard | NeMo Guardrails | Custom |
|---|---|---|---|---|
| Injection detection | Good | Excellent | Good | Basic |
| Native ACL filtering | Via vector store | No | Partial | Custom |
| Output filtering | Yes (validators) | Yes | Yes (flows) | To be developed |
| Open source | Yes | SaaS | Yes (Apache 2) | N/A |
| Added latency | +20ms | +50ms | +40ms | +5ms |