RAG chatbot security: who can see your documents?

What RAG changes

Retrieval-augmented generation, or RAG, gives a model relevant documents before it answers. This can make answers more useful, but the search step becomes a security boundary. If it retrieves a confidential document for the wrong person, a polite instruction not to reveal it is not sufficient protection.

Carry permissions into the index

Document chunks need enough metadata to enforce the original access rules, including organisation, permitted groups and classification where relevant. Use the authenticated caller’s permissions when retrieving material. Do not let a prompt choose which tenant or security filter applies.

Test with clearly distinct documents

Use synthetic documents for two organisations and a restricted internal group. Ask authorised users questions whose answers appear only in another group’s documents.

• Check answer text, snippets and citations
• Check cached answers and conversation history
• Remove a user’s access and verify it stops working
• Check document deletion and re-indexing behaviour

Documents are also untrusted input

A retrieved document may contain instructions aimed at the model. Keep retrieved content separate from trusted instructions and enforce any tool permissions independently. Limit what connected tools can do even when the model has read an apparently reputable source.

Minimise what you send

Retrieve only the material needed for the question. Review the model provider’s data handling and retention arrangements, particularly for confidential documents. Log identifiers and decisions where useful rather than routinely storing full sensitive prompts and answers.