SecureLLM.ai

Home › Can we prove who saw what?

Can we prove who saw what?

Keeping a record: who asked what, and what they were shown

Updated November 2026 · About 5 minutes to read

The short answer

If something goes wrong, you'll need to say exactly who asked what, when, and what they were shown. That's only possible if the assistant keeps a record from day one.

A good record notes who, when, where and which documents were used for each answer. It's kept long enough to be useful, only a few people can read it, and it doesn't become a second copy of everything sensitive.

Think of a hotel's key-card system

The hotel knows which card opened which door, and when. It doesn't film what happened inside the room.

If a guest complains that something went missing, the manager can check who entered. And the door log itself is kept locked away, not pinned up in the lobby.

Where the picture stops working: an AI answer can carry a document's contents out of the room, so the record also needs to show which documents each answer drew on.

What a useful record contains

The OWASP security foundation's rule for any software: logs "must record "when, where, who and what" for each event" (OWASP Logging Cheat Sheet). For an AI assistant, that means:

  • Who asked, and when. The signed-in person, not just "the assistant".
  • Which documents the answer used. Microsoft's records for Copilot, for example, include "references to files, sites, or other resources" it used to answer (Microsoft Learn; Microsoft sells this auditing tool).
  • How sensitive those documents were. Microsoft's record notes each file's sensitivity label, which it says helps in "identifying whether Copilot accessed any sensitive information".
  • What was blocked. The same records note when access to a file "was blocked or restricted based on some policy". Refusals matter: repeated attempts to reach HR files are a warning sign.
  • Any action it took. If the assistant can send or change things, what it did and for whom. When the assistant can do things, not just answer.

An example record

Made up for illustration. Real records vary by product.

Who
A sales manager, signed in
When
Tuesday, 9:42 a.m.
Where
The assistant in the company's chat app
Question
Kept with the conversation, not copied here
Documents used
Price list 2026 (everyone)
Sales playbook (Sales)
Blocked
Salary bands 2026 (HR only): not available to this person
Actions taken
None

With that, "did anyone outside HR see the layoff plan?" becomes a search, not a guess.

How a well-run record is kept

  • It's switched on before launch. A record can't be created after the fact.
  • It's kept long enough. Problems are often found months later. In Microsoft 365, audit records are kept "for 180 days by default", and keeping them longer, up to 10 years, depends on the license (Microsoft Learn). Check what your plan includes.
  • Few people can read it, and reading it is recorded too. OWASP: "All access to the logs must be recorded and monitored", and the right to read them "should be restricted and reviewed periodically" (OWASP).
  • It points to the conversation rather than copying it. A record that lists the question, the person and the documents used can answer most questions without storing every word of every answer in yet another place.
  • It leaves out what it shouldn't hold. OWASP lists things that usually shouldn't be written into logs directly, including "Authentication passwords", "Access tokens", "Sensitive personal data" and "Commercially-sensitive information".

How long you must keep records, and what privacy law allows, are legal questions covered on our sister site EnterpriseLLM.ai.

When the record becomes the leak

DeepSeek, January 2025

Security researchers found databases belonging to the Chinese AI company DeepSeek open on the internet. They "held over a million log entries containing user chat history in plaintext form", and anyone could search them without a password. DeepSeek closed the exposure quickly after being told (BleepingComputer; found by Wiz, which sells cloud security tools).

  • Logs are a target. OWASP warns that "logs may be a target of attacks" and "may contain personal and other sensitive information" (OWASP).
  • No record, no answer. Without a record, after a suspected leak you can't say who saw what, so you may have to assume the worst.
  • Records nobody looks at. A record helps after the fact. Reviewing blocked attempts and unusual searches helps you catch problems sooner.
  • Records are another copy. When something is deleted, its traces in the record may need clearing too. Deleting something everywhere.

Questions to ask your IT team or vendor

  1. For each answer, does the record show who asked, when, and which documents were used?
  2. Does it record blocked attempts, and does anyone review them?
  3. How long are records kept, and does that depend on our license?
  4. Who can read the records, and is their reading recorded too?
  5. Do the records copy full questions and answers, or point to them? Where are those kept, and how are they protected?
  6. If we suspected a leak tomorrow, how quickly could you tell us who saw a given document?

Other names you may hear

Audit log, audit trail
The record of who did what, and when.
Prompt and response logging
Keeping the questions asked and the answers given.
Citations, sources, grounding data
The documents an answer drew on.
Retention period
How long records are kept before they're deleted.
SIEM
Software that gathers logs from many systems so security staff can search and monitor them.
eDiscovery
Searching company records for a legal case or investigation.
Traceability
Being able to trace an answer back to who asked and what it used.