Why chat history is not enough
Most AI tools keep a conversation history. That is useful, but it is not an audit trail. It lives in the AI provider’s service, it can often be deleted by the user, it may not show exactly which data was retrieved, and it is split across however many AI tools your organisation uses.
An audit trail answers a different question: what actually happened to our information and systems? That is best recorded where the AI reaches your systems, not inside the AI.
What to record
For every AI request, aim to capture:
- Who. The person the AI was acting for.
- Which agent. The AI client or agent that made the request.
- When. A reliable timestamp.
- What resource. The system, record or document involved.
- What action. Read, create, update, send, publish, delete.
- The decision. Allowed or denied, and which rule decided it.
- Approval. Whether a person approved, and who.
- The outcome. What changed, including identifiers of anything created.
Record denied requests as well as allowed ones. A pattern of refusals is often the first sign that an agent is misconfigured, or that someone is testing the boundaries.
Where the records come from
- Business systems’ own logs. Microsoft 365, accounting systems and code platforms keep their own audit logs. These show what happened in that system, but often only as the connector’s account, not the person behind it.
- Connector or MCP server logs. Useful, but each connector logs differently, and some barely log at all.
- A central gateway. If every AI request passes through one layer, it can record every request in one consistent format, with the person and the agent attached.
Making the trail useful
- Keep it tamper-evident. People being audited should not be able to edit the record.
- Decide on retention. Keep records for as long as your regulatory and contractual obligations require, and no longer than you need.
- Make it searchable. “Show me everything any AI did on ABC Limited last month” should be a quick question.
- Avoid over-collection. Record requests and outcomes. You rarely need to store the full text of every private conversation, and doing so raises its own data protection questions under UK GDPR.
A short checklist
- Can you list every AI tool connected to your systems?
- Can you show what a given agent did last week, and for whom?
- Can you show which requests were refused and why?
- Can you show who approved a specific email, payment or publication?
If any answer is “no”, start there.
How Kroy approaches it
Every request that passes through Kroy, allowed or denied, is recorded: who, which agent, what resource, what action, which policy decided it, any approval, and what happened. Because Claude, ChatGPT, Copilot and other AI clients all reach your systems through Kroy, the record is in one place and one format. Kroy Observe presents it as an organisational view rather than employee surveillance.