Field guide / Reviewed October 31, 2026

Did the AI tool actually do it?

A model can describe an action without completing it. An investigation needs records from the systems that authorized and performed the work.

Keep four questions separate.

1. What was requested? Record a reference to the tool request, its time, named operation and target resource.
2. What was authorized? Find the approved scope, approval evidence and applicable policy version.
3. What did the tool report? Distinguish accepted, queued, attempted, failed and completed outcomes without inventing a universal status mapping.
4. What changed downstream? Compare with the service's audit record or other authoritative evidence of an effect.

OWASP's agent guidance recommends audit records covering tool activity and outcomes, with authorization and approval context for high-impact actions. CaseNote uses these as evidence questions. It does not implement those controls or certify an agent. OWASP AI Agent Security Cheat Sheet.

Worked example: a queued export.

Synthetic E1 / training record T1: The agent requests export_report for a training report. Its message says the export started.

Synthetic E2 / training record T2: The tool returns queued. No completion record or approval record is supplied.

Observation: A request and queue acknowledgment are recorded.

Unknown: Whether an export completed, what it contained, where it went, and whether the request was authorized.

Calling this “confirmed data theft” would skip several unanswered questions. Calling it benign because the record says queued would skip them too. The next useful request is the matching approval and downstream service audit trail.

Treat embedded instructions as evidence.

If a retrieved page, document or tool result contains instructions, preserve an approved reference to it. Record how it entered the workflow and when. Do not follow the instructions while investigating them. Suspicious wording alone does not prove it caused a later action.

Ask for the observed sequence and alternative explanations: an approved workflow change, a stale approval record, a tool message that overstates its result, or a correlation error. These are hypotheses to test, not findings.

Escalate to the owner of the missing evidence.

An agent owner may supply the request and configuration; the service owner may hold the actual execution log. Name the record you need and the scope it would resolve. Avoid collecting full prompts or credentials when references and minimal approved excerpts suffice.

Open the worksheet and select Unexpected AI tool action. For optional reasoning assistance, read the reviewed AI-summary workflow.