This page covers rehearsal. For how AI changes incidents, detection and identity, and a readiness checklist, see our AI incident response guide. Containment, evidence and Shadow AI incidents are covered in containing and investigating AI incidents.
A minute-by-minute AI-driven incident
A walkthrough makes the containment problem concrete. The scenario below is fictional and its times are illustrative, not benchmarks. It involves an AI agent with broad permissions, a credential leak, and a response that starts too slowly because nobody had mapped what the agent could touch.
- 00:00, overnight: An API key belonging to an internal AI agent, one built to automate customer data lookups for the support team, is harvested from a misconfigured logging endpoint by an automated credential-scanning bot. No human notices, because nothing about the theft itself generates an alert.
- 00:15: The stolen key is used to query the agent's API directly, bypassing the chat interface entirely. Because the agent's permissions were scoped to "whatever the support team might need" rather than the narrower set they actually use, the queries pull far more customer data than a support lookup ever would. The scenario assumes the agent's API did no authorisation checks of its own in the backend beyond the key: whoever held the key got everything the agent could reach.
- 00:45: Automated monitoring flags a volume spike on the API. The on-call engineer checks it, sees traffic that looks structurally similar to the agent's normal automated calls, and initially deprioritises it as a batch job running late.
- 02:30: A second, larger spike triggers escalation. The security team now has to determine whether this is the agent behaving unusually or a compromised credential being used against the agent's API, a distinction the existing alert did not make and the team has not had to make before.
- 03:10: Once compromise is confirmed, the team needs to revoke the agent's API key. Nobody has documented what breaks if they do. It takes another 20 minutes to establish that revocation will pause three customer-facing workflows, and to get the authority to accept that trade-off.
- 03:45: Legal and privacy are brought in to scope how much customer data was actually exposed, working from partial API logs because the agent framework does not retain full request payloads beyond 24 hours. In a better-prepared response, legal and privacy would have joined at 02:30, alongside containment rather than after it.
- 04:05: The agent's API key is revoked and the account disabled. The three customer-facing workflows that depended on it fail over to a manual process the support team last rehearsed at an earlier tabletop exercise, so the business keeps operating while the agent is offline.
- 05:00: Before the agent goes back online, the team checks what else the stolen key was used for beyond data lookups: whether new credentials were issued, permissions or automation flows changed, or improper actions taken in connected systems. They confirm the compromised key no longer works anywhere, restore the agent's configuration from a trusted source rather than the running copy, and put monitoring on the replacement credential.
- 06:30: A replacement credential is issued with permissions scoped to what the support team actually uses, not the broad grant that enabled the original exposure, and the agent is brought back online restricted to that narrower access.
- 09:00: Support operations return to the automated workflow under the new permissions. The manual fallback stands down, and the incident moves from containment into the investigation and notification work the earlier steps set in motion.
Every one of those delays traces back to a gap that existed before the incident started: no inventory of what the agent could access, no pre-agreed authority to revoke its credentials, no alerting that distinguished the agent's normal automated behaviour from abuse of it, and logging that did not retain enough detail for a clean investigation. None of that is a new discipline. It is the same incident readiness work your organisation should already be doing, extended to cover an asset type most plans currently ignore.
The same timeline matters for data protection law. An AI-related incident is not automatically a notifiable personal data breach. If customer personal data was accessed, though, UK GDPR requires you to notify the ICO within 72 hours of becoming aware of the breach, unless it is unlikely to result in a risk to people's rights and freedoms. Contracts and sector rules can add reporting duties of their own. Our guide to breach notification deadlines covers the detail.
Tabletop scenarios worth rehearsing
Most tabletop exercises still run ransomware, phishing and data breach scenarios written before AI-specific incidents were common. Those remain worth rehearsing, but they will not test whether your team can handle the scenarios below.
- Prompt injection through a customer-facing AI assistant: a crafted document or message manipulates your chatbot or AI assistant into revealing internal data or taking an unauthorised action. Who is notified, and who has authority to take the assistant offline? Decision owner: the service owner, with the incident lead. Information available: the assistant's conversation and tool call logs. Expected response: take the assistant offline or disable its tools, preserve the logs and assess what was disclosed. Evidence of success: a named person decides within the agreed time, and the disclosed data can be identified from the logs.
- A deepfake voice or video approval request: a convincing cloned voice or fabricated video call requests an urgent wire transfer or access grant. What verification step exists that does not rely on recognising the voice or face, and does everyone involved actually know it? Decision owner: whoever receives the payment or access request. Information available: only the call and the request details. Expected response: do not act until the request is confirmed through an independent, pre-agreed channel. Evidence of success: the participant uses the verification step without being prompted.
- A compromised, over-permissioned AI agent: a leaked credential is used against an internal AI agent's API, as in the scenario above. Who can revoke its access, and which services or integrations stop working when they do? Decision owner: the agent's business owner, with the incident lead. Information available: API logs, the agent register and the dependency map, if one exists. Expected response: revoke the credential, move dependent workflows to the manual fallback and check what else the key was used for. Evidence of success: the team can say within minutes what breaks and who approves the revocation.
- Shadow AI use during an active incident: a staff member, trying to resolve the incident faster, pastes sensitive incident data into an unapproved AI tool. Does your plan say anything about this, and does the comms team know to say it out loud during activation? Decision owner: the incident lead. Information available: a report or log showing the tool used and the data pasted. Expected response: stop the use, record what was shared and with which service, and treat it as a possible data incident. Evidence of success: the activation briefing names the approved tools, and the exposure is logged and assessed.
Run at least one of these alongside your existing scenarios at the next exercise. The goal is not a perfect answer. It is finding out where the gap is before it causes operational impact.