When AI Makes Decisions During A Cyber Incident

The first incident may be the attacker; the second may be the response.

An AI-enabled security system identifies behaviour that appears malicious and acts before a human investigation can establish the complete incident boundary. It blocks an identity, revokes a session, isolates a workload, closes a route or terminates a process. The dashboard records decisive action, yet the same intervention may interrupt care, stop production, delay settlement, remove a recovery path or disconnect the people trying to stabilise the incident.

The attraction is understandable. Attackers are using automation to accelerate reconnaissance, vulnerability analysis, credential abuse and movement through connected systems. A response that waits for a complete manual investigation may arrive after the consequence. Faster analysis and tightly bounded automation can therefore add real defensive value.

The risk begins when speed is granted more authority than the available context can support. A machine can make a technically plausible decision before your teams understand what depends on the account, route or infrastructure it is about to change.

AI systems interpret patterns, assign confidence and choose an action under uncertainty. The physical and operational consequence is less ambiguous. An identity is disabled, a route is closed, a server becomes unreachable or a shared virtual environment is isolated.

In financial services, an agent may correctly identify abnormal administrator activity but respond across an identity scope that also supports settlement, reconciliation or treasury workflows. In healthcare, isolating a shared workload may stop a suspicious path and simultaneously withdraw several clinical-support services. In manufacturing or energy, closing a remote-access route or network segment without understanding its dependencies may remove the engineering visibility or specialist access needed to operate safely.

None of those scenarios requires the AI to malfunction in an obvious way. The model may have recognised a genuine threat and still chosen an intervention whose operational radius was too broad. Technical plausibility is therefore not the same as a safe outcome.

The most dangerous automated mistake may initially look like success. If the objective is to stop ransomware spread at any cost, disconnecting every reachable environment may satisfy the instruction. If the objective is to prevent information leaving, closing every relevant data route may appear equally successful. If the objective is to remove suspicious access, disabling all related identities may produce a clean security result.

The ransomware did not spread. The data route closed. The identities stopped working. Yet care, production, payment, logistics or recovery may have stopped with them.

The system met its objective while your business absorbed the failure. The weakness lies not only in the model, but in an operating objective that omitted the boundaries, dependencies and exceptions needed to keep the action proportionate.

Good defensive automation therefore begins with a more exact question: which harmful behaviour must stop, at which technical layer, across what maximum scope and with what route back if the action proves wrong?

A serious cyber incident does not hold still. Early signals are incomplete, assumptions change, identities are reset, systems are isolated and dependencies emerge only after the response has begun. An action that was reasonable ten minutes earlier may become harmful when the operating picture changes.

An agent may continue to optimise around the first visible problem, overweight an early assumption or lose a weak but important signal inside a growing volume of new context. People can make the same mistakes, but a machine can repeat them across more systems and at greater speed.

Context therefore needs an expiry point. High-impact actions should not inherit unlimited authority from an assessment that is no longer current. The operating model needs a way to pause, narrow or reverse the action when the incident moves beyond the assumptions that justified it.

The emergency brake is not a meeting convened after an automated action has already changed the business. It is a designed ability to stop, narrow or reverse machine action, connected to clear authority and a technically executable control.

Low-impact, local and reversible actions may be suitable for automation. Actions that alter critical services, shared infrastructure or enterprise-wide access require a different threshold. The distinction must exist before the incident because the time to invent it disappears once the agent and attacker are both moving.

The Human Decision Layer develops the detailed models for human-in-the-loop, human-on-the-loop and human-out-of-the-loop control. The argument here establishes why those models matter: if the response can create a second incident, someone must be able to stop it before the consequence becomes irreversible.

The relevant question is no longer only whether AI can recognise a threat or recommend a response. It is whether your business remains controllable when an automated action is plausible, fast and wrong.

That question moves the discussion from model capability to operational consequence. It asks which actions can be safely automated, how far they may reach, what assumptions support them, how quickly they can be reversed and who can intervene before the response becomes the larger incident.

AI will become part of cyber defence because human teams cannot manually process every signal at machine speed. The advantage will belong to businesses that use that speed without surrendering control over what the action is allowed to change.