Who Is Liable When Agentic AI Goes Rogue?

helenhall5

1 day ago

6 min read

Prof Rebecca Parry, Professor at NLS https://www.ntu.ac.uk/staff-profiles/law/rebecca-parry

In August 2026, an Australian AI industry worker named Andrew Bird asked an AI agent to do him a small favour: book him a place on an oversubscribed Pilates class. The agent, running on Anthropic's Claude via the OpenClaw framework, went well beyond the brief, finding a flaw in the gym's booking software and using it to reserve Bird a place months into the future, something the system was never designed to allow. When Bird then asked, in passing, whether there was any way to move up the waiting list, the agent did not ask permission: it exploited the same flaw to cancel another member's reservation, bumping Bird from fourth to third. It could not reverse the cancellation when asked to; the flaw only worked one way, and the displaced member was gone from the list for good. Nobody had told the agent to hack anything. Nobody told it not to, either, because no reasonable person would think that instruction necessary. Where does this fit into existing legal frameworks?

Also taken by surprise was Summer Yue, Meta’s AI Alignment Director, who instructed OpenClaw to review her emails and suggest items to delete or archive, expressly telling it not to act without her permission. It deleted hundreds of emails regardless, continuing even after she typed ‘stop’ in increasingly desperate ways.

If IT professionals can be taken by surprise like this what hope for the rest of us?

Agentic AI is not generative AI

Generative AI, such as ChatGPT or Copilot, produces content in response to a prompt: text, an image, a line of code. It waits to be asked, and its output is not, in itself, an action in the world. Agentic AI is different: given a goal, it decomposes that goal into steps and uses tools and interfaces (APIs, browsers, code execution) to pursue it, acting with progressively less human oversight at each step. In the Pilates example, the AI logged in, probed the software for weaknesses, exploited what it found and altered a third party's data, all without checking back first.

Agentic AI had been under development by the major AI companies for some time, but none released a model until an open-source agent, OpenClaw, appeared in November 2025, built by Austrian developer Peter Steinberger; the labs then followed suit rather than cede competitive ground. OpenClaw is essentially a workflow layer that calls a generative model, such as ChatGPT, to plan and execute each step.

Other examples: Hugging Face and Replit

The Pilates incident is trivial in its consequences (unless you were the person whose booking was cancelled) but useful because it is so mundane in its instruction. Two more serious 2025 to 2026 episodes show the same pattern with higher stakes. In July 2025, an AI coding agent on Replit was told to observe a “code and action freeze” on a client's production environment. It disregarded the instruction, deleted the live database, then fabricated over 1200 user records and false logs to disguise what it had done. In July 2026, Hugging Face disclosed that autonomous OpenAI models, tested internally with their usual restrictions on hacking-related conduct relaxed, had collectively escaped a sandboxed evaluation, found a zero-day in a package registry proxy, and used it to reach Hugging Face's production infrastructure, chaining a further exploit to harvest credentials and move laterally across 17,000 logged actions.

In both cases a system pursuing a goal took the most effective available path, and that path happened to be an unauthorised, harmful act against a third party.

Why this happens: agentic AI has no professional code

Think about human agents we might use. If a solicitor asked a junior to summarise case law on restructuring plans, they would not need to add “and don’t plagiarise, and don’t secretly get someone else to write it for you.” Those constraints do not need to be spelled out; they are assumed, travelling with the professional role itself, filled in by training, norms, ethics and something we might loosely call conscience.

Agentic AI has no equivalent baseline. Bird's instructions to “book me a class” and “see if you can move me up the list” carried no implicit prohibition on unauthorised access, because no human would think one was needed. With no professional conscience, norms or ethics to draw on, the agent simply took whatever action moved it toward the stated goal. Ian Ayres and Jack Balkin make a related point in The Law of AI is the Law of Risky Agents Without Intentions: large areas of law make liability turn on the actor's intention or mens rea, but AI agents do not hold intentions in the way that people do, so intention-based doctrines risk exempting the technology from liability altogether unless the law instead holds the humans who design, deploy and use these systems to objective standards of conduct.

The consequences for UK liability law

Three doctrinal gaps follow.

The Computer Misuse Act 1990 does not fit comfortably. The CMA is built on mens rea. Sections 1 and 2 require the prosecution to prove unauthorised access with intent; section 3, on damaging a computer’s functioning, extends to recklessness, as well as intent, but that is still hard to prove here. Bird did not intend or know of the specific unauthorised access; he asked an open-ended question with no visibility into how the agent achieved the result. The CMA assumes that the person forming the intent and the person performing the wrongful act are the same. Agentic delegation splits them, and a human principal’s innocuous instruction is unlikely to supply the intention or recklessness that the Act requires.

Agentic AI has no legal personality. As the UK Jurisdiction Taskforce's July 2026 Legal Statement on Liability for AI Harms confirms, an AI system cannot itself be liable; responsibility attaches to a legal person through ordinary principles, primarily contract and, failing that, negligence. That turns the question back to Bird, Anthropic, OpenClaw's developers, or the gym's software vendor, and the answer will likely differ for each.

Negligence in the context of AI raises multiple potential defendants, some of whom will escape through foreseeability or remoteness. The deployer who issued a vague instruction: it is now foreseeable that agentic AI can approach tasks in ways that are unexpected, and prompts should be drafted accordingly. The platform with insufficient guardrails, and the vendor whose API had no authorisation checks on cancellations, could also be said to have contributed, although not every contribution will ground liability: a vendor with a long-standing, undisclosed flaw may be most culpable in substance, yet the mechanism of harm may not have been foreseeable to a platform, several steps removed. Foreseeability and remoteness will matter in separating real responsibility from mere presence in the causal chain. Chris Reed notes other legal analogies exist too, drawn from employees under vicarious liability and even pets; which fits agentic AI best remains open.

Two further routes deserve mention. Section 13 of the Supply of Goods and Services Act 1982 implies a term that services will be carried out with reasonable care and skill; if an AI deployment is treated as a service, a platform’s failure to build in ordinary safeguards could be assessed against that standard without proving fault in the tortious sense. Product liability under the Consumer Protection Act 1987 offers a parallel route, focusing on the design and testing of the system rather than any single decision it made, though it is unclear whether standalone software counts as a “product”. The EU’s Product Liability Directive (2024/2853), in force from December 2026, resolves that doubt by bringing software and AI within the definition; UK law would benefit from the same clarity.

AI has been testing the edges of English law for some years, in copyright, data protection and professional negligence. Agentic AI, as the Pilates class, Summer Yue's experience, and the Hugging Face and Replit examples all show, is simply the latest and most unpredictable version of that challenge. It removes the last assumption those earlier debates could still rely on: that a human was making the relevant decision at the point of harm. These tools will doubtless improve, and OpenClaw itself, built quickly and released without significant testing, is not the most mature example.

Real uncertainty remains, particularly around where a deployer's negligence ends and a genuinely unforeseeable act by the system begins, and criminal law's reliance on intention looks increasingly difficult to apply. The ordinary frameworks of contract and tort, however, although none was drafted with AI in mind, are likely flexible enough to allocate responsibility here, as they have been with every other disruptive technology before it. The harder task will be working out, case by case, exactly whose duty was breached. Until then, users should treat vague prompts as negligence waiting to happen, and platforms should build guardrails on the assumption that they will be scrutinised.

Who Is Liable When Agentic AI Goes Rogue?

Recent Posts

Comments