Ask an organisation what happens if an AI system produces a consequential error tomorrow, who’s accountable, and through what process. The length of the pause tells you most of what you need to know.
In firms that have thought about it, the answer is specific and practical. In firms that haven’t, it’s vague, and the vagueness is the finding.
The question that doesn’t get asked
Most of the attention in an AI programme goes on capability. What can it do, how fast, how well. The question that actually decides your exposure is a different one: what should it do, and where are the limits. Those limits get assumed rather than defined, which means they aren’t limits at all.
AI tends to get treated as another participant in an existing system rather than a genuinely different way of working that needs its own risk and governance layer. That layer is still early everywhere. Nobody has fully solved it. But the firms doing it well are treating it as a discipline in its own right, not a box to tick before the pilot goes live.
Accountability doesn’t transfer to the machine
There’s a temptation to treat an AI system as if it owns its own decisions. It doesn’t, and the law is starting to make that explicit. When an AI output causes harm, the organisation that deployed it is accountable, not the tool. Framing the system as a separate actor responsible for itself is not a defence anyone should rely on.
Which means the practical questions matter. Where is AI currently making or materially influencing decisions, and is that list complete and current? If it produced a systematically biased or incorrect output, how would you know? Who owns the fix? Most firms can’t answer these cleanly, and the ones deploying AI into regulated or client-facing work without those answers are operating in territory they haven’t mapped.
The regulatory floor is rising
The environment is moving faster than most firms’ governance. There are now mandatory safeguards around automated decision-making that affect individuals, requirements to be able to explain significant decisions and to offer meaningful human review. For firms touching EU employees or customers, a further compliance layer arrives in August 2026 that the domestic framework doesn’t automatically cover.
This isn’t a reason to panic. It’s a reason to know where AI is influencing decisions about people, because you can’t demonstrate compliance for something you haven’t mapped, and “we didn’t realise it counted” is not a position anyone wants to defend.
What good governance actually is
It’s not a document. It’s a set of answered questions. What is AI permitted to do here, and what is it not. Who’s accountable when it’s wrong. How you’d detect a systematic error. What the escalation path is. Who owns the boundary, and who’s allowed to move it.
The firms that answer these before deployment protect themselves and, not incidentally, give the AI a much better chance of working. The ones that treat governance as something to sort out later usually sort it out during an incident, which is the most expensive possible time to start.