AI in regulated work: how to keep a named human accountable.

In banking, wealth management and any regulated business, the question is never only whether AI works. It is who answers for it.

Accountability is a design decision

In Camunda's September 2026 survey, 40% of organisations said they had an AI-related compliance or governance issue in the past 12 months, and process-related challenges contributed to 84% of those incidents. Compliance and risk operations was also the process organisations most wanted to redesign, named by 68%.

Read together, those numbers say something specific. Most AI governance failures are not caused by a rogue model. They are caused by a workflow where nobody wrote down who decides what.

The three-tier control model

For every workflow, agree three lists with the business owner and compliance before anything is built:

  • AI does alone: reading, matching, ranking, summarising and recording. Reversible steps with a log.
  • AI drafts, a person approves: anything a client, customer or regulator will see, and any decision with money or risk attached.
  • AI never: decisions written down in advance as human-only.

The third list matters most. It is short, it is specific, and it is the first thing an auditor will ask for.

Three use cases that put a person at the decision

The relationship manager's client pack

AI drafts the client review pack before the meeting, with a source on every line and the lines compliance would question flagged. The RM edits and owns the pack, and compliance reviews the flagged lines. Every version is kept, so the pack that was actually used can be found a year later.

Regulatory change: from new rule to gap list

AI monitors an agreed list of official sources, summarises each change with a link to the text, compares it with current policies and drafts a ranked list of gaps. The compliance officer decides whether a change applies and assigns every action. AI never marks a rule as not applicable and never closes a gap.

An AI register and audit trail

One register of every AI use: its owner, purpose, risk level, model and prompt versions, overrides, escalations and incidents. Every AI use has a named owner, and risk signs off before go-live. Each new workflow updates the register automatically, so it shows what happened rather than what was planned.

What a regulator will ask

Expect four questions about any AI-assisted decision. Who owned it? What did the AI do? Who approved the outcome? What changed since? If the workflow was designed with the three lists above and logs every action, each answer is a lookup, not an investigation.

Get the use cases

The short guide below covers these three workflows. Newsletter subscribers get the full playbook, with the before-and-after workflow, the control points, a baseline sheet and a readiness checklist written for regulated firms.

Questions and answers

What does human in the loop mean in practice?

It means a named person approves a defined set of decisions before they take effect, and the approval is recorded. It is not a general promise that people are involved. The list of decisions that need a person is written down before the workflow is built.

Who should own an AI system in a regulated firm?

A named business owner for each AI use, not a committee and not the vendor. Risk or compliance signs off the risk level before go-live, and material changes, such as a new model or prompt, go back through sign-off.

What records should we keep for AI decisions?

What the AI was given, what it produced, which model and prompt version produced it, who approved or overrode it and why, and the version that was finally used. Keep them for as long as you keep the business record they support.

Does the EU AI Act require human oversight?

Yes, for systems it classes as high-risk, human oversight is one of the requirements. Firms outside the EU often face similar expectations from their own regulators, so designing for oversight from the start is the safer default.

Sources

  1. Camunda, The AI Process Gap, September 2026 (survey by Sapio Research of 1,000 process decision-makers and 5,000 employees at organisations with 1,000+ employees in the US, UK, Germany and France) ↗
  2. Dataiku, The Ultimate Guide to AI Agent Use Cases, p17 and p19: regulatory change monitoring and client relationship intelligence ↗
  3. Neptune Software, The AI Execution Gap (FY26), p11 and p15: AI inventories, audit trails and EU AI Act oversight requirements ↗

Statistics are quoted as published, with the survey base stated. They describe large organisations in the US, UK, Germany and France, not every market.