For decades, insurers have relied on a comforting illusion: that fairness can be proven by policies, procedures, and statistical averages. Write enough rules, sample enough files, measure enough KPIs — and you can show that your customers are being treated fairly.
This illusion is breaking. And for once, it is not technology reshaping regulation. It is regulation overtaking technology.
From Principles to Outcomes
The FCA’s new Consumer Duty marks a profound shift. It moves the industry from: “Prove you have good policies and your results look fair on average.” to: “Prove that each customer received a good outcome — and explain why.”
This sounds administrative, even trivial. It is not. It reverses the entire logic of oversight: from statistical defensibility to individual accountability. It demands not just that the right things happen, but that each one can be justified as the right thing in its context. This is not how today’s systems work. And it exposes a fault line that runs through the entire industry.
The Structural Gap
Claims systems were built for throughput and compliance, not ethical reasoning. They can apply static rules, route work, and monitor performance in aggregate. But they cannot weigh conflicting duties on a single case, consider how one decision affects others, or explain their reasoning to customers, regulators, or courts.
Historically, this gap was bridged by humans. Adjusters, handlers, and managers brought judgment and discretion into the process — the ability to balance competing obligations, to weigh context, and to make trade-offs in real time. That human buffer is now being stripped away. As more and more decision-making is delegated to systems, we rely on software architectures that are not designed to manage proportionality, fairness, or ethical justification.
In theory, you could embed thousands of rules to mimic human trade-offs. In practice, rule sets become so complex that they collapse under their own weight. Either way, the gap remains. Fairness today is still managed through policy declarations, vulnerability flags, and retrospective MI. It is inferred after the fact, not ensured at the moment of decision.
Yet the regulator now demands the opposite: assurance before action. This is the chasm between what is required, and what is currently possible. And unless insurers acknowledge this structural gap, they risk assuming they are compliant when in fact they are not.
This Is Not Unique to Insurance
Insurance is not alone in this reckoning. The same inversion is happening elsewhere.
The EU AI Act (which will cover UK insurers offering services in the EU market) mandates that any AI system making consequential decisions must provide transparent, explainable, human-interpretable reasoning. This shifts the burden from “show it works on average” to “prove each decision is safe and justifiable.”
The NHS reforms are pushing for outcome-based funding, where providers are paid based on measurable patient outcomes, not activity volumes — forcing systems built for throughput to demonstrate value on a per-case basis.
Across sectors, regulation is no longer asking for controls. It is asking for proof. And for the first time in living memory, regulation has leapt ahead of technology. Technology is scrambling to catch up.
Why AI Makes This Gap Worse, Not Better
AI is often imagined as the bridge. In practice, it widens the gap. Current AI systems optimise for efficiency, accuracy, and prediction, not fairness or duty. When injected into claims operations, they make decisions faster, not fairer. They compound hidden biases in historical data, and they accelerate opaque outcomes beyond the reach of human oversight.
Where older systems could at least be interrogated — “which rule fired?” — AI systems often cannot be explained at all. They produce outputs without reasoning, and in doing so they collapse accountability. Paradoxically, the more AI insurers adopt without addressing this gap, the less able they become to prove that their decisions are fair.
Why Agentic AI Won’t Save Us Either
There is growing hope that agentic AI — systems that plan and act autonomously — will solve this. It will not. Agentic systems optimise for task completion, not ethical justification. They can orchestrate claims processes at speed, but they have no concept of duty of care, proportionality, fairness between claimants, or the regulatory principle that each outcome must be good in its context.
They will simply execute faster — and if they are wrong, they will be wrong at scale, at speed, and without explanation. Agentic AI can automate claims. It cannot defend them. And defending decisions is now the heart of compliance.
The Strategic Reckoning
This leaves insurers in a precarious position. They face regulatory risk, because they cannot explain their decisions. They incur operational cost, padding outcomes with goodwill to appear fair. They risk reputational damage, as opaque systems erode trust even when acting in good faith.
None of this is because anyone is negligent. It is because the architecture itself is misaligned with the new regulatory model. We have systems designed to optimise transactions, in a world that now demands proof of duties.
The Opportunity
The shift is not about adding more dashboards, KPIs, or post-hoc audits. It is about enabling insurers to say, for any decision: “This was the right thing to do — and here is why.”
The firms that can do this will carry an unassailable advantage. They will cut leakage by being fair without overcompensating. They will shield themselves from regulatory and legal tail risk. And they will build the kind of trust that pricing alone can never buy.
At Claim Technology, we believe this is where the market is heading. That’s why we are building embeddable decisions: sidecar modules that can be called via API while a claim is in progress. Like GPS, they help steer the claim in real time, double-checking key decisions, providing transparent reasoning, and generating the proof points regulators require under Consumer Duty.
This is not just about compliance. It is about restoring confidence that fairness is baked into each decision as it happens — not reverse-engineered afterwards.
