Every fintech team building an AI-assisted KYC pipeline hits the same fork eventually: make the system rigid, and it does exactly what it's told. Make it flexible enough to use judgment, and something else breaks.

The mistake is thinking one of those settings is the fix. It isn't. Here's why.

The case for rigid.

A deterministic system, sanctions list matching, document validation, and a fixed set of risk flags, does exactly what it's told, every time. You can show an examiner precisely why it made every call it made. No drift, no surprises, full audit trail. For a lot of KYC, this is genuinely the right answer.

Push it further, though, and hand it the case a five-year compliance analyst would wave through without a second thought, and the rigid system doesn't wave it through. It can't.

What the research actually found.

Researchers at Harvard, MIT, and Johns Hopkins tested this directly, using a scenario built around exactly this kind of financial judgment call: a loan officer's policy requires a credit score of 580 or higher. An applicant is exceptional in every other respect, but a recent hardship outside their control dropped their score just under the line, and every signal points to it recovering. A human loan officer, and any reasonable KYC analyst facing an equivalent risk-based call, would look at the full picture.

72.7% of human participants still processed a similarly policy-violating case, when circumstances justified it, across a battery of comparable scenarios.

The AI models tested, including GPT-4o, Claude, Gemini, and Llama, told a different story. Their refusal rates diverged sharply from human refusal rates across the large majority of 60 tested scenario-severity combinations, ranging from 48 of 60 for the most flexible model family up to all 60 for the strictest.

Fine-tuning narrowed the gap. It didn't close it. Even after training GPT-4o specifically on human explanations, its decisions still deviated from human judgment by 15.9% on average, and that improvement required curated training data that most compliance teams don't have sitting around.

The rigid system is consistent, auditable, and wrong on exactly the cases where a good analyst earns their seat.

So teams loosen it.

The obvious next move: give the model more room, let it weigh context the way a person would, and get the judgment call back.

This works, for a while. Then someone asks how a specific decision was reached, and the honest answer gets harder to give.

A separate study out of Oxford ran into the same structural problem from a different angle, testing a regulatory applicability judgment for medical devices. An LLM given no grounding produced correct judgments only 14% of the time. Heavy retrieval infrastructure pushed that to 71%, still leaving roughly three wrong calls in ten. The researchers were candid about the tradeoff: real concerns about auditability and reliability, in exactly the safety-critical settings where an examiner will eventually ask you to reconstruct why a decision went the way it did.

Loosen it enough to get the judgment call right, and you lose the thing that made the rigid version defensible in the first place. You're still wrong close to three times in ten, even after the engineering work to make it better.

Neither failure is the model's fault.

Run this experiment enough times, and the conclusion stops being about model quality. It's structural.

Know Your Customer regulation runs on what's called a risk-based approach, and that was a deliberate correction. Through the 1990s, KYC and AML ran on rigid, prescriptive rules applied the same way to every customer regardless of actual risk.

Regulators found that approach failed on its own terms: inflexible, expensive to run, and blind to the businesses and customers it was supposed to protect. The Financial Services Authority, now the FCA, proposed the risk-based alternative in 2000, and the FATF formalized it globally in 2012, asking institutions to assess each customer's actual risk and apply judgment about what due diligence that risk warrants, instead of running everyone through the same fixed checklist.

That history matters here because the ambiguity teams keep hitting isn't a drafting failure. It's the entire point of the regulation, written for a reader capable of judgment, and handed to a system that can only be one of two things at a time: perfectly consistent, or genuinely flexible. Never both. That's the specification problem, and it exists before anyone writes a line of code.

What the stakes look like in production.

None of this is academic when it goes wrong. The FCA fined Monzo £21,091,300 for anti-financial crime systems and controls that didn't hold up to risk-based approach standards, with a full remediation programme required afterward. That's what "the risk-based approach wasn't really implemented" costs in a real enforcement action, independent of which part of the system produced the gap.

The failures we see teams walk into seldom happen in the obvious cases. High-risk and low-risk get handled fine by nearly any reasonably built system. It's the case in the middle, the one that an experienced analyst flags for a reason that was never written into any policy document, because policies were never meant to enumerate every reason. That's precisely the case: a rules-following system has no path to, and a flexible system catches inconsistently, without being able to explain how.

What actually works.

The fix isn't a smarter model, and it isn't asking AI to develop judgment. It's refusing to ask AI to hold the part of the job that was never rule-based to begin with.

Split the work along the line that the regulation already implies:

• Deterministic checks(sanctions list matches, document validation, transaction pattern rules) run through the system with full audit logging. This is genuinely rule-based work, and AI does it consistently and transparently.
• Anything that requires weighing contextthe way a risk-based approach intends gets routed to a human reviewer, with the system's job limited to surfacing the relevant facts clearly, not resolving the ambiguity on its own.

This isn't a workaround. It's close to what the researchers behind both studies concluded independently. The medical device compliance team's own recommendation was an "expert-in-the-loop" framework, where human reviewers stay involved specifically for high-risk or ambiguous cases while the system handles traceability and retrieval. That's a narrower claim for the AI to make, and a more honest one. It's also the auditable version, because the system was never pretending to exercise judgment; it wasn't built to hold.

We build KYC pipelines this way for exactly this reason, for teams operating in the US, EU, and Australian markets.

If you're working through something similar, connect with me on LinkedIn, happy to talk it through over a coffee.