By Danny Goh, Terence Tse and Rob Casper
In regulated industries, AI’s true advantage lies not in capability alone, but in embedding human accountability, professional judgment, and trust into deployment.
The following scenario is common enough in financial services. Teams implement AI for contract drafting, fraud detection, or report generation. Although pilots have shown promising results, risk committees often halt these projects. The typical response is to improve models, increase testing, and tighten controls. Yet, despite these performance-improving measures, projects still stall at the committee stage. The core issue is not AI capability. Instead, it is the lack of accountability.
Accountability, not capability
Consider a hospital using advanced medical AI that outperforms physicians in diagnosis. Would it allow the AI to issue prescriptions on its own? The answer is a firm no. But this has more to do with the law requiring licensed physicians to sign prescriptions rather than doubts about AI’s capability. AI can prepare recommendations, but only a doctor can authorize them. Competence and accountability are two very different constructs. Only competence can be handed to a machine.
Producing an answer is easy for AI these days. Establishing traceability and accountability at scale is the real challenge. This is especially the case for highly regulated industries such as financial services. When banks evaluate AI systems, the compliance team isn’t necessarily concerned with the technical capability. What matters more is identifying who should be responsible if the system fails. Far more important is that current regulations in financial disclosure, banking oversight, and outsourcing in the US, UK, EU, and Singapore at least, require a named, accountable individual. It isn’t a role that AI is allowed to assume.
Two kinds of trust
Confusion between accountability and capability may arise because the term “trust” has two meanings. Execution trust is created or strengthened by improvements in the technology itself, such as increased accuracy, more detailed logs, and a more up-to-date performance history. Put differently, execution trust answers the question of whether the system is doing what it was designed to do.
Yet, what institutions actually need is business trust, which involves a qualified individual endorsing an outcome. An auditor signing a company’s accounts or a doctor signing a treatment order isn’t merely confirming that a process ran correctly — they are attaching their professional experience and standing to the result. In other words, business trust is unrelated to the properties of the AI systems and answers an entirely different question: who stands behind this.
Unfortunately, the AI industry has – erroneously – assumed that if execution trust becomes strong enough, business trust will follow. This is one reason many AI vendors and companies alike have focused their efforts on making their AI systems better. Even transparency and traceability can’t, by themselves, produce accountability; they merely produce more credible information. Establishing business trust requires a completely different approach, at least in the financial services sector.
AI you can sign
From our front-row seat in financial services’ AI work, we believe that building business trust requires a tool that predates computers by millennia: the signature. This mechanism underpins a doctor’s chart, an auditor’s opinion, and a CEO’s quarterly earnings sign-off. A signature carries legal weight and authenticity beyond a mere copy or electronic mark. It helps determine enforceability, evidentiary value, and compliance with governing laws. A signature is the source of business trust.
Yet much of the enterprise AI deployed today has only two stages: In the first, the system ingests and processes input data to produce a specific outcome; and in the second, the outcome is checked against a standard as a quality-assurance step. What is missing is a third stage: a named individual identified as accountable for the outcome. Without a sign-off, no one is truly accountable for what the AI produces.
Not all sign-offs are equal
We believe three tiers of sign-off exist, each with a different level of accountability. At the lowest tier, a human clicks “approve” on an output. Yet this can be done without the person actually reviewing – or even worse, understanding – the output. At most, this amounts to nominal supervision. This is one reason why the renowned “human-in-the-loop” is frequently not sufficient to build much business trust.
The next tier of sign-on is similar to a warranty: an assurance that an AI provider guarantees the output will meet specified conditions of quality or performance. But even in this case, no individual is personally accountable. A warranty merely compensates for failure. In short, it is just corporate accountability at work and not professional accountability.
The highest tier is the third, where a licensed professional reviews the work against a defined standard and signs it under their own name, putting their reputation and liability on the line. This is the level at which business trust can actually be created.
Building in accountability by design
For companies in regulated sectors, the goal should be to integrate the third tier directly into AI deployment, building the Stage 3 mentioned above into the AI architecture. We call this approach “machine-first,” human-final.” In this way, the AI platform manages 90% of the overall processing volume, automatically clearing standard, routine, low-risk cases and creating a full audit trail along the way. The AI passes the remaining 10% representing more complex and exceptional cases to a senior professional for sign-off.
For example, in wealth management, many firms can currently only spot-check a fraction of client statements to see if they are correctly presented. Now, with the “machine-first, human-final” model, AI can review every statement, clear most automatically, and escalate only genuine anomalies, allowing professionals to focus on significant exceptions and real exposure.
This approach can confer advantages that even the best AI model cannot offer. First, when a professional signs off on a case, the decision is documented along with the context, evidence, outcome, and the professional’s name. This documentation is especially valuable in regulated industries. Most AI systems do not capture this information because they tend to minimize rather than preserve and respect human involvement. Over time, a company can build a unique library of expert decisions that competitors and AI models cannot replicate. Indeed, this library, in turn, creates a compounding flywheel, in which every human correction becomes structured training data to further improve the AI in place.
Second, this approach redefines automation by focusing on judgment rather than merely labor. Judgment improves with experience, often well into later stages of a career. This approach values professionals whose expertise was previously constrained by workload, not by the quality of their judgment. As a result, careers can be extended, allowing experienced individuals to guide AI-driven processes even as their pace slows. The goal of AI deployment should not be to replace people. Instead, AI deployment should eliminate routine tasks and enable professional judgment to have greater impact.
Design, not technology
Business leaders need to stop designing AI around tasks and start designing it around that which needs human accountability. Managers would do well to remember that the winners in cloud computing weren’t the quickest to roll out the infrastructure but the ones who made compliance their selling point. The firms that win in the enterprise AI world tomorrow won’t be the ones with the most clever models. They’ll be the ones whose output is worthy of – and signed by – a professional.
About the Authors
Danny Goh is the Co-founder & CEO of Nexus FrontierTech. He also co-founded the AI Native Foundation. Danny is a co-author of Becoming AI Native: Charting the Next AI Frontier (Routledge, 2026).
Terence Tse is the Co-founder & Executive Director at Nexus FrontierTech. He also co-founded the AI Native Foundation. Terence is a co-author of Becoming AI Native: Charting the Next AI Frontier (Routledge, 2026).
Rob Casper is Principal at Ridgeview Digital.
