Verifiable vs. Explainable AI

Let’s talk about “Verifiable” vs. “Explainable” AI.

If you’re in the field of finance, investment, or accounting you’re no doubt aware that using AI in your work directly implies a high need for correctness. Depending on the nature of your particular work or the data you’re interacting with, it might also imply regulatory compliance.

Verifiability and Explainability are two related concepts which pertain to these concerns but differ in the type of outcome which results from a particular workflow.

Verifiability pertains to output generated by AI. The output is an artifact: reports, emails, meeting agendas, decision documents, etc. Verifiability is concerned with being able to confirm that every detail in the output is correct so you can have confidence submitting it for its intended use. It is scoped to a single moment in time which exists at the very end of the AI processing chain.

Explainability, on the other hand, pertains to decisions executed by AI. The decision is an action: loan approvals, flagging transactions for AML investigation, buying vs. selling securities, etc. Explainability is concerned with satisfying the regulatory requirement for full rationalization of automated decisions by any system — AI-driven systems being no exception. It is scoped in time to the entire arc of the processing chain — which data was used in each subsystem, what was the result of each subsystem, and by what logic does each subsystem use to arrive at the results it yields?

In short: Verifiability is a snapshot, a final inspection; Explainability is a timeline, an audit chain.

Much more has been said about Explainability. It’s an exciting subdomain of AI, particularly in the financial world, it’s extremely technically complex, and it concerns the mass automation of financial transactions and decisions which feels close to the Holy Grail of what AI has been promised to eventually deliver.

However, it also isn’t directly relevant to the vast majority of daily workflows related to knowledge work, internal & external communications, and reporting which comprise a massive amount of what makes the gears of the industry turn on a daily basis. When automating these types of workflows, regulatory compliance is still a concern and correctness matters to the highest possible degree but the auditability requirements are not the same as when e.g. denying a loan application in an automated system. Thus, building a system which yields Verifiable output is the bar to reach for most automated financial workflows.

The problem is that these types of workflows clearly could be automated by AI to some extent but not many people seem to actually know what a Verifiable automation looks like. So what gives?

A good place to start is by getting clear about what Verifiability buys us. A major complaint I’ve heard again and again is that, sure, AI automations generate output impossibly quickly but that you end up having to do a lot of work to confirm all the facts and analysis in the output. This verification effort often eats up nearly as much time as just doing the work manually in the first place which utterly destroys the ROI of the automation. Being able to quickly and easily verify output collapses the verification effort down to a fraction of the baseline effort which, in turn, delivers the productivity gains it was supposed to yield all along.

What does this look like? Simply put, you need to build a system where all source data is tagged, all claims in the generated output are directly linked to a data tag, and the UI provides the ability to directly inspect the source data associated with each claim directly in the same interface. Failing this means your staff has to go hunting through any number of systems to find supporting claim data. By centralizing claims and source data, verification becomes easy and reliable yet this is where most automations break down.

At the moment, not enough attention is being paid to understanding Verifiable workflows yet these are where the majority of automation ROI will ultimately be gleaned. The technical and regulatory bar is lower than Explainable workflows but nevertheless still carries complexity which shouldn’t be trivialized.

Yet the answers exist. Don’t let your organization fall behind by failing to take action when such a golden opportunity is currently at hand.