From AI Hallucinations To Trust Scores: How Enterprises Can Quantify And Govern GenAI Risk

For any business considering the use of a generative artificial intelligence (GenAI) large language model (LLM) for customer-facing and other critical decisions, hallucinations pose a quandary of epic proportions: How do you know if the model is hallucinating unless you already know the correct answer to any given prompt?
 
The short answer is, you don’t. A recent YouGov survey found that only 5% of Americans say they “trust AI a lot,” while 41% express distrust in this technology. The resulting trust gap, the mathematical difference between use of AI and trust in it, spans generations; 51% of Gen Z uses AI tools at least weekly, compared with 35% of US adults, yet both groups figure prominently into the 41% of respondents who remain skeptical of AI.  
 
The good news is that a focused language model (FLM), accompanied by a separate trust score model, presents one of the most effective ways available today to combat hallucinations. FLMs are small language models (SLMs) focused on a very narrow domain and a specific task. A fine level of specificity ensures the appropriate data is chosen to train the model, which is further painstakingly task-trained to ensure its correctness on a narrowly defined task.
 
But it’s still possible for FLMs to hallucinate. So how can businesses trust FLM outputs? That’s what the trust score is all about.
 

Trust scores measure GenAI hallucinatory risk

 
In business, ambiguity often translates into risk, which data scientists have spent decades building analytic models to quantify. Many of these models score the likelihood (odds) of a specific event occurring, such as a customer defaulting on a loan. Accordingly, trust becomes a quantifiable risk decision, allowing businesses to definitively assess risk and choose precise tolerance levels to maintain trust.
 
Companies like FICO are experts at developing financial risk scores and risk ranking. In the GenAI world, a trust score applies the same risk quantification principles to FLM outputs and allows organizations to prescribe their risk tolerance to hallucinations.
 
More specifically, a trust score is an analytic construct produced by a secondary model that runs independently alongside the FLM. The trust score measures two things:
 

• Whether a question asked the FLM, and the answer the FLM provides, has sufficient statistical coverage in the FLM training data

• The question and response’s relative alignment with the sanctioned answers known as knowledge anchors

 
This auxiliary model produces a trust score from 1 to 999; the higher the trust score, the less likely the FLM is to hallucinate. For users, the trust score is of critical consideration when its value determines whether the GenAI outputs will be used.
 

Knowledge anchors are the grail of truth

 
Trust scores are rooted in the focused, specialized data used to train FLMs, the auditable prompts and responses the FLM is designed to process, and the correct answers to output. As the first step in the training process, data scientists create specialized coverage statistics on the entirety of the FLM corpus. For example, some tokens could be rare and with insufficient representations, making it difficult for the user to have confidence in the FLM’s outputs. 
 
For the second step, data scientists and business subject matter experts (SMEs) work together to define heavily scrutinized, sanctioned knowledge anchors: the questions the FLM is designed to answer, and the correct responses. There may be 100 knowledge anchors, a thousand, or even more, depending on the scope of the FLM.  
 
Knowledge anchors are defined by human experts and are auditable. They evolve as the FLM is operationalized but are also the governed “base truth” that establishes the FLM responses as statistically accurate and aligned with governed and auditable truths.
 
In the third step, data scientists develop the trust model independently of the FLM. The trust model works with a specialized domain vocabulary that overlaps, but is a subset of, what was used in the FLM development. It tokenizes the relevant vocabulary of the knowledge anchors and creates latent space representations of them—think of this as the “audit latent space” that measures the core, correct intended business use and behavior of the FLM. 
 
When a question is asked of the FLM, the trust model removes non-domain vocabulary, computes an audit latent space representation, and measures how close this vector is to the knowledge anchors. If it’s close within the domain latent vector space, the question is aligned with what the model is designed to answer and moves to the next computation in the trust score.
 
The same process occurs with the response. When it is produced by the FLM, extraneous non-domain vocabulary is removed, and the result is embedded in the audit latent knowledge space. There, the trust model compares the provided response to sanctioned, prescribed answers in the knowledge vectors. If the provided answer is misaligned, the trust model indicates that the FLM is not responding in the way it should be, producing a low trust score.
 
Conversely, an answer vector that closely aligns with one or more knowledge anchors produces a high trust score if the coverage of the tokens is also well-represented in the original FLM training corpus. This high-trust-score response is more likely to be correct because it aligns with defined and auditable knowledge anchors and is also statistically significant to the corpus of FLM training data.
 

Built-in trust through audit

 
The trust model intrinsically audits FLM answers, a critical requirement in operationalizing a risk-based approach to managing GenAI hallucinations. The trust model leverages the statistical support of questions and answers in the FLM training corpus and creates an independent audit latent space of knowledge anchors using latent Dirichlet allocation (LDA). Each LDA audit representation of knowledge anchors persisted to the data science organization’s private blockchain, providing an immutable record of the trust model’s decisioning.
 
Importantly, the audit LDA operations act as a Level 2 (L2) monitoring system and are operationalized within the trust model, supervising the operations of the Level 1 (L1) function that creates an FLM response. L2 calculates the trust score, auditing whether the FLM prompts and responses are sanctioned and as expected. This built-in auditing adds a second, critical layer of assurance that the FLM is functioning in its intended, non-hallucinatory way.
 

Why trust scores matter

 
The audit LDA in the trust model, coupled with the FLM’s coverage statistics, allows businesses to raise and lower risk tolerances to hallucinations and non-ideal answers by choosing different thresholds of the trust score. In operationalizing FLM use, this capability solidifies the trust score’s status as an essential tool in the fight against GenAI hallucinations.
 
For the first time, trust scores remove ambiguity about hallucinations, allowing businesses to move to a quantifiable, risk-based way of managing GenAI outputs that is transparent and auditable. As any Chief Risk Officer (CRO) can attest, the phrase “accurate risk scoring” is music to their ears, and to C-suite executives everywhere.