New method could separate AI fact from fiction
- A new method developed by Michigan State University researchers aims to increase the reliability of AI information by creating a “trust meter” that reports the accuracy of information produced from large language models (LLMs).
- The method, called Calibrating LLM Confidence by Probing Perturbed Representation Stability (CCPS), applies tiny nudges to an LLM’s internal state while it forms an answer to test its confidence and stability.
- According to the researchers, the CCPS method is significantly better at predicting when an LLM is correct, cutting the calibration error between an AI’s expressed confidence and its actual accuracy by more than half on average.
- The breakthrough has profound clinical implications for medicine, addressing the primary safety barrier for LLMs in medical applications, which is their tendency to state errors with high confidence.
- The CCPS method improves an LLM’s internal confidence calibration, enabling the model to reliably know when it doesn’t know and defer to human expert judgment, enhancing safety and trust in AI systems.
A new method could help artificial intelligence users separate fact from fiction.
While artificial intelligence, or AI, tools like ChatGPT might be great for helping you pick where to go for dinner or which TV show to binge watch, would you trust it to make decisions about your medical care or finances?
AI tools like ChatGPT and Gemini include a disclaimer that the information they find scanning the internet may not always be accurate. If someone was researching a topic that they didn’t know anything about, how would they know how to confirm the information as truth? As AI tools become smarter and gain more widespread use in daily life, so do the stakes for the accuracy and dependability of using this evolving technology.
Michigan State University researchers aim to increase the reliability of AI information. To do this, they have developed a new method that acts like a trust meter and reports the accuracy of information produced from AI large language models, or LLMs.
Reza Khan Mohammadi, a doctoral student in MSU’s College of Engineering, and Mohammad Ghassemi, an assistant professor in the computer science and engineering department, collaborated with researchers from Henry Ford Health and JPMorganChase Artificial Intelligence Research on this work.
“As more people rely on LLMs in their daily work, there’s a fundamental question of trust that lingers in the back of our minds: Is the information we’re getting actually correct?” says Khan Mohammadi. “Our goal was to create a practical ‘trust meter’ that could give users a clear signal of the model’s true confidence, especially in high-stakes domains where an error can have serious consequences.”
Though a person can repeatedly ask an AI tool the same question to check for consistency—a slow and energy costly process—the MSU-led team developed a more efficient internal approach. The new method called Calibrating LLM Confidence by Probing Perturbed Representation Stability, or CCPS, applies tiny nudges to an LLM’s internal state while it’s forming an answer. These nudges “poke” at the foundation of the answer to see if the answer is strong and stable or weak and unreliable.
“The idea is simple but powerful, and if small internal changes cause the model’s potential answer to shift, it probably wasn’t very confident to begin with,” says Ghassemi. “A genuinely confident decision should be stable and resilient, like a well-built bridge. We essentially test that bridge’s integrity.”
The researchers have found that their method is significantly better at predicting when an LLM is correct. Compared to the strongest prior techniques, the CCPS method cuts the calibration error—the gap between an AI’s expressed confidence and its actual accuracy—by more than half on average.
“The CCPS method has profound clinical implications because it addresses the primary safety barrier for LLMs in medicine, which is their tendency to state errors with high confidence,” says Kundan Thind, coauthor on the paper and division head of radiation oncology physics with Henry Ford Cancer Institute.
“This method improves an LLM’s internal confidence calibration, enabling the model to reliably ‘know when it doesn’t know’ and defer to human expert judgment.”
This breakthrough has been tested on high-stakes examples in medical and financial question-answering, and its potential to enhance safety and trust in AI is vast.
This research was recently presented at the Conference on Empirical Methods in Natural Language Processing in China.
Funding for the research came from the Henry Ford Health + Michigan State University Health Sciences Cancer Seed Funding Program and the JPMorganChase Artificial Intelligence Research Faculty Research Award.
Source: Michigan State University
The post New method could separate AI fact from fiction appeared first on Futurity.