ANSPRE Delivers More Accurate, Concise Responses From Large Language Models — Through Prefixing

Researchers from the Japan Advanced Institute of Science and Technology have come up with way to make large language models (LLMs) more accurate in their responses — by giving them part of the answer in advance, through what they're calling Answer-Prefix Generation (ANSPRE).

"ANSPRE can improve the generation quality of LLMs, allow them to output the exact answer phrase, and produce reliable confidence scores. Additionally, it can be incorporated into any LLM and complex architecture," claims project lead Nguyen Le Minh of his team's creation. "Our method can lead to more concise and accurate question answering in critical fields like medical diagnosis, legal assistance, and education, and improve customer support. Furthermore, in the long term, our research could foster widespread human-artificial intelligence collaboration by increasing trust in AI systems."

Large language models, which return token-based most-likely answers to their users' prompts, have exploded in popularity over the past few years. They're not without their problems, though — even excluding ongoing furors and legal battles over their creators' hoovering up masses of copyrighted content to act as a data set for training: LLMs have a tendency to be verbose and, lacking any actual understanding of the response or even the core concept of truthfulness, can "hallucinate" useful-sounding but entirely inaccurate "answers."

It's here that ANSPRE aims to help, and it's surprisingly simple: giving the LLM a head-start by providing part of the answer in advance and having it fill in the blanks. "Consider the example question, 'What gambling game, requiring two coins to play, was popular in World War I?'," Nguyen offers by way of demonstration. "An answer prefix for this question could be, 'The gambling game requiring two coins to play that was popular in World War I was ___.' As most LLMs are trained with causal language modeling, using the answer prefix would allow the LLM to generate the exact answer phrase in place of the blank."

Rather than coming up with the prefixes by hand, ANSPRE uses a few-shot examples to generate a prefix for a given question. The system then uses an existing retriever to pull relevant content from a knowledge base, which is combined with the question and the answer prefix to provide a detailed prompt for the target LLM. An extended version, Self-Reflective Answer-Prefix Generation (SELF-ANSPRE), further improves the results by ranking responses based on confidence scores and how useful each retrieved knowledge base document was in informing the answer.

The team's work was presented at the 27th European Conference on Artificial Intelligence over the weekend, and is available under open-access terms from IOS Press as part of the Frontiers in Artificial Intelligence and Applications series.