A program intended to tackle large language model (LLM) biases and hallucinations has made progress toward its launch, the director of the Intelligence Advanced Research Projects Activity (IARPA) said on Thursday.
The leading research intelligence organization housed within the Office of the Director of National Intelligence (ODNI) announced its Bias Effects and Notable Generative AI Limitations (BENGAL) program in late 2023. After seeking proposals and more information for its new program last year, IARPA Director Rick Muller said that they have decided source selections and performers.
“We are looking at, what do bias effects look like – how are those introduced in AI models? To what extent can hallucinations occur in these [systems],” said Muller, speaking at the GovCIO Media & Research 2024 AI Summit.
BENGAL performers will focus on one or more topic domains including detecting biases and inducing diverse analytical perspectives; detecting and mitigating AI hallucinations and inferences; methods to ensure the safe flow of information in sensitive environments; and improving LLM reliability through quality evaluations and improving poisoned sources.
Understanding the potential consequences of AI and LLM systems is increasingly important for chief information officers as systems become more integrated into agencies and workflows, Muller said.
“How can we trust AI when human lives on the line, right?” said Muller. “Those are the same types of issues that we’re going to have to understand … as soon as possible. I don’t know the answers to all of those things. What IARPA is doing, I think, is a very good foundation for that.”
BENGAL is one of 12 current research programs in IARPA’s Office of Analysis, including the TrojAI program which aims to defend AI systems from intentional and malicious attacks. Using research programs like these to improve and maintain AI and LLM security in the intelligence community is especially important, Muller added.
“We’re going to see spy-versus-spy interactions transition to AI-versus-AI,” said Muller, citing a prior conversation of his with someone in the intelligence community. “So, maybe cyber intrusions aren’t going to be … typing [into] a terminal anymore to get into or to compromise some system.”
The BENGAL super seedling program, which began this year, is set to be a two-year effort. It evaluates multi-modal and text-only LLMs.
