Isha Marathe: RAG Is Far From Magic and Prone to Hallucinations

Extract from Isha Marathe’s article “RAG Is Far From Magic and Prone to Hallucinations”

In their quest to integrate and market generative AI-powered technologies, legal tech companies have often cited a process they claim keeps hallucinations at bay: retrieval augmented generation (RAG).

RAG shows up in press releases, at trade shows, and in many product demos as a solution for large language models’ (LLMs) hallucination problem. 

For technologists, RAG is a little more nuanced than that. It can be a useful mechanism for reducing hallucinations, but it’s not a permanent solution for every type of AI tool, and often shows up in different ways depending on implementation, they told Legaltech News. 

For example, a recent study evaluating the accuracy of generative AI-powered legal research stalwarts Thomson Reuters (TR) and Lexis Nexis, both of which cite RAG as a mechanism for increasing accuracy, caused some surprise from the industry for the high hallucination rates it found. 

The Stanford pre-print research study, “Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools,” conducted by Stanford’s RegLab and Human-Centered Artificial Intelligence (HAI) research center, showed that TR’s Westlaw AI and Lexis+AI hallucinated 33% and 17% of the time, respectively.

Read more here