Extract from Isha Marathe’s article “RAG Is Far From Magic and Prone to Hallucinations”
In their quest to integrate and market generative AI-powered technologies, legal tech companies have often cited a process they claim keeps hallucinations at bay: retrieval augmented generation (RAG).
RAG shows up in press releases, at trade shows, and in many product demos as a solution for large language models’ (LLMs) hallucination problem.
For technologists, RAG is a little more nuanced than that. It can be a useful mechanism for reducing hallucinations, but it’s not a permanent solution for every type of AI tool, and often shows up in different ways depending on implementation, they told Legaltech News.
For example, a recent study evaluating the accuracy of generative AI-powered legal research stalwarts Thomson Reuters (TR) and Lexis Nexis, both of which cite RAG as a mechanism for increasing accuracy, caused some surprise from the industry for the high hallucination rates it found.
The Stanford pre-print research study, “Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools,” conducted by Stanford’s RegLab and Human-Centered Artificial Intelligence (HAI) research center, showed that TR’s Westlaw AI and Lexis+AI hallucinated 33% and 17% of the time, respectively.
