What role should generative artificial intelligence play in e-discovery processes? Are claims of the efficacy of using large language models in e-discovery bold, naked assertions or have they been tested and proven?
In a forthcoming paper to be published this Winter in The Advocate, a publication of The Litigation Section of the State Bar of Texas, Professors Maura Grossman, Gordon Cormack and Jason Baron explore these questions and more as the e-discovery industry grapples with and begins to come to terms with how Gen AI and LLMs will impact the e-discovery process.
Spoiler alert! The conclusion these three scholars reach is simple, yet powerful: “LLM tools and protocols have not yet been demonstrated to be as effective as currently recognized methods for legal research, nor for TAR.”
Fear not, however, because they are not saying Gen AI and LLMs are incapable of helping to sort through large volumes of electronically stored information; they’re simply suggesting that the processes, the protocols, and the tools themselves have not been properly tested and validated.
“The bottom line is that . . . there is no well-defined protocol for how to employ LLMs to find substantially all documents responsive to matter-specific requirements (e.g., RFPs) in a matter-specific collection of documents. The selection of tools, the engineering of prompts, and protocols for fine-tuning are largely unspecified, inscrutable, and no such selection has been demonstrated to improve on established TAR tools and practice.”
The day may yet come when we have the receipts, in much the same way we had the receipts when TAR rose to prominence 10 years ago, but we’re just not there yet.
You can read the full text of the Grossman, Cormack, Baron article titled Does the LLMperor Have New Clothes? Some Thoughts on the Use of LLMs in eDiscovery here: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4949879
