Extract from Keeling & Mangum’s article, “Results Are In: 5 Lessons from an Independent Study of aiR for Review.”
Generative AI has generated plenty of conversation in legal circles. Some of it is optimistic. Some of it is skeptical. Much of it is theoretical.
So we decided to test it at Redgrave LLP.
We conducted a head-to-head study comparing Relativity aiR for Review, a generative AI review workflow, against a traditional first-pass managed review workflow using active learning. We chose a deliberately difficult document population involving ~45,000 documents from a real-world public data set and a nuanced responsiveness standard tied to pharmaceutical marketing, controlled substances, and federal compliance obligations.
In other words, documents were not responsive simply because they mentioned opioids, sales activity, or drug promotion. They had to contain evidence related to compliance with, violation of, or reckless disregard of federal requirements. That made the exercise less about finding documents “about” a topic and more about applying judgment to the contents of the document.
