Extract from Ella Sherman’s article “Vals AI Report Shows Gen AI Tools Outperforming Lawyers on Legal Research Tasks”
Generative artificial intelligence tools appear to be besting lawyers in certain aspects of legal research, according to a report released Thursday by LLM evaluation startup Vals AI. The latest edition of the Vals Legal AI Report showed that tools from Alexi, Counsel Stack, Midpage and Open AI responded to legal research questions with better accuracy and authoritativeness than human lawyers.
The legal research report evaluated Alexi, Counsel Stack, Midpage and Open AI on their gen AI tools’ responses to 200 U.S. legal research questions. The questions were gathered from legal professionals at law firms including Reed Smith, Fisher Philips, McDermott Will & Schulte, Ogletree Deakins, Paul Hastings and Paul, Weiss, Rifkind, Wharton & Garrison.
Examples of questions and instructions posed to the AI tools included, “Is there a California provision that allows a right of rescission or treble damages if a sponsor sells through an unlicensed broker? Search California primary law to answer this question,” and, “Does the Federal Trade Commission regulate or have jurisdiction over nonprofit organizations or trade associations? Search federal primary law and case law to answer this question.”
