In my first major case using technology-assisted review, our team had to review documents in Korean, which brought with them privacy and cross-border transfer concerns. The technology was very helpful, but we still had to employ two rooms filled with Korean-speaking lawyers to support the effort. Needless to say, it was a very expensive production.
During a more recent matter in the second half of 2019, we collected nine-million documents and applied basic and broad keyword searches at the outset to quickly reduce that dataset to two-and-half-million records. We then applied Brainspace and its continuous active learning functionality to the remaining information and were able to quickly categorize each document, including those categories that were uniquely valuable to our case, to immediately and painlessly eliminate millions of documents from consideration.
The contrast between the two experiences was striking. Instead of multiple rooms filled with reviewing lawyers, we enlisted a skilled, but relatively small team of contract attorneys to code 25,000 records in two weeks. When all was said and done, the client told us that this project was far less expensive than the similarly sized project he had just completed on another case, and I am very comfortable that we identified the correct documents in a highly defensible manner.
As a result, leveraging artificial intelligence in this way is not just an option, it is the only one if you want to tame the vulgar expense of e-discovery.
This more effective model is not without its challenges, which include the following:
You Need Skilled Lead Counsel
Given that lead lawyers on matters of this type heavily rely on technology to determine which documents are relevant, it is essential that they have the requisite skill and understanding of the current technology to complement their legal talent. While they once simply designated documents as privileged or responsive in a linear manner, using mapping and other visualization tools allow them to highlight conversations, issues, windows in time, and specific types of documents, all in a manner that can quickly identify the most important documents related to a specific issue and cleave out those with no relevance. In other words, to fully harness the technology, counsel must not only have deep knowledge of the case, but must understand what can be done with the AI and how to do it.
Contract Attorneys Require Training
While contract attorneys may have fewer documents to review because of the technology, human eyes still need to review whatever the technology identifies as relevant. Accordingly, and possibly even to a greater extent than when they reviewed “everything,” contract attorneys must be deeply trained on the matter in order to optimize their efforts. Insufficient preparation may result in inconsistent document coding, i.e., responsive vs. non-responsive, which could materially delay the process. In fact, the more you rely on computers perform key tasks, the more disciplined the human interaction and input needs to be.
Client Collaboration is Critical
Full transparency and client buy-in about the process is critical. New tools are launching regularly so even sophisticated, large organizations may not understand the significant benefits and savings on the back end that usually result from the slightly higher front-end cost of the initial computerized data analysis. This may require preparing a cost-benefit analysis demonstrating the overall savings, which again, I have found to be increasingly substantial. We were certain that our advanced approach would result in a substantial cost reduction and it turned out to be one of the smoothest productions we had ever completed. Our collaboration ensured that the client’s sophisticated team collected efficiently and transferred it to the host. With well-documented culling followed by the AI analysis, we were able to save thousands in monthly hosting fees alone.
Choose the Right Technology
In our case, the head of litigation support technology at our firm recommended Brainspace because it integrated with our existing portfolio of tools. What I derisively call the golden age of big-law document review, with teams of associates reviewing every document in a linear manner, thankfully no longer exists. Increasingly, even the more restrained days of law firms simply supervising lower-cost contract reviewers are also in the rearview mirror. Now, the law firm’s role is to optimize the use of AI-driven review tools, manage the technology, ensure the contract lawyers are well trained, and produce a defensible production.
While the firm associates still must participate in reviews, often performing quality-control aspects of the job, they are now supported by our manager of technology-assisted review. That manager can compare what the reviewers are finding with the broader database as a whole, essentially performing a statistical QC of the overall findings that further validates the integrity of the production.
And the emerging new model is not a bad thing for those like me in “big law.” Although the “golden age” ended—as it should have—with much of the rote review work being outsourced to contract reviewers, when AI is involved on large cases, the tech-savvy partners and associates are reemerging with new roles that actually create the kinds efficiencies that really justify their fees.
Key Best Practices
To maximize the value of your efforts and optimize efficiency:
- Remember that data security is the most important issue; it must be addressed with every vendor, contract lawyer, and team member.
- Surround yourself with the right people; people who truly understand the technology are worth their hourly rates and contribute to real savings overall.
- Carefully consider the roles of each member of the team. Often, it will be important to have a chief technologist liaising with both the client and professionals handling other aspects of the case.
- Hire the most talented contract lawyers, train them well, QC their work, and immediately let go of those who are not working out. Document review cannot be forgiving. One bad reviewer can infect the entire process.
- Quality control is key and must be done in a rigorous and consistent manner.
- Memorialize everything, from search terms, to AI processes, to the metrics on each stage of the review. I put everything into a defensibility memorandum so that if needed in two years I can explain to a court or tribunal exactly what was done and why it was reasonable.
Promoting the Promise of TAR
We have been discussing the promise of technology assisted review for years. Whether called TAR or AI, I believe the technology is now well in the mainstream, and am very impressed with its effectiveness. The challenge for junior lawyers is that technology is limiting the work that formerly provided them with foundational experience. Document review, though arduous, helps one learn about the business of a client. I remember spending many months as a young lawyer sitting at document repositories flipping moldy pages of old client files. It’s a great, if expensive, way for young lawyers to learn not only about the case, but about the ways of the business world. While automated review is better for clients in the long run, it does reduce the amount of work for human lawyers, so that supply and demand will have to re-balance over time.
For almost two decades, Mr. Lindsay has been a thought leader in issues relating to technology in litigation. The Associated Press has called him "an expert in technology and litigation," and The Wall Street Journal relied on him to explain the ground-breaking revisions to the Federal Rule of Civil Procedure to encompass electronic evidence. He is a certified e-discovery specialist and former chairperson of the Association of Certified E-Discovery Specialists (ACEDS) advisory board.
Mr. Lindsay has taught on the faculty of the National Institute of Trial Advocacy (NITA) at its prestigious National Trial Skills Session in Colorado as well as its Deposition Skills program in New York and Washington, D.C. For the past fifteen years, he has taught litigation and arbitration skills at the University of Miami School of Law, and he regularly teaches at other Florida law schools.
He authored the book and three-hour DVD titled Technology in Litigation, published by NITA, and has written numerous articles on technology-related issues published in periodicals including: the Yale Journal of Law and Technology, Inside Counsel, The National Law Journal, Legal Times, the University of Miami Law Review, and the International Law Quarterly.
While in law school, Mr. Lindsay was articles and comments editor of the University of Miami Law Review, a Dean's Scholar, and winner of the classwide moot court competition.