Kangaroo Court

Kangaroo Court: Developing Realistic AI Goals

Share this article

K. Anders Ericsson was a Swedish psychologist who spent much of his academic career researching the psychological nature of expertise and human performance. He studied expertise across a variety of domains to understand how expert performers acquired their extraordinary ability by focusing intently on the mastery of specific skills. Ericsson was the first to introduce to concept that mastery of a given task required 10,000 hours of practice. This idea was later popularized by Malcolm Gladwell in his book, ‘Outliers’.

The point Ericsson and Gladwell convey is the importance of perseverance and determination. Success in any goal requires a process that needs time and patience to be fulfilled. This idea is more common throughout history, psychology, and philosophy than we might think. But a history lesson is not needed to appreciate the outcomes associated with committing to one task rather than six. If you take on too many tasks, each one will suffer. At the very least, the result will be a shadow of its true potential. This is important for law firms and corporate legal teams to consider when embarking on their AI journey. We all want a strategic advantage and there is often internal pressure to get there as quickly as possible. However, a rushed effort will create confusion, reduce quality control, lack appropriate testing, and ultimately require a complete overhaul when standards are not met.

The best example of the importance of process can be found in Brazilian Jiu Jitsu (BJJ). Originally inspired by the mixed-martial-arts style of Mitsuyo Maeda, the artform was created by the Gracie family in Brazil after Carlos Gracie watched a demonstration by Maeda in 1917 at the Da Paz Theatre in Belem. The fighting style focuses on submissions, sparring and live drilling throughout the teaching process. As students learn individual skills, they progress through a series of colored belt grades, similar to that of other martial arts, like Karate or Judo. What separates BJJ is the time it takes to graduate from one belt to the next. In Karate a student can expect to follow a relatively clear timeline before they gain a black belt. BJJ is completely different. Progression is entirely determined by the instructor, and can take years based on the student’s commitment to mastering the art.

There are five belts. They follow the order of white, blue, purple, brown, and black. The white belt is about learning to defend yourself. The blue belt is about learning how to attack. Once the instructor is satisfied these basics can be mastered to a suitable standard, then the real commitment begins. The purple belt is generally where students can expect to spend the meat of their time while learning BJJ. During this period, the student will develop their style. Think of it as moving from conscious incompetence to conscious competence. The edges are smoothed out during the brown belt phase, before finally the student achieves mastery and is awarded their black belt. From start to finish this process could take anywhere between 7-15 years. There are exceptions, but this is extremely rare.

Gaining the benefits of AI requires the same degree of commitment and patience. The good news is you do not need 10,000 hours or 15 years to build a dynamic AI ecosystem within your organization. Of course, these are just metaphors. What you do need is realistic goals and a firm-wide commitment to process so the value of AI can be realized throughout the journey itself. Here is an example of what that process could look like:

Step 1: Be clear about your goals.

Even though your AI goals might take some time to achieve, a broad plan is required so that every stakeholder understands where the team is headed, and how they are going to get there. This is the nucleus of your AI strategy. For this example, let us assume the end goal is transforming a law firm’s historical data into AI models that are both practice and need specific. Examples of practice specific models could be collusion or price fixing for Antitrust, or FCPA for Securities Litigation. Need specific could mean the identification of PII, or document management. This is a herculean task for the AI white belts who have only begun the process of understanding the ins and outs of machine learning. But the goal is worth achieving for the advantage it gives to a committed firm. AI is about extracting value from data, and a law firm with a library of carefully crafted models is the equivalent of a BJJ black belt in litigation. They will understand key insights before opposing counsel, giving their clients a strategic advantage, and ultimately helping the firm secure more business.

Step 2: Go for the low hanging fruit.

A white belt in BJJ will still be more effective in a fight than someone without any BJJ skills. For starters, they can defend themselves when the fight goes to the ground, which it inevitably will. They will also understand the basics of submitting an opponent and be able to leverage that knowledge to their advantage. Just like the white belt, a law firm should set early goals that are S.M.A.R.T and take advantage of existing AI tools by folding them into eDiscovery workflows. Early case assessment is ripe for opportunities to gain value out of AI today. Unsupervised and supervised machine learning techniques can cluster key themes, reveal hidden patterns, and make the process of understanding case data significantly easier than traditional TAR. Attaining this goal requires partnering with eDiscovery services firms that both understand AI and can provide its value within existing workflows. Realistic early goals should include better document reduction, improved threading suppression, and a more effective process for identifying privilege.  Low hanging fruit, but a big early win to demonstrate the success of your AI initiative and continue to grow support.

Step 3: Set AI model priorities.

AI models can be created for a wide variety of applications and practice groups. Remember that mastery takes time, and the machine learning process for creating models requires planning to ensure the data is suitable and outcome can be achieved. Despite already seeing the value in the low hanging fruit, this next step will take time to master. When deciding which models should be prioritized there are a few questions to consider:

  1. Which practice groups generate the most revenue?
    • If a firm has ten practice groups, and three of them produce 80% of the firm’s revenue, these might take priority.
    • On the other hand, consider how creating models could enhance the value of underperforming practice groups and increase new business.
  2. What are the needs-based models with the greatest value? (e.g. PII, sentiment, behavior)
    • Which models could be used across practice groups?
    • Which models will deliver your largest clients the greatest value?
    • Which models will increase the business with your smaller clients?

This is the blue belt phase. Map out the strengths and weaknesses across your practice groups and begin work on the models that provide the greatest overall value to the firm. This will change from firm to firm, so it is important to engage with a trusted service provider that understands the technology and can assume project management responsibilities. Whilst it is essential to engage the firm’s partners in this discussion, you do not want to commit attorney time if you have a vendor that can do the heavy lifting.

Step 4: Ericsson and Gladwell return!

AI is now part of your workflow. You are seeing value through the low hanging fruit of AI powered ECA and Priv identification, and you have a roadmap for model development that aligns with the firm’s priorities. Now begins the purple belt phase of building the models and enabling your litigation teams. This process is not an overnight win. It will take time, collaboration with clients, assistance from services firms, and commitment to the cause. You are mastering your craft. Every day you commit to improving your firms AI initiative you bring your clients greater value. This may not be obvious day to day, or even month to month, but over time the value will be realized. Legal teams are slowly waking up to the idea of data being an asset class with incredible value. The purple belt phase is the realization of that value.

Step 5: The law firm of tomorrow

The Brown and Black belt stages represent a bank of skills acquired and ready to be deployed on command. Unconscious competence. There is a need, and you have a solution. By now your firm has been building models for some time. Perhaps years. These models would form a catalogue that practice groups can select from based on client, industry, need, and any other category considered relevant to the firm strategy. However just like in Brazilian Jiu Jitsu, the learning about AI is never complete. If humans can innovate, they will. Add competition and the growth of new ideas and applications will be never ending.

These stages are used as an example of what a roadmap could look like. Naturally, there are details missing. The key is to appreciate the commitment needed to achieve AI enablement. It is a process that cannot be delayed, less you risk facing the opposing counsel version of a black belt while you are still learning how to defend yourself. Very quickly you will find your firm on the wrong end of a submission to an opponent with a strategic advantage you cannot climb over. What matters most is taking that first step and appreciating the process. Be clear on where you want to be, commit to the art, and work at it one goal at a time. That is the Gracie BJJ way.

Chip Delany on Email
Chip Delany
Strategy Director at Lineal Services, previously worked as a strategist for Legal AI tech firm NexLP and before that as a consultant in continuous improvement and labor modelling. Australian National and US permanent resident.

Share this article