Can an AI lawyer really replace a human one?

w/Premium Content

Are we on a path towards a future run by AI?

Since OpenAI’s ChatGPT woke the world up to the power of artificial intelligence, this is the question I see being asked and debated a lot. It’s a question that spikes the imagination and allows us to see the best and the worst of tech at work. It’s easy to see how the answers to the science fiction question can run away with themselves. 🤦‍♂️ The end of the world is nigh!

But here’s the thing, we’ve been here before, haven’t we?

I’m old enough to remember life before the Internet, smartphones and TikTok. When I was at school, the electronic trickery known as a calculator was banned from exams. The only assistance I could rely on was a slide rule and my well-thumbed log book.

But today, I couldn’t imagine working without a laptop, WiFi and a bunch of software tools from the likes of Notion, Canva and Ghost. I hardly put pen to paper anymore and haven’t bought Tipex in 20 years!

With all these marvellous productivity tools, we didn’t have two decades ago, you’d think we’d be on our way to putting humans out of business. Except that that hasn’t happened. In terms of productivity in the United States, the last 20 years have been the most economically stagnant period of the last century. In some European countries, productivity is actually declining over the same period.

Productivity performance is looking through the rear view mirror, what about looking straight ahead? In this market forecast report by Halperin, Mazlish and Chow, (it’s a long and technical assessment of the impact of transformative AI on interest rates), they conclude there will be a  short term impact from the current advances in AI. But, “the market seems to be strongly rejecting timelines of less than ten years, and does not seem to be placing particularly high odds on the development of transformative AI even 30-50 years from now.”

So, can AI really replace human talent, knowledge and capability?

When it comes to the law, it seems like the answer is yes.

AI lawyers doing what human lawyers do

The role of AI in the legal process is not a brand new subject;

• In 2020, a California task force dedicated to exploring ways to expand access to legal services recommended allowing unlicensed practitioners to represent clients.
• Also in 2020, the American Bar Association told judges using AI tools to be mindful of biases instilled in the tools themselves.
• UNESCO, the international organisation dedicated to preserving culture, has a free online course covering the basics of what AI can offer legal systems.

But the news that DoNotPay are to use ChatPGT to represent a defendant in a US court has pushed the debate into the spotlight. Not just because it’s a well-timed stunt that has shone the spotlight on this 6 year old firm, but this is the story of expensive lawyers being replaced by artificial intelligence.

It’s a story about inclusion!

My client pleads Not Guilty

In February 2023, if the hype and self-promotion by DoNotPay is to be believed, an artificial intelligence chatbot programmed to respond to questions and hold a conversation, is expected to advise two individuals fighting speeding tickets in courtrooms in undisclosed cities.

Their lawyers will wear CATXQ Smart glasses, a relatively cheap and low tech pair of glasses with built in bluetooth and a hearing aid. They look like ordinary glasses but contain a hearing aid that delivers audio to the ear of the lawyer and, via bluetooth, to an app on the lawyer’s smartphone, connected to DoNotPay’s AI.

The claim is that the lawyer in the court will only say what the AI tells them to say.

There’s little doubt that this is a stunt. If it goes ahead, DoNotPay have committed to pay the fines if they lose.

"Most people can't afford legal representation," Browder said in an interview. Using the AI in a real court situation "will be a proof of concept for courts to allow technology in the courtroom."

If this wasn’t getting attention enough, DoNotPay CEO Joshua Browder tweeted that his company will "pay any lawyer or person $1,000,000” if they were to do the same thing in the US Supreme Court, the highest court in America. This is a safe bet to make as it will definitely not happen. The Supreme Court has strict rules on the use of electronic gadgets and outside assistance.

As for the two speeding fine cases, Browder is tight lipped over exactly where and when the cases will take place for fear of being prevented.

Who are DoNotPay?

DoNotPay is a "robot lawyer" startup known for helping people negotiate lower bills. It was founded by Joshua Browder in 2015 while he attended Stanford University. The firm states on its website that its mission is to help consumers “fight against large corporations and solve their problems like beating parking tickets, appealing bank fees, and suing robocallers.”

The lawtech firm claims to have already helped over two million people fight small claims, such as speeding fines, parking tickets, consumer disputes and overcharging.

Up until recently, DoNotPay have been using IBM’s Watson to run it’s AI law firm. Watson is famous for winning the TV game show Jeopardy in 2011. However, the world changed for DoNotPay, as it did for many other sectors, with the release of ChatGPT last November and it’s chat capability to “talk” with the large language model.

Examples of what DoNotPay can help with:

• How to Cancel BT Sport Before You're Charged Again
• Harassed by NCO Debt Collectors? Here's How to Stop Them!
• Secret Hack That'll Help You Avoid Grubhub's Hidden Fees
• How to Remove My Case From The Internet Instantly
• Should I Pay Debt Collector or Original Creditor?
• Can a Landlord Charge for Plumbing Repairs?

Here’s The Thing: Whether DoNotPay go ahead with the court cases or not, this one story has demonstrated how legal representation will/can/might change in the near future. Most significantly, making legal representation affordable and accessible to those who can’t afford it today.

Legal representation and inclusion

According to a report in the Washington Post, “*80% of low-income Americans don't have access to legal help, while 40% to 60% of the middle class still struggle to get such assistance.* With those kind of stats, there’s clearly demand in the market. Which partly explains how DoNotPay have already managed to raise of $27 million in startup investment.

However, it’s not all plain sailing for AI robot legal representation. There are still hurdles to overcome, such as;

• the AI is not a lawyer with a licence. To practice law in the US requires passing the Bar Exam  and getting a licence to operate. DoNotPay’s AI has neither.
• the privacy of the court prevents proceedings being transmitted outside. To make this work, DoNotPay will have to communicate with the lawyer from outside the courtroom. This is itself could prevent the DoNotPay stunt from going ahead.
• over-reliance on incomplete and/or biased information leading to negligence. In 2020, the American Bar Association warned of using AI tools because of the danger of inherent biases in the language models. This was pre-ChatGPT, which has already shown a terrific ability to produce total garbage without detection.

Having said all that, in 2020, the California Bar Association called for the greater use of AI tools to provide legal services. So the question now is, “how good are they?”

We now that legal chatbots are not new. They’ve been used for some time for answering basic questions with pre-formated answers. However, ChatGPT has taken this to another level with the ability to generate unique answers specific to the question being asked. (The disclaimer that the answer can be complete garbage still stands though!)

GPT takes the Bar Exam

To test exactly how good ChatGPT is at the law, I just read a paper by 2 law professors from Chicago and Stanford. The two law professors have just tested OpenAI’s “text-davinci-3” on a section of the Bar Exam. (Note: “Text-davinci-3” is the large language model commonly referred to as “GPT3.5”.)

To give this some context, it usually takes a student seven years to learn the law sufficiently well enough to pass the Bar Exam and earn their licence to practice law. Having put the GPT3.5 model through sections of the Bar Exam, the professors concluded that the AI “exceeded their expectations.”

They reported that GPT3.5 “achieves a passing rate on two categories of the Bar and achieves parity in one.”  Remember, this is “vanilla” GPT3.5, there has been no fine tuning to turn this into a legal specific AI.

To be clear, the sections of the Bar Exam used for the test are multiple-choice questions where students rank the pre-defined options for answers.

To test that the results of the GPT3.5 test where not simply good guesses, the professors tested for “correctness.” They found that the “rank-ordering of possible choices is strongly correlated with correctness in excess of random chance, confirming its general understanding of the the legal domain.”

In their conclusion, the two law professors, Michael Bommarito II and Daniel Katz, said “we did not expect GPT3.5 to demonstrate such proficiency.”

Whilst their white paper has not been peer reviewed, it is being seen as a credible examination of the capability of AI in legal representation. And remember, the large language model used for this research had not been fine-tuned for this examination.

Which led the professors to predict that an “LLM may soon pass the Bar Exam.”

Read the White Paper “GPT takes the Bar Exam”

Further Reading

Snapple use ChatGPT to create user engagement and brand awareness

Consumer brands are exploiting emerging technologies to find new ways to engage with customers. Following its moves in the Metaverse and NFTs, the iced tea drinks brand, Snapple, has wrapped its brand around ChatGPT to create the fAIct Generator, for user driven engagement and brand awareness.

🤔 Wiser! - technology, innovation & strategyRick Huckstep

ChatGPT wrote this article (with a little help from me)

The promise and potential of generative AI has created much excitement, headlines and discussion. There’s little doubt that the tools using artificial intelligence to create content are fascinating. But can they be relied upon?

🤔 Wiser! - tech innovation & strategyRick Huckstep

7 AI Creation Tools To Save Time and Be More Productive

The widespread use of generative AI has exploded in 2022. First, with the release of GPT-3, followed by DALL-E and Stable Diffusions, there are now countless content creation and productivity tools utilising AI. Here are 7 AI tools I use the most.

🤔 Wiser! - tech innovation & strategyRick Huckstep

Future of Emerging Tech with Paul Armstrong

In this video interview with Rick Huckstep, digital strategist Paul Armstrong discusses the future of emerging technology, including virtual reality, blockchain, and artificial intelligence.

🤔 Wiser! - tech innovation & strategyRick Huckstep

🎙️Big Tech Little Tech Podcast Episode #14

Artificial intelligence is a broad topic. From science fiction to enterprise-level supply chain product tracking, AI has had time to grow up. Are people still generally suspicious of AI, or will it prove its usefulness in time?

🤔 Wiser! - tech innovation & strategyRick Huckstep

How to create unique content with ChatGPT and QuillBot

Content creators know how time consuming the process of writing long form content is. From research to storyboarding, writing to editing. But, with generative AI tools like ChatGPT and QuillBot, all of that has changed. Here’s one way that you can save time and make money quicker using AI.

🤔 Wiser! - tech innovation & strategyRick Huckstep

w/Promotions

👻 Turn Your Audience Into A Business.

Ghost is a powerful app for new-media creators to publish, share, and grow a business around their content. It comes with modern tools to build a website, publish content, send newsletters & offer paid subscriptions to members.

Try Ghost completely free for 14 days.

Try Revolut Premium for free for 3 months!