Errors found in US judge’s withdrawn decision stink of AI

A US district court judge has withdrawn his decision in a biopharma securities case after lawyers noted that his opinion referenced fake quotes and other erroneous case information — mistakes mirroring errors in other legal cases that have been attributed to artificial intelligence tools.

In a letter sent to New Jersey Judge Julien Xavier Neals, lawyer Andrew Lichtman said that there was a “series of errors” in Neals’ decision to deny a lawsuit dismissal request from pharmaceutical company CorMedix. These citation errors include misstating the outcomes in three other cases, and “numerous instances” of made-up quotes being falsely attributed to other decisions.

As reported by Bloomberg Law, a new notice published to the court docket on Wednesday says “that opinion and order were entered in error,” and that a “subsequent opinion and order will follow.” While it’s not unusual for courts to make small revisions to decisions following a ruling — such as correcting grammatical, spelling, and style errors — major modifications like removing paragraphs or redacting decisions are rare.

There is no confirmation that AI was used in this case. Nevertheless, the citation errors carry the same telltale signs of AI hallucinations that have appeared in other legal filings as lawyers increasingly turn to tools like ChatGPT for help with legal research. Attorneys defending MyPillow founder Mike Lindell were fined earlier this month for using AI-generated citations, and Anthropic blamed its own Claude AI chatbot for making an “embarrassing” erroneous citation in its own legal battle with music publishers — just two of many examples showing that LLMs won’t be replacing real lawyers anytime soon.