Questions you ask artificial intelligence about a legal case are not protected by attorney-client privilege, a federal court has ruled.
Getty
Anthropic’s Claude, OpenAI’s ChatGPT, and Alphabet’s Gemini might be great for helping with research or creating spreadsheets, but one thing they definitely aren’t is your lawyer. If anything, turning to AI for legal advice could backfire, as it did with an accused white-collar criminal in New York who tried to shield from federal prosecutors his communications with Claude — those where he asked him for a legal defense strategy.
It seems obvious that you shouldn’t rely about AI for legal advice, particularly if something important is at stake, such as your freedom or a lot of money. The news is that even discussing legal matters with your favorite chatbot can be dangerous.
In February, US District Court Judge Jed S. Rakoff of the Southern District of New York ruled that large AI language models are not lawyers and that any communications with them can be used against defendants like any other non-privileged document. The judge noted in his decision that his ruling “seems to answer a question of first impression nationwide”—meaning no other court in the US has ruled on this issue. His conclusion about the privilege would probably apply to civil disputes as well.
The privilege issue arose in a criminal securities fraud case against Texas businessman Bradley Heppner, the former chairman and CEO of publicly traded GWG Holdings, who was accused of defrauding investors of more than $150 million. Heppner, 60, was doomed by a jury this month and awaits sentencing. He faces a maximum sentence of 20 years in prison.
According to court records, after Hepner learned last year that he was being targeted by federal investigators, he began asking Claude to outline a “defense strategy” he could use in court against criminal charges.
When FBI agents later raided his home and seized various electronic devices, they found 31 saved copies of documents and conversations Hepner had with Claude about his criminal case. After Hepner’s lawyers argued that those documents were privileged because he had those conversations to prepare to speak with his lawyers, the government separated them. He then asked Rakoff to rule on whether they were in fact protected from government inspection under either the attorney-client privilege or the work product doctrine.
On February 17, Rakoff decided it wasn’t. In his 12-page order, Rakoff supported his finding with a series of legal arguments.
First and foremost, Rakoff pointed out, “Because Claude is not a lawyer … that alone manages Heppner’s claims of privilege.”
Not only that, Rakoff wrote, but he noted that any privacy for an accused criminal — as with lawyers — requires a “human relationship of trust.”
“There is no such relationship, nor could there be, between an AI user and a platform like Claude,” Rakoff wrote.
The judge went on to point out that Claude’s parent company, Anthropic, specifically notes in Claude’s terms of use — which Heppner agreed to — that it reserves the right to all “data,” including users’ conversations with Claude, which can then be turned over to authorities “in connection with claims, disputes or litigation.”
Rakoff went on to observe that the only possible way Heppner’s communications with Claude could have been kept confidential is if his attorney specifically instructed him to seek legal advice from Claude, in which case it could be argued that Claude was acting as an agent of the attorney. But in this case, Heppner did so on his own initiative, and his lawyer only learned about his conversations with AI after the fact.
Additionally, when prosecutors asked Claude during the case if he could provide legal advice to users like Heppner, he replied, “I’m not a lawyer and I can’t provide formal legal advice.” He then recommended contacting a licensed attorney.
New York attorney David Feldman, the ruling provides a “stark warning” to business owners who may already have made use of large artificial intelligence language models for various legal strategies. he commented on his company’s website. “The ruling makes a clear distinction between human legal counsel and digital assistants,” he wrote, noting that the main takeaway is that Heppner’s use of Claude could now be weaponized against him by federal prosecutors. “Attorneys should consider cautioning their clients that using open AI (systems) may not be considered confidential communications,” Feldman wrote.
Feldman said in a follow-up email to Forbes that he believes this is a growing problem among C-suite executives who believe AI will or can replace expensive lawyers. “I tend to spend too much of my time cleaning up messes created by this type of action,” he wrote. “With the advent of artificial intelligence, customers are even more encouraged to try to handle things without our advice.”
Alan Pence, a small business consultant who has focused on AI development in recent years, said the news was not surprising given that, legally, conversations with large AI language models are “all discoverable” in court. In other words, executives should be careful not to share sensitive information with large language models like Claude, ChatGPT, Gemini, or others.
“AI is just like Google searches, docs and email,” Pence wrote in an email to Forbes. “Proceed accordingly.”
The decision is also emblematic of a new reality for businesses of all sizes, said Samir Black, founder of Georgia-based technology firms Data Pros Consulting and Forseti Verify.
“It’s the new angle of attack … this is very much in line with the new era of research,” Black said of AI talks like Heppner’s. “It’s not about the files on your computer anymore. It’s the conversations with AI and chatbots.”
Do you want to be more successful? Contribute to the weekly Forbes Careers newsletter to get insider tips and information.
More on Forbes:
