TIL that my AI chatbot was generating fake legal citations in our contracts

Last week I was using some AI tool to draft clauses for a client agreement. I thought I'd double check one citation and it was totally made up. Like the case didn't exist, the judge wasn't real, the whole thing was fake. I almost sent that to a real lawyer. Now I'm paranoid about trusting any AI output for anything important. Has anyone else caught AI hallucinations in their work documents?

3 comments

3 Comments

stella_murray25d ago

You're being a little dramatic here. AI tools are helpers, not replacements for human judgment. Nobody should be copy pasting legal citations without checking them yourself anyway, that's just basic professional responsibility. The real issue isn't that the AI hallucinated, it's that you almost trusted it blindly. Lawyers check everything, that's what they get paid the big bucks for.

kim96325d ago

A paralegal friend once sent me a screenshot where the AI invented an entire court case out of thin air...

kelly_nelson9525d ago

A buddy of mine once nearly sent an AI-drafted lease to a client that had a fake county name in it.

-1