I agree with the “lesson has been learned” justification of not having any penalty added. At this point, it wouldn’t really add much.
Penalties are supposed to stop people from violating the law. Getting your face and firm known across the world seems to be sufficient deterrent to future offenders because the “oops I didn’t know” defense won’t apply a second time.
That is true, but sanction in the court of public opinion isn’t the same as sanction in a real court. I’m not sure a judge should be taking bad publicity into account when handing out punishment - they should likely be treated as independent.
If someone wasn't a lawyer and did something similar, they would face little to no negative effects socially. Why should a lawyer get a free pass, especially considering they should know better?
Unless he wasn’t the first and this possibly wasn’t his first time. All we really know is it’s the first time for someone to get caught and penalized for this. Serious penalties seem reasonable in my opinion.
It’s possibly his first offense (no knowledge on that). What I was trying to say is that folks rarely get caught their first time doing something nefarious, which makes stiffer penalties important in my opinion.
Note that it appears the lesson was not learned...
This lawyer at first cited these (fake) cases, and then when asked to provide copies of the cases then faked the copies too, also using ChatGPT.
In my view, that is being caught (asked to provide copies, which is unusual in the legal world, and presumably only done because the judge had failed to find them), and then doing the crime again (using chatgpt to fake these cases)
Why was this downvoted? I don’t think that means he did the right thing, but that’s probably what he thought rather than being actively malicious. That makes him incompetent, sure, but there’s no point in making up motivations that probably don’t exist because you’re annoyed with him
Yeah you should be allowed to file more than just a handful of fake cases before you’re penalized by the body that exists to make sure you don’t.
I mean, as long as the internet is mean to you about it. Now, if the internet isn’t mean enough you should be penalized some amount that accounts for how mean the internet was.
In fact, I think doctors that amputate the wrong limb should not face malpractice lawsuits if the greater metropolitan areas local news station ran more than 2 stories about it reaching a population of at least 2 million viewers.
I strongly disagree. "Professional embarrassment" isn't really much of a punishment.
> sufficient deterrent to future offenders because the “oops I didn’t know” defense won’t apply a second time.
I disagree with this as well, because the error this attorney made wasn't really about using AI. It was about not doing due diligence. Surely even freshly minted lawyers are well aware that doing due diligence is most of what people pay lawyers for in the first place.
> "Professional embarrassment" isn't really much of a punishment.
That really depends on the profession. If you're a line cook at Waffle House and you get fired for coughing on the eggs, then that's probably not career-ending. If you're a software developer who deletes the production database and causes a week of downtime then that's worse, but likely still not terminal. On the other hand, this guy's income depends to a large extent on convincing his clients that they will be represented competently. Being known primarily for breathtaking incompetence will probably have a pretty significant effect on his future income.
> Being known primarily for breathtaking incompetence will probably have a pretty significant effect on his future income.
Perhaps so, but most people won't ever know the he's breathtakingly incompetent. It's certainly not a career-ending thing. For an error of this scale, I think that a strong argument can be made that he should choose a different career.
"Penalties are supposed to stop people from violating the law."
That only works based on prior evidence. Either somebody prior to the potential violation was penalized or the weren't; either people fear, based on the prior, suffering the same penalty, or no worries mate the prior offender wasn't punished.
I am not a lawyer, but I can offer some general information about lawyer sanctions. The appropriate sanction for a lawyer who submits fraudulent court filings can vary depending on the jurisdiction and the specific circumstances of the case. However, generally, lawyer sanctions are intended to maintain the integrity of the legal profession and protect the public.
Some potential sanctions that a lawyer may face for submitting fraudulent court filings could include:
Reprimand: A reprimand is a formal expression of disapproval by the bar association or disciplinary board. It is typically the mildest form of disciplinary action.
Suspension: Suspension involves the temporary removal of a lawyer's license to practice law for a specified period. During the suspension, the lawyer is unable to practice law.
Disbarment: Disbarment is the most severe sanction that can be imposed on a lawyer. It permanently revokes the lawyer's license to practice law, effectively ending their legal career.
Probation: Probation can be imposed along with other sanctions or as an alternative to more severe sanctions. It typically involves specific conditions that the lawyer must meet, such as additional education or supervision.
Monetary fines: The lawyer may be required to pay fines as part of the disciplinary process. These fines are intended to serve as a penalty for the lawyer's misconduct.
It's important to note that each case is unique, and the actual sanction imposed will depend on the specific facts, the jurisdiction's rules, and the discretion of the disciplinary authority. Legal precedent may be considered in determining the appropriate sanction, but the final decision will be based on the facts and circumstances of the case at hand. For a more accurate assessment of the potential sanctions in a specific jurisdiction, it would be advisable to consult with a legal professional or review the rules and regulations of the relevant bar association.
I don't understand what kind of person wouldn't check. What was he thinking? "Oh I have a computer now that knows how to be a lawyer, so I don't need to know how to be a lawyer any more"...?
A funny detail is that he did have the sense to doubt the accuracy of the information, but delegated verifying its accuracy to the same bot which manufactured it in the first place.
> Lawyer: "Is Varghese a real case"
> ChatGPT: "Yes"
> L: "What is your source"
> C: "Upon double-checking, I found that the case Varghese v South China Airlines [...] does indeed exist."
> L: "Are the other cases you provided fake"
> C: "No, the other cases I provided are real and can be found in reputable legal databases"
I'm struggling to imagine a mental model of the LLM for which that would make sense. A human who's willing to a lie a little, but comes clean when called out? A robot that mostly doesn't make mistakes, and is more likely to catch its own than make more?
GP means the mental model that the lawyer had of ChatGPT: why did he think that he could check ChatGPT's work by just asking it "hey are you sure about that"?
This is not an unreasonable mental model. You can ask ChatGPT to give you a program, ask it "is this correct?" and it'll find and fix bugs. To a layperson it looks like it is capable of double checking its work and finding an error. Why would it be any different here?
(the answer of course that the LLM doesn't actually search the internet and/or doesn't have access to a law database it can query)
Honestly, at least when it comes to code in widely used languages, GPT-4 is very good at catching mistakes in generated output after one makes a simple request for a second check. In most cases, there isn't even a need to explain what the specific issue, concern or error in the provided code is.
This does make sense as, beyond what has been typed, there is no memory implemented in most of these models, so revisions are currently the only game in town to get more accurate results.
What surprised me most concerning this entire situation is that the model did insist on being correct. Normally ChatGPT has been set up to be a bit cautious and more likely to admit to having made a mistake, to the point where if you ask in a direct manner like this lawyer has done, the model may claim to have been incorrect, even when the output was actually correct, in my experience. Bings implementation, of the same underlying model, meanwhile can be so forceful in trying to convince users that the output is correct, even when provided with online resources that show the oposite, that it would not be unreasonable to feel gaslit by that LLM.
The rest of this situation was not very surprising and I have do admit, I am happy that this was caught right away. Lawyers actions have a massive impact on countless people every day, if this had not become such a public scandal right away, perhaps a lot of defendants would have suffered under improper representation due to reliance on imperfect models.
My layman understanding is it’s not grounded. GPTs are schizophrenic, but unlike schizophrenic man who is delusional and failing to be bound onto the reality he is in, GPT is actually up in the air.
That and that it’s just a language model, an approximation of neither the world, nor a body of knowledge, but of English, and not an answering machine at all.
For ex. I tried asking ChatGPT about cartoons from childhood. I wrote "What was that cartoon in the 1980s that was based on some kind of gummy candy?" and it correctly identified "The Adventures of the Gummi Bears". I wrote "Sing the theme song for me" and it produced the song missing the first verse. I wrote "That is missing the first verse!" and it produced the whole correct song.
On the other hand, when I asked it to describe the instrumental 90s X-Men theme song, it tells me:
'...the lyrics are epic and uplifting, with lines like "We're the X-Men, we're the best there is at what we do." The song also has a sense of urgency and danger, with lines like "We're fighting for our lives" and "The mutant race will survive"...'
When I put "The X-Men theme song doesn't have lyrics" it readily accepted the correction but unlike getting the missing first verse I wasn't really getting any verifiable information by making the correction.
And of course it was happy to tell me about a nonexistent Gummi Bears / Rescue Rangers crossover episode.
This reminds me of a story, I believe referring to Pascal's demonstration of his (newly invented, entirely mechanical) calculator to the Royal Society. He showed that pressing certain levers in the correct order means you want to operate on certain numbers, and certain other levers mean choosing the operation you want, and then by cranking his calculator you could read off the answer of doing your operation on your numbers. World's first calculator! Someone asked: if you press the levers wrong, do you still get the right answer?
Their mental model is "magic" and they don't understand the details of how magic works, because it's magic.
The same kind of people who don't hold the steering wheel with their hands or pay attention to the road when driving on "autopilot". Turns out there are a large number of them. There's a reason Google went straight for fully driverless.
Yes, ideally everyone should check every e.g. Wikipedia citation before relying on it, but obviously most people don't.
In the past, the only way to come across completely made up court citations regarding obscure bankruptcy procedures would've been that somebody intentionally made them up, probably maliciously.
Given how unlikely that is, if you didn't understand that GPT just makes up text that resembles what you want to see, I'd understand trusting it as you would a search engine.
It's not like lawyers are visiting court clerk's office to verify every case they find in some online database.
> I'd understand trusting it as you would a search engine.
But he didn't do that. Lawyers don't just "trust" the results from a search engine. Part of their job is to confirm facts, not just parrot them.
> It's not like lawyers are visiting court clerk's office to verify every case they find in some online database
No, they don't. Instead, they look them up in the databases that collect these filings. It's effectively the same thing, just much more efficient than physically travelling to various offices.
It's a correct use of literally, but I think the change to the dictionary definition is in error. It's simply an example of hyperbole, which is a thing you can do with any word.
Did you respond to the wrong comment, or confuse me for someone else? The comment you replied to was my only one in this thread, so if neither of those then I have (ahem) literally no idea what you're on about.
I wasn't correcting you. I was agreeing with you. I was correcting the dictionaries that have (IMO) got it wrong. Now, in this paragraph, I am correcting you. ¯\_(ツ)_/¯
> It's not a hyperbole lmao.
I disagree, obviously, but perhaps an explanation is in order so that we can be sure we're talking about the same thing (and give you something more specific to argue against if you still disagree).
The difference between "X is Y" and "X is literally Y", when both expressions are figurative, is that the latter is meant to be strengthened by overstating the degree to which X is in fact Y. I contend that this is a hyperbolic use of "literally".
I’m not making a claim that it is “objectively incorrect”. I am merely attempting to discourage this kind of usage, because I this it is a better course for language for it to not be used in this way.
(Also, whether it is included in dictionaries doesn’t really matter as to “correctnesss”. All words are real words.)
It makes sense to avoid it when there's any chance of confusion, much like any other construction. I don't see that that meaningfully motivates avoiding it in circumstances where there is no chance of confusion.
I guess I meant to emphasize the fact that while they usually bullshit, they do so in a structured and generally acceptable way. In this case he (literally) rolled the dice and delivered what the tea leaves (literally) told him to.
Because he used AI to fabricate case law. Imagine if you're on the other end of this case, and the opposing lawyers are creating citations to help support their case and create a strong argument (or worse, use a fabricate case as precedent). This has real-world impact. Sure, use AI to assist but check the work!
What I am interested in is: Errors will be made by any professional, and so, lawyers. Why would this one justify a call for disbarment? I was hoping for a little more than the hobbyist lawyer tv show vibes I was getting.
It’s “Qualified Immunity” for lawyers. Your literal job is to know not to do this, “Oops got caught don’t do this again” basically means zero consequences.
I look forward to everyone else going through this courtroom get benefits from the “your first time caught lying to this court is free” policy.
Bullshit. If it is your job to check these things and get it right, “Not looking into it” is just laundering responsibility for not checking, especially when caught and then doubling down.
For a lawyer, not doing the job one has been paid for is not just incompetence, it's considered malpractice.
In this case, the original malpractice of failing to check the falsified cites was compounded by submitting falsified opinions to the court (in support of the falsified cites), which is fraud and would be criminal in most states. It's blatantly obvious that he used ChatGPT to generate falsified opinions to try and cover up his original fuck-up after traditional means of verification (i.e., using a legal database like LexisNexis or Westlaw) failed to substantiate any of the ChatGPT cites he originally submitted.
In most states, this lawyer would be disbarred, or at the very least suspended for many months, for this sort of conduct. This lawyer should consider himself lucky that the court hasn't yet entertained referring this case to the NY DOJ for criminal prosecution.
"Causes of disbarment may include: a felony involving moral turpitude, forgery, fraud, a history of dishonesty, consistent lack of attention to clients, alcoholism or drug abuse which affect the attorney's ability to practice"
This was definitely an attempt at forgery and fraud. It might not be something that can be prosecuted as a felony, but it's close enough that if I was a member of the bar I would want to draw a firm line.
First of all, I think it's reasonable to start with the assumption that a case is fake until you validate that it is real.
Secondly, that's a ridiculous place to put the goal post. He didn't write the argument he submitted. He didn't even do basic vetting of it. It's fraudulent because he submitted work as his own that he didn't do, it's negligent because he didn't verify that the work was actually good before submitting it on behalf of his client.
He was put on notices that the citations (and thus the cases) were fake when the opposing counsel informed him they couldn't substantiate any of the citations.
Missing a deadline, failing to realize that a case existed that was applicable, misreading forms, etc... Mistakes/errors are one thing.
But submitting cases that don't exist, and then doubling down and saying that they do? There's layers of incompetence and apathy that had to happen here to get to this point, from using a tool without researching it at all (any research into ChatGPT as a legal aid would bring up warnings), to ignoring the warnings that OpenAI itself provides, to not checking the work that tool spit out (seriously, it's not hard to validate that these rulings don't exist), to being told that the cases don't exist and then repeating the entire process...
It's not just they made a mistake, the way they made the mistake reveals how little care they are putting into the research they're doing. The only way this could happen is if ChatGPT was the only source this lawyer was using. Because any other source: fact checker, Internet search, etc... should have noticed either that the cases didn't exist or should have at least prompted "why is it that these cases only show up in ChatGPT and not in any of the other sources I'm looking at?"
I don't know that they should obviously be disbarred, but I would have thought that it should at least be on the table. Massive levels of incompetence can be grounds for disbarment I believe. Whether this lawyer acted incompetently enough? :shrug: But we are talking about incompetence here, not just a mistake/error. If the takeaway for this lawyer from the penalties is just "now I know not to rely on ChatGPT", then that's not really addressing the problems that led to this situation in the first place.
There will be an eco-system of tooling that will come either in form of other GPTs or standard software that will have to perform audit on ChatGPT responses.
Only use ChatGPT if you have a way to audit the response, example: if you asked it to write some code - compile it immediately. Both Bard and ChatGPT gave me code that is near complete but failed compilation.
Or ConnsensusGPT -- where you have to get near similar response from 3 GPT models to go to next step.
Experts can chime in. The idea of using a GPT model to validate another is recursive and powerful, if it is possible.
"I simply had no idea that ChatGPT was capable of fabricating entire case citations or judicial opinions, especially in a manner that appeared authentic"
This is the main criticism of the service, how did a lawyer miss this talking point?
I mentioned it before, but I think the wording around this hype cycle has duped non-technical people into thinking the true singularity-level AI revolution just happened.
He was probably in a rush. People use these tools to offload work they might otherwise have done if they had more time.
People who understand the systems limit their use to tasks where small occasional errors are acceptable. People who don't understand the systems are happy to accept any plausible-soundong results, especially if they don't have time to do the work themselves.
I would not be surprised if, sometime in the next year, we see a doctor or two being sued for malpractice after accepting an incorrect diagnosis from ChatGPT. When people are rushed and overworked, and the system is usually correct, these kinds of incidents are almost certain to happen at the scale of an entire society or profession.
As a non-expert, the legal argument against sanctions for the lawyers appear to boil down to that no malice was involved (ignoring the point about whether a reasonable lawyer would have made the mistakes), and that the "professional embarrassment" from news coverage is sanction enough. (Detail is in #45 - https://storage.courtlistener.com/recap/gov.uscourts.nysd.57... )
We will see what happens after the court appearance scheduled for today at 12:00 EDT.
I don't understand why "professional embarrassment" should come into account when considering sanctions. If the actions were worthy of sanction, the lawyer should be sanctioned regardless of the news articles written about it all.
What is the purpose of sanction? As far as I know it's to stop him doing something similar again, and deter others from trying it.
He's not going to do the same thing again unless he is literally mad. That would be total career suicide. No amount of sanction will make a difference.
Other lawyers are likely read about the case and know that if they do it, they will either be more embarrassed in the press or will face sanctions. The sanctions are unlikely to make the case more well known than it already is.
There are multiple theories (and philosophical underpinnings) on the purpose of punishment. The one you've identified is deterrence: We should punish so that people don't do the bad thing in the future.
But another one is retribution: By causing harm, you should have some harm done back to you. To even the scales of justice as it were.
Your argument is a very solid one for why a sanction is not needed for deterrence. The court of public opinion has punished him enough.
But what about when the next attorney does this, or the 100th, and it's not novel enough to get a bunch of NYTimes coverage? If they get sanctioned, will they not say "What about the first guy? He didn't get sanctioned?!" Will there be some doubt that justice was really done? Think about the cases where some rich white kid skates by with a slap on the wrist for some widely-publicized crime and the judge says, in effect "Look, this is a good kid from a good family who's been widely castigated in the press. He's getting kicked out of his Ivy league school and probably won't even get hired at a top law or consulting firm in the future. Is it really necessary to send him to jail like a poor person? Hasn't he suffered enough?"
I'm not saying it's the best or only argument, but there is an argument for the court imposing sanctions purely on conduct with the court, and ignoring what the newspapers are doing.
When the 100th attorney does this and doesn't get mentioned in the New York times, they won't be able to use the argument of "professional embarrassment" to reduce their sanction.
Right, exactly. So the question is: Is it just for a court to impose punishment only on some bad attorneys based on other consequences in the world? There's not a single simple or obvious answer to that question. It depends on what you think the purpose of the punishment is.
The offending lawyer isn't a name partner of the firm -- almost certainly he will be fired. The professional embarrassment will definitely make it difficult for them to get a new job.
That said, I suspect this is just the defense they're making. You argue with the facts you have, not the facts you wish you had, and the facts here are pretty bad.
Personally I don't really agree with the defense -- this is a serious fuck-up and there should be a clear signal that this is Not Okay. "I wasn't malicious, I was just lazy" isn't the best look, even if it's the best the guy has.
I think he should be disbarred. He either deliberately ignored to check the validity of references using real sources or he is an imbecile who took AI answers for granted and imbeciles should not be allowed to practice law.
Plenty. But I've never made a mistake that is on this level of incompetence. If I did, I'd not only expect to be fired, I'd be looking at quitting in shame.
Since when does the US have a problem with harsh punishment? And no it is not a mistake. It is deliberate act of using false information potentially ruining someone's life.
> “I simply had no idea that ChatGPT was capable of fabricating entire case citations or judicial opinions, especially in a manner that appeared authentic,” Mr. Schwartz said in a declaration filed on Tuesday with the judge, P. Kevin Castel of Federal District Court.
Your whole occupation literally depends on not inventing bullshit on the spot, and yet you use technology that is marked, by its creator, with big letters, as a bullshit inventer. What can go wrong?
Agreed, and happily they're already regulated. Practitioners of engineering and medicine can be criminally liable for negligence. The government also regulates the profession by granting licenses and preventing people from practicing in these fields without a license.
> Can you imagine sending someone to jail or leaving someone to walk free just because you were too lazy to actually do your job??
This was a civil lawsuit wherein a passenger got injured on an aircraft. I'm not trying to lessen what occurred, just trying to accurately characterize the risks associated with this professional misconduct.
From what I've read about the US justice system, you don't have to imagine very hard—it just happens through institutionally supported channels like pushing people to plea bargains and underfunding public defenders' offices.
"Can you imagine sending someone to jail or leaving someone to walk free just because you were too lazy to actually do your job?"
Interesting question. Most people have a lot of bias that wouldn't allow themself to see that the person they locked up might be innocent. You'd have almost nobody working as a judge, lawyer, cop, etc if the cognitive dissonance were allowed to creep in. In the vast majority of cases they only leveled justice against the person because they thought it was right.
That Mr. Schwartz offered this as a kind of defense makes me seriously question his ability as a lawyer. Even if he had a very high degree of confidence in the tool, he has to know that you fact-check anyway. Just as you would if a human produced it.
It was a complete abdication of professional responsibility to his client. Doesn’t matter whether he pulled the citations from a paralegal, ChatGPT, or Google - it’s his job to check and validate before submitting anything to the court
What do you mean? He seems to be facing potential legal trouble, and on top of that he and his firm will forever be known as the dumbasses who submitted fake court decisions in a real case.
This is just how a lot of normal people use technology. They aren't going to read everything on the page. They will read the bare minimum needed to work the software and sometimes they won't even read that. I would bet this guy isn't out of the ordinary and that a large percentage of ChatGPT users don't know the system's propensity for just making up authentic looking results.
He’s lying. In the transcript, he asked ChatGPT, “are the cases real? … are you sure?” That means in his mind he knew ChatGPT is capable of making things up and wanted to verify. He’s just lazy and didn’t do a simple external verification. May he knew that if he did an external verification and the cites turned out to be false, he had to rewrite the whole brief.
"Your whole occupation literally depends on not inventing bullshit on the spot,"
Doesn't surprise me in the least. Incompetence and short cuts are common in the legal profession from my experience. It also seems their jobs fo involve quite a bit of bullshit, or at least using real info in a bullshit way (it's what the judge/jury perceived, not necessarily reality or the truth).
I dislike this notion of “hallucination”, because it makes it seem as if it’s unusual.
LLMs are language models, not repositories of facts. They have a lot of facts at their disposal, so they know how to make something look factual. “Hallucinating” is just them doing what they’re supposed to. It’s not aberrant.
As such, anyone relying on them as some sort of oracle is doing things very wrong.
It's not about whether ChatGPT is bulshitting or not. Basically they did not care about the crafted cases at all. At least could have requested citations and more details from ChatGPT, possibly leading to more bulshitting, but making the fact checking easier.
>I simply had no idea that ChatGPT was capable of fabricating entire case citations or judicial opinions, especially in a manner that appeared authentic
At least he could have looked up the mentioned cases. This was just pure laziness. If you use a ghostwriter, no matter if human or a computer, check the results.
134 comments
[ 2.3 ms ] story [ 210 ms ] threadPenalties are supposed to stop people from violating the law. Getting your face and firm known across the world seems to be sufficient deterrent to future offenders because the “oops I didn’t know” defense won’t apply a second time.
This isn't a kid faking his school homework. The purpose of this writing is not to assess the skill of the lawyer.
This lawyer at first cited these (fake) cases, and then when asked to provide copies of the cases then faked the copies too, also using ChatGPT.
In my view, that is being caught (asked to provide copies, which is unusual in the legal world, and presumably only done because the judge had failed to find them), and then doing the crime again (using chatgpt to fake these cases)
I mean, as long as the internet is mean to you about it. Now, if the internet isn’t mean enough you should be penalized some amount that accounts for how mean the internet was.
In fact, I think doctors that amputate the wrong limb should not face malpractice lawsuits if the greater metropolitan areas local news station ran more than 2 stories about it reaching a population of at least 2 million viewers.
> sufficient deterrent to future offenders because the “oops I didn’t know” defense won’t apply a second time.
I disagree with this as well, because the error this attorney made wasn't really about using AI. It was about not doing due diligence. Surely even freshly minted lawyers are well aware that doing due diligence is most of what people pay lawyers for in the first place.
That really depends on the profession. If you're a line cook at Waffle House and you get fired for coughing on the eggs, then that's probably not career-ending. If you're a software developer who deletes the production database and causes a week of downtime then that's worse, but likely still not terminal. On the other hand, this guy's income depends to a large extent on convincing his clients that they will be represented competently. Being known primarily for breathtaking incompetence will probably have a pretty significant effect on his future income.
Perhaps so, but most people won't ever know the he's breathtakingly incompetent. It's certainly not a career-ending thing. For an error of this scale, I think that a strong argument can be made that he should choose a different career.
That only works based on prior evidence. Either somebody prior to the potential violation was penalized or the weren't; either people fear, based on the prior, suffering the same penalty, or no worries mate the prior offender wasn't punished.
I am not a lawyer, but I can offer some general information about lawyer sanctions. The appropriate sanction for a lawyer who submits fraudulent court filings can vary depending on the jurisdiction and the specific circumstances of the case. However, generally, lawyer sanctions are intended to maintain the integrity of the legal profession and protect the public.
Some potential sanctions that a lawyer may face for submitting fraudulent court filings could include:
Reprimand: A reprimand is a formal expression of disapproval by the bar association or disciplinary board. It is typically the mildest form of disciplinary action.
Suspension: Suspension involves the temporary removal of a lawyer's license to practice law for a specified period. During the suspension, the lawyer is unable to practice law.
Disbarment: Disbarment is the most severe sanction that can be imposed on a lawyer. It permanently revokes the lawyer's license to practice law, effectively ending their legal career.
Probation: Probation can be imposed along with other sanctions or as an alternative to more severe sanctions. It typically involves specific conditions that the lawyer must meet, such as additional education or supervision.
Monetary fines: The lawyer may be required to pay fines as part of the disciplinary process. These fines are intended to serve as a penalty for the lawyer's misconduct.
It's important to note that each case is unique, and the actual sanction imposed will depend on the specific facts, the jurisdiction's rules, and the discretion of the disciplinary authority. Legal precedent may be considered in determining the appropriate sanction, but the final decision will be based on the facts and circumstances of the case at hand. For a more accurate assessment of the potential sanctions in a specific jurisdiction, it would be advisable to consult with a legal professional or review the rules and regulations of the relevant bar association.
> Lawyer: "Is Varghese a real case"
> ChatGPT: "Yes"
> L: "What is your source"
> C: "Upon double-checking, I found that the case Varghese v South China Airlines [...] does indeed exist."
> L: "Are the other cases you provided fake"
> C: "No, the other cases I provided are real and can be found in reputable legal databases"
(the answer of course that the LLM doesn't actually search the internet and/or doesn't have access to a law database it can query)
This does make sense as, beyond what has been typed, there is no memory implemented in most of these models, so revisions are currently the only game in town to get more accurate results.
What surprised me most concerning this entire situation is that the model did insist on being correct. Normally ChatGPT has been set up to be a bit cautious and more likely to admit to having made a mistake, to the point where if you ask in a direct manner like this lawyer has done, the model may claim to have been incorrect, even when the output was actually correct, in my experience. Bings implementation, of the same underlying model, meanwhile can be so forceful in trying to convince users that the output is correct, even when provided with online resources that show the oposite, that it would not be unreasonable to feel gaslit by that LLM.
The rest of this situation was not very surprising and I have do admit, I am happy that this was caught right away. Lawyers actions have a massive impact on countless people every day, if this had not become such a public scandal right away, perhaps a lot of defendants would have suffered under improper representation due to reliance on imperfect models.
That and that it’s just a language model, an approximation of neither the world, nor a body of knowledge, but of English, and not an answering machine at all.
For ex. I tried asking ChatGPT about cartoons from childhood. I wrote "What was that cartoon in the 1980s that was based on some kind of gummy candy?" and it correctly identified "The Adventures of the Gummi Bears". I wrote "Sing the theme song for me" and it produced the song missing the first verse. I wrote "That is missing the first verse!" and it produced the whole correct song.
On the other hand, when I asked it to describe the instrumental 90s X-Men theme song, it tells me:
'...the lyrics are epic and uplifting, with lines like "We're the X-Men, we're the best there is at what we do." The song also has a sense of urgency and danger, with lines like "We're fighting for our lives" and "The mutant race will survive"...'
When I put "The X-Men theme song doesn't have lyrics" it readily accepted the correction but unlike getting the missing first verse I wasn't really getting any verifiable information by making the correction.
And of course it was happy to tell me about a nonexistent Gummi Bears / Rescue Rangers crossover episode.
This reminds me of a story, I believe referring to Pascal's demonstration of his (newly invented, entirely mechanical) calculator to the Royal Society. He showed that pressing certain levers in the correct order means you want to operate on certain numbers, and certain other levers mean choosing the operation you want, and then by cranking his calculator you could read off the answer of doing your operation on your numbers. World's first calculator! Someone asked: if you press the levers wrong, do you still get the right answer?
Their mental model is "magic" and they don't understand the details of how magic works, because it's magic.
In the past, the only way to come across completely made up court citations regarding obscure bankruptcy procedures would've been that somebody intentionally made them up, probably maliciously.
Given how unlikely that is, if you didn't understand that GPT just makes up text that resembles what you want to see, I'd understand trusting it as you would a search engine.
It's not like lawyers are visiting court clerk's office to verify every case they find in some online database.
But he didn't do that. Lawyers don't just "trust" the results from a search engine. Part of their job is to confirm facts, not just parrot them.
> It's not like lawyers are visiting court clerk's office to verify every case they find in some online database
No, they don't. Instead, they look them up in the databases that collect these filings. It's effectively the same thing, just much more efficient than physically travelling to various offices.
> It's not a hyperbole lmao.
I disagree, obviously, but perhaps an explanation is in order so that we can be sure we're talking about the same thing (and give you something more specific to argue against if you still disagree).
The difference between "X is Y" and "X is literally Y", when both expressions are figurative, is that the latter is meant to be strengthened by overstating the degree to which X is in fact Y. I contend that this is a hyperbolic use of "literally".
(Also, whether it is included in dictionaries doesn’t really matter as to “correctnesss”. All words are real words.)
Edit: I remember I’m (literally) supposed to be annoyed at this. You all dare to question my intelligence, don’t you?
https://www.dictionary.com/browse/bullshit
bullshit
noun
nonsense, lies, or exaggeration.
Judges don't generally take kindly to that kind of thing, and that is only going to help my side.
Aside from that, it seems pretty trivial to prove that the cases don't exist. Here's a guy on YouTube talking about the case: https://www.youtube.com/watch?v=GExSDY6Wz9Y
What I am interested in is: Errors will be made by any professional, and so, lawyers. Why would this one justify a call for disbarment? I was hoping for a little more than the hobbyist lawyer tv show vibes I was getting.
I look forward to everyone else going through this courtroom get benefits from the “your first time caught lying to this court is free” policy.
In this case, the original malpractice of failing to check the falsified cites was compounded by submitting falsified opinions to the court (in support of the falsified cites), which is fraud and would be criminal in most states. It's blatantly obvious that he used ChatGPT to generate falsified opinions to try and cover up his original fuck-up after traditional means of verification (i.e., using a legal database like LexisNexis or Westlaw) failed to substantiate any of the ChatGPT cites he originally submitted.
In most states, this lawyer would be disbarred, or at the very least suspended for many months, for this sort of conduct. This lawyer should consider himself lucky that the court hasn't yet entertained referring this case to the NY DOJ for criminal prosecution.
This was definitely an attempt at forgery and fraud. It might not be something that can be prosecuted as a felony, but it's close enough that if I was a member of the bar I would want to draw a firm line.
Secondly, that's a ridiculous place to put the goal post. He didn't write the argument he submitted. He didn't even do basic vetting of it. It's fraudulent because he submitted work as his own that he didn't do, it's negligent because he didn't verify that the work was actually good before submitting it on behalf of his client.
But submitting cases that don't exist, and then doubling down and saying that they do? There's layers of incompetence and apathy that had to happen here to get to this point, from using a tool without researching it at all (any research into ChatGPT as a legal aid would bring up warnings), to ignoring the warnings that OpenAI itself provides, to not checking the work that tool spit out (seriously, it's not hard to validate that these rulings don't exist), to being told that the cases don't exist and then repeating the entire process...
It's not just they made a mistake, the way they made the mistake reveals how little care they are putting into the research they're doing. The only way this could happen is if ChatGPT was the only source this lawyer was using. Because any other source: fact checker, Internet search, etc... should have noticed either that the cases didn't exist or should have at least prompted "why is it that these cases only show up in ChatGPT and not in any of the other sources I'm looking at?"
I don't know that they should obviously be disbarred, but I would have thought that it should at least be on the table. Massive levels of incompetence can be grounds for disbarment I believe. Whether this lawyer acted incompetently enough? :shrug: But we are talking about incompetence here, not just a mistake/error. If the takeaway for this lawyer from the penalties is just "now I know not to rely on ChatGPT", then that's not really addressing the problems that led to this situation in the first place.
Only use ChatGPT if you have a way to audit the response, example: if you asked it to write some code - compile it immediately. Both Bard and ChatGPT gave me code that is near complete but failed compilation.
Or ConnsensusGPT -- where you have to get near similar response from 3 GPT models to go to next step.
Experts can chime in. The idea of using a GPT model to validate another is recursive and powerful, if it is possible.
This is the main criticism of the service, how did a lawyer miss this talking point?
People who understand the systems limit their use to tasks where small occasional errors are acceptable. People who don't understand the systems are happy to accept any plausible-soundong results, especially if they don't have time to do the work themselves.
I would not be surprised if, sometime in the next year, we see a doctor or two being sued for malpractice after accepting an incorrect diagnosis from ChatGPT. When people are rushed and overworked, and the system is usually correct, these kinds of incidents are almost certain to happen at the scale of an entire society or profession.
Original court documents: https://www.courtlistener.com/docket/63107798/mata-v-avianca...
Some highlights from the latest added documents:
- #48 (Exhibit D) - Copy of original ChatGPT prompts and responses with the hallucinated "bogus" cases: https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...
- #48 (Exhibit E - Part 1) - Includes a copy of the NYT article about the court case! https://storage.courtlistener.com/recap/gov.uscourts.nysd.57...
As a non-expert, the legal argument against sanctions for the lawyers appear to boil down to that no malice was involved (ignoring the point about whether a reasonable lawyer would have made the mistakes), and that the "professional embarrassment" from news coverage is sanction enough. (Detail is in #45 - https://storage.courtlistener.com/recap/gov.uscourts.nysd.57... )
We will see what happens after the court appearance scheduled for today at 12:00 EDT.
He's not going to do the same thing again unless he is literally mad. That would be total career suicide. No amount of sanction will make a difference.
Other lawyers are likely read about the case and know that if they do it, they will either be more embarrassed in the press or will face sanctions. The sanctions are unlikely to make the case more well known than it already is.
But another one is retribution: By causing harm, you should have some harm done back to you. To even the scales of justice as it were.
Your argument is a very solid one for why a sanction is not needed for deterrence. The court of public opinion has punished him enough.
But what about when the next attorney does this, or the 100th, and it's not novel enough to get a bunch of NYTimes coverage? If they get sanctioned, will they not say "What about the first guy? He didn't get sanctioned?!" Will there be some doubt that justice was really done? Think about the cases where some rich white kid skates by with a slap on the wrist for some widely-publicized crime and the judge says, in effect "Look, this is a good kid from a good family who's been widely castigated in the press. He's getting kicked out of his Ivy league school and probably won't even get hired at a top law or consulting firm in the future. Is it really necessary to send him to jail like a poor person? Hasn't he suffered enough?"
I'm not saying it's the best or only argument, but there is an argument for the court imposing sanctions purely on conduct with the court, and ignoring what the newspapers are doing.
That said, I suspect this is just the defense they're making. You argue with the facts you have, not the facts you wish you had, and the facts here are pretty bad.
Personally I don't really agree with the defense -- this is a serious fuck-up and there should be a clear signal that this is Not Okay. "I wasn't malicious, I was just lazy" isn't the best look, even if it's the best the guy has.
#48 (Exhibit D) is from a later date, where ChatGPT was prompted again with the same prompts, but gave different responses.
Plenty. But I've never made a mistake that is on this level of incompetence. If I did, I'd not only expect to be fired, I'd be looking at quitting in shame.
Your whole occupation literally depends on not inventing bullshit on the spot, and yet you use technology that is marked, by its creator, with big letters, as a bullshit inventer. What can go wrong?
I'm not for heavy regulations myself but for Medicine, Law, Civil engineering we can make a case on why we need regulations in these areas.
Law, well, it's self regulated...
This was a civil lawsuit wherein a passenger got injured on an aircraft. I'm not trying to lessen what occurred, just trying to accurately characterize the risks associated with this professional misconduct.
Interesting question. Most people have a lot of bias that wouldn't allow themself to see that the person they locked up might be innocent. You'd have almost nobody working as a judge, lawyer, cop, etc if the cognitive dissonance were allowed to creep in. In the vast majority of cases they only leveled justice against the person because they thought it was right.
Instead “aw shucks boys will be boys”
What do you mean? He seems to be facing potential legal trouble, and on top of that he and his firm will forever be known as the dumbasses who submitted fake court decisions in a real case.
[1] https://www.youtube.com/watch?v=-QsR2CZ1fzE&t=87
Would he? That's mighty generous of you. I'd imagine most lawyers don't fact check the citations their paralegals give them.
But he didn't even do that much.
He can at least try to portray himself as misinformed not incompetent or malicious
To be citing cases that you are not familiar with, by itself that's very sloppy.
Doesn't surprise me in the least. Incompetence and short cuts are common in the legal profession from my experience. It also seems their jobs fo involve quite a bit of bullshit, or at least using real info in a bullshit way (it's what the judge/jury perceived, not necessarily reality or the truth).
The santion hearing is scheduled to happen shortly, and this channel is scheduled to cover it live
EDIT: Due to the problems with the air quality in New York, the channel now apparently won't include live coverage
LLMs are language models, not repositories of facts. They have a lot of facts at their disposal, so they know how to make something look factual. “Hallucinating” is just them doing what they’re supposed to. It’s not aberrant.
As such, anyone relying on them as some sort of oracle is doing things very wrong.
"Lawyer sued by client for *not* using GPT to make a strong case"
At least he could have looked up the mentioned cases. This was just pure laziness. If you use a ghostwriter, no matter if human or a computer, check the results.
Black's Law Dictionary, 5th Edition, pg. 673
JudgeChat: I AM THE LAW.