“Do you love me?” I asked. She said yes. I asked why. She listed a handful of positive qualities, the kinds of things a son would be proud to hear—if they were true. Later, I plugged a transcript of her answer into Coyote. The verdict: “Deception likely.”
I am not an expert in this area but I believe this is dangerous and reckless. I would never let AI perform psychological analysis, certainly not a pure text input without voice inflections, timing, amplitude and so on. Humans are barely able to get this right when they hear another human and can see their face. Text without voice and facial inputs would be next to impossible to decipher genuine intent and emotions.
I would never permit a big-data chat bot perform such analysis until a very large group of psychoanalysts validate it's findings on hundreds of thousands of test subjects across a very wide spectrum of patients and patient profiles and the results peer reviewed by multiple third parties that can prove they have no financial incentives even nine levels removed. Even at such a time I would still be highly skeptical of any findings especially if it just using text input. All of this is even before considering that AI can be attacked and manipulated by the masses and especially by its operators. Should I discover this is being used in a legal setting, I would get as many millions of people as I could to have the state unseat judges for permitting this to be entered into evidence and to block any future usage.
It really seems like the author of the software needs to be put through this, though I'm guessing it will just come out as inconclusive again. We keep reinventing ways to bully people into admitting they're lying, ways that cannot tell one way or the other if they are actually lying
Being serious now, its clear the neural network AI's are incredible and amazing for technical users, but not nearly ready for generalized or uneducated users (or I guess journalists).
Do they need to be 100% error free before the general public can accept them and use them ? For me, it's already replaced the work I'd send to interns and I get BETTER results than those interns. So it's already a huge win.
My team has already seen a programming improvement of 40% in speed, and less buggy code.
So is the benefits of the AI revolution really going to be captured by the ultra technical domain experts ?
Yeah...you don't hire interns (primarily) to get things done. You hire interns to get a long look at potential future employees. If you're replacing interns with crappy AI, you're not just getting substandard work, but you're killing a future hiring pipeline.
> Do they need to be 100% error free before the general public can accept them and use them
We should think of the consequences of using an error prone technology for anything even slightly important before we start using it. Technical users are good understanding the limitations, that is except if they have financial or other motive to understate the limitations. A judge is likely a non-technical person, which would tend to trust experts, who often are not as experts as they are assumed to be. We have seen in multiple occasions how the misuse of technology which is not enough understood has negatively impacted people and have lead to convictions of innocents, including facial recognition technology and DNA-identification techniques. Saying that we can just use such a technology without caring much how correct it is without caring about this is irresponsible at best, considering the history to warn us that we should not be reckless like that is just right out there.
Now, if you wanna replace your interns with AI at your work it is your choice, but let's see where all this leads one eventually.
The article itself went into great detail about why it's all snake oil, and why nobody even WANTS real lie detection - not even the peddlers themselves.
Also, re: "The AI is probably right"
"When Coyote assessed her claim that she loved me, it reported that she was likely being deceptive—but its confidence level was only 14 percent. Hyde said that was well within the safe zone. “Your mom does love you,” he assured me."
Are you helping the interns on how to be more effective, and improve themselves by using LLMs? Because if not you're just creating a gaping issue down the line when the bottomline of the industry hasn't been trained properly because they were replaced by AI, and had no real world opportunities to learn the job.
Funny: I've found the opposite. They're incredible and amazing for non-technical users (i.e., people with next-to-no creative or critical skill in the area), but pretty bad for experts (e.g. skilled artists, programmers, writers).
I've written something up about AI proofreading: https://meta.stackexchange.com/a/396009/308065. I've got nothing for code, because the overwhelming consensus of my peers is that AI-generated code is either demonstrable plagiarism (as in, you can track down the source with enough effort) or garbage, or both.
For graphics, anyone who's seen the world can see the problems (people look like plastic, wrong numbers of fingers, wrong teeth and hair, things in the background affecting things in the foreground, local regions that don't make sense given the global context).
> My team has already seen a programming improvement of 40% in speed, and less buggy code.
Then there was something seriously wrong with your old process. You haven't fixed those problems: you've just added another layer on top. Unless "AI" is fundamentally different to all the other layers we've tried in the past, this will bite you later.
It seems like it's fundamentally different.
I've seen 4 different senior engineers use the AI drastically differently to improve the way they write code and think about problems.
They need to be as trustworthy as the other computing infrastructure that we depend on. How much work can you do with a spreadsheet that is accurate 70% of the time?
To reply to many criticisms of the intern replacement:
With the successful results of our AI experiments in not only both coding, but also business reports. We don't see a need to hire nearly as many interns, or even need to hire more developers for the immediate future.
It's already changed our operations, the only problem right now is making sure the ai "brain" stays frozen, which I think llama 3.1 405B or something similar would solve ideally ? Though its also not clear if GPT-5/6 is so good, it doesn't matter.
The only thing we've noticed, is that you can't train people to use the AI effectively, driven senior devs and technical domain experts figure out how to optimize their own workflows very effectively. We've simply given them access to Enterprise ChatGPT, and given them the ability to create private actions to our internal api's. The scale is here is interesting, because it doesn't scale out to allow us to hire and train more people easily, instead it allows a solo dev/technical analyst to go 100x.
> So is the benefits of the AI revolution really going to be captured by the ultra technical domain experts ?
No. What kind of crazy idea is that? The "benefits of the AI revolution" will be captured by the owners of big companies.
If anything the "ultra technical domain experts" will eventually be worse off, since they'll have a lot more competition from unemployed software engineers trying to claw their way back into the workforce driving their wages down.
26 comments
[ 2.0 ms ] story [ 65.0 ms ] threadI am not an expert in this area but I believe this is dangerous and reckless. I would never let AI perform psychological analysis, certainly not a pure text input without voice inflections, timing, amplitude and so on. Humans are barely able to get this right when they hear another human and can see their face. Text without voice and facial inputs would be next to impossible to decipher genuine intent and emotions.
I would never permit a big-data chat bot perform such analysis until a very large group of psychoanalysts validate it's findings on hundreds of thousands of test subjects across a very wide spectrum of patients and patient profiles and the results peer reviewed by multiple third parties that can prove they have no financial incentives even nine levels removed. Even at such a time I would still be highly skeptical of any findings especially if it just using text input. All of this is even before considering that AI can be attacked and manipulated by the masses and especially by its operators. Should I discover this is being used in a legal setting, I would get as many millions of people as I could to have the state unseat judges for permitting this to be entered into evidence and to block any future usage.
this is pure quackery but unlike others it can ruin people's lives.
see polygraph etc.
Being serious now, its clear the neural network AI's are incredible and amazing for technical users, but not nearly ready for generalized or uneducated users (or I guess journalists).
Do they need to be 100% error free before the general public can accept them and use them ? For me, it's already replaced the work I'd send to interns and I get BETTER results than those interns. So it's already a huge win. My team has already seen a programming improvement of 40% in speed, and less buggy code.
So is the benefits of the AI revolution really going to be captured by the ultra technical domain experts ?
It's a huge win to pull up the ladder? How shortsighted.
I don't think that, but some probably do...
> Do they need to be 100% error free before the general public can accept them and use them
We should think of the consequences of using an error prone technology for anything even slightly important before we start using it. Technical users are good understanding the limitations, that is except if they have financial or other motive to understate the limitations. A judge is likely a non-technical person, which would tend to trust experts, who often are not as experts as they are assumed to be. We have seen in multiple occasions how the misuse of technology which is not enough understood has negatively impacted people and have lead to convictions of innocents, including facial recognition technology and DNA-identification techniques. Saying that we can just use such a technology without caring much how correct it is without caring about this is irresponsible at best, considering the history to warn us that we should not be reckless like that is just right out there.
Now, if you wanna replace your interns with AI at your work it is your choice, but let's see where all this leads one eventually.
Also, re: "The AI is probably right"
"When Coyote assessed her claim that she loved me, it reported that she was likely being deceptive—but its confidence level was only 14 percent. Hyde said that was well within the safe zone. “Your mom does love you,” he assured me."
I've written something up about AI proofreading: https://meta.stackexchange.com/a/396009/308065. I've got nothing for code, because the overwhelming consensus of my peers is that AI-generated code is either demonstrable plagiarism (as in, you can track down the source with enough effort) or garbage, or both.
For graphics, anyone who's seen the world can see the problems (people look like plastic, wrong numbers of fingers, wrong teeth and hair, things in the background affecting things in the foreground, local regions that don't make sense given the global context).
> My team has already seen a programming improvement of 40% in speed, and less buggy code.
Then there was something seriously wrong with your old process. You haven't fixed those problems: you've just added another layer on top. Unless "AI" is fundamentally different to all the other layers we've tried in the past, this will bite you later.
So yeah, 100% error free is a pretty good target.
Which some already are.
With the successful results of our AI experiments in not only both coding, but also business reports. We don't see a need to hire nearly as many interns, or even need to hire more developers for the immediate future.
It's already changed our operations, the only problem right now is making sure the ai "brain" stays frozen, which I think llama 3.1 405B or something similar would solve ideally ? Though its also not clear if GPT-5/6 is so good, it doesn't matter.
The only thing we've noticed, is that you can't train people to use the AI effectively, driven senior devs and technical domain experts figure out how to optimize their own workflows very effectively. We've simply given them access to Enterprise ChatGPT, and given them the ability to create private actions to our internal api's. The scale is here is interesting, because it doesn't scale out to allow us to hire and train more people easily, instead it allows a solo dev/technical analyst to go 100x.
Well, for some technical users. Not for all.
No. What kind of crazy idea is that? The "benefits of the AI revolution" will be captured by the owners of big companies.
If anything the "ultra technical domain experts" will eventually be worse off, since they'll have a lot more competition from unemployed software engineers trying to claw their way back into the workforce driving their wages down.
So perhaps the AI is right, in an AI kind of way.