The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court.
In many cases the skills are available in house to do the necessary vetting, but these people are already overwhelmed with their existing day to day.
Anyone remember that item a few months back about Amazon now having senior engineers vet generative AI output (https://news.ycombinator.com/item?id=47323017)? I had to LOL when I read that. These folks are already slammed. And the idea that Amazon would allow human bottlenecks to multiply across projects and underlying infrastructure development is ridiculous.
>>The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people
I am particularly interested in Education and Human Knowledge Management. I have seen the rate of IT training going to zero. Think about specialized training, where if you make a mistake, the consequence of your errors, are talked about on the tv news of the evening.
The whole idea everybody is just planning to save their butt, using these strings coming out of these numeric matrices, while suspending judgement, just shudders me in horror. A bit like those South Asia Airline companies, that were forbidding their pilots from landing airplanes with manual piloting, leading to an increase loss of skills causing some well known disasters...
If well paid consultants cant even bother to check their links...
>> Amazon now having senior engineers vet generative AI output
Software and news are apples and oranges. The software engineer looks at AI code to spots errors and make/suggest corrections. But the software still exists. A doctor vetting a news article ("Owning a Boat Causes Cancer") isn't there to tweak language. The doctor's job is to stop the entire article. It would be like the amazon engineer deleting all the source code and telling the dev team to abandon the project. That will never be a popular task.
It's similar to tokenmaxxing in various companies. Who cares about reviewing the design and the code, just code generation all the way. If it runs it's good
It looks like when you sum up: the cost to generate information using an LLM + the cost to actually verify the information, the result on average is the same as not using the LLM. That does not mean some times it isn’t faster and cheaper. It just means other times it’s slower and more costly. This along with different people’s tolerance for accuracy explains why we see such diverging experiences with it.
So in order to make it pan out the forces at play are trying to make everyone believe we now have to accept wrong, even dangerous results.
This sort of thing is a complete embarrassment to a firm like EY, where people are paying them a lot of money for advice. They’ve basically demonstrated that their market leading research is just someone asking questions to ChatGPT.
If you ever needed evidence to not buy “advice” from such outfits, this is exhibit one.
Hopefully they at least fired the partner that published this steaming pile of AI slop.
I don't quite get it why they can't take another LLM and vet the output of the first with the second one. Surely they would not have the same hallucinations and would be able to detect hallucinations of the earlier LLM. Maybe it would cost too much in terms of tokens?
I don't know but I would expect it to be realtively easy for an LLM to detect "hallucinations".
I am not exactly sure if this would solve the overall problem. The main one being lack of oversight. The solution to a social issue generally isn’t to throw more technology at it.
I think the AIs don't have enough information about the problem. There's many things those who wrote the prompts forgot to mention. And some of it maybe is tacit knowledge?
Then, it doesn't matter if you add 1000 frontier models -- they still can't generate a good report.
But yes I suppose you can get rid of hallucinated citations though
I don't quite get it why they can't take another LLM and vet the output of the first with the second one. Surely they would not have the same hallucinations and would be able to detect hallucinations of the earlier LLM. Maybe it would cost too much in terms of tokens?
I don't know but I would expect it to be relatively easy for an LLM to detect "hallucinations".
What’s strange about how things have developed is that this report 12-18 months ago would have been a massive scandal and would have caused durable brand damage.
This proves (again) one think for sure: The "Big x" Consulting Firms were always BS - and now them generating all their work themselves using LLMs just profs that their 'clients' can just skip their Million Dollar fees and just ask the LLM directly.
Basically the entire consulting industry should die due to AI.
Performative executives of yesteryear that constantly need external validation and direction and operate through hive mind and groupthink are weak and will die.
I believe some of the biggest problems in today's business leaders are an inability to be open to new information, to think across traditional professional boundaries, or to ask meaningful questions.
AI simply exposes this unapologetically.
Bad management (this includes most government): up your game or get out of the way.
Firefox has a handy "Reader view" (Opt + CMD + R on Mac) that you can activate to get a stripped down view of just the text on the page. Unfortunately, it also removes the images which contain some of the sources they use.
The real comedy is seeing this garbage come down from senior management, clumsy prompting, hallucinated garbage that’s all fluff and zero actionable information, zero real informed analysis. “See this analysis of our support issues from jira, we must fix these top three problems!!!” And it’s all the stuff everyone has known for years but management has refused to give anyone the authority to fix anything. I’ve seen this more than twice now; needs a name. Garbagemaxxing?
> we must fix these top three problems!!!” And it’s all the stuff everyone has known for years but management has refused to give anyone the authority to fix
I wish we could just stop destroying people's jobs and lives using AI. The statistics I have heard quoted say, that merely 25% of the people actually like their job. Meaning they like doing what they do for its own sake, not because it gets them money, which they desperately need to live. I get it, most people don't want to do the work. But can we stop ruining the jobs of people, who are actually dedicated to their job and would like to keep doing their job properly?
But I guess since EY is a CYA hedge anyway, no one really cares about whether the reports are hallucinations or not. Someone high up spent money on EY, so that they can justify some decision and won't be held responsible that much, when it turns out the decision was shit. All that matters to them is, that it has the appearance of something genuine and then they can base the decision on what they receive from EY, which better be what they already wanted to hear/read anyway.
I guess this is a great report, but the parallax landing page shenanigans disrupt my reading flow, you cannot easily scroll back to get a overview of the key facts, so I stopped.
61 comments
[ 3.3 ms ] story [ 77.8 ms ] threadIn many cases the skills are available in house to do the necessary vetting, but these people are already overwhelmed with their existing day to day.
Anyone remember that item a few months back about Amazon now having senior engineers vet generative AI output (https://news.ycombinator.com/item?id=47323017)? I had to LOL when I read that. These folks are already slammed. And the idea that Amazon would allow human bottlenecks to multiply across projects and underlying infrastructure development is ridiculous.
I am particularly interested in Education and Human Knowledge Management. I have seen the rate of IT training going to zero. Think about specialized training, where if you make a mistake, the consequence of your errors, are talked about on the tv news of the evening.
The whole idea everybody is just planning to save their butt, using these strings coming out of these numeric matrices, while suspending judgement, just shudders me in horror. A bit like those South Asia Airline companies, that were forbidding their pilots from landing airplanes with manual piloting, leading to an increase loss of skills causing some well known disasters...
If well paid consultants cant even bother to check their links...
Software and news are apples and oranges. The software engineer looks at AI code to spots errors and make/suggest corrections. But the software still exists. A doctor vetting a news article ("Owning a Boat Causes Cancer") isn't there to tweak language. The doctor's job is to stop the entire article. It would be like the amazon engineer deleting all the source code and telling the dev team to abandon the project. That will never be a popular task.
So in order to make it pan out the forces at play are trying to make everyone believe we now have to accept wrong, even dangerous results.
It's unsurprising that trying to do more with less results in lower quality.
If you ever needed evidence to not buy “advice” from such outfits, this is exhibit one.
Hopefully they at least fired the partner that published this steaming pile of AI slop.
I don't know but I would expect it to be realtively easy for an LLM to detect "hallucinations".
Then, it doesn't matter if you add 1000 frontier models -- they still can't generate a good report.
But yes I suppose you can get rid of hallucinated citations though
I don't know but I would expect it to be relatively easy for an LLM to detect "hallucinations".
Now nobody will remember or notice.
Performative executives of yesteryear that constantly need external validation and direction and operate through hive mind and groupthink are weak and will die.
I believe some of the biggest problems in today's business leaders are an inability to be open to new information, to think across traditional professional boundaries, or to ask meaningful questions.
AI simply exposes this unapologetically.
Bad management (this includes most government): up your game or get out of the way.
Sycophantic consultant firms: die.
The Economist should do an article on this.
As is noted in the conclusion.
So a net positive then?
But I guess since EY is a CYA hedge anyway, no one really cares about whether the reports are hallucinations or not. Someone high up spent money on EY, so that they can justify some decision and won't be held responsible that much, when it turns out the decision was shit. All that matters to them is, that it has the appearance of something genuine and then they can base the decision on what they receive from EY, which better be what they already wanted to hear/read anyway.
~ A greedy, dishonest and unethical capitalist.
Unfortunately, they forgot to check the name beforehand, and it turned out that "EY!" already existed: a gay porn magazine.
Yes, this really happened.