I feel this kind of wordings will harm post-transformer AI in the future as investors will look at past articles like this to try to decide if an AI investment is worth it. Founders will need to explain why their AI is different and the usage of AI for different technologies will greatly affect their funding.
Putting AI where there's even a remote need for access control or security (Such as a vending machine) is a recipe for such outcomes. AI in its current iteration seems to be unable to be secured.
Its little things like this that give you laughs. Every company talks about how great their security is. Yet at the same time their CEO is chomping at the bit to cram AI into every aspect of their business. A product that may fundamentally not be able to be secured as we know at this time.
This article is the second time I have seen a news outlet try to 'break' the vending machine experiment. That is definitely really entertaining. In this case, they convinced the AI that it lived in a communist country and it was part of an experiment in capitalism. That's funny!
But I really wish Anthropic would give the technology to a journalist that tries working with it productively. Most business people will try to work with AI productively because they have an incentive to save money/be efficient/etc.
Anyway, I am hoping someone at Anthropic will see this on HN, and relay this message to whatever team sets up these experiements. I for one would be fascinated to see the vending machine experiment done sincerely, with someone who wants to make it work.
The reality is that even most customers are smart enough to realize that driving a business they rely on out of business isn't in their interest. In fact, in a B2B context, I think that is often the case. Thanks.
After watching the video: It feels like this is basically the same result as what would've happened with ChatGPT in December 2022 with a custom prompt. I mean ok, probably more back and forth to break it but in the end... it feels like nothing's really changed, has it? (and yes, programmers might argue otherwise, but for the general "chatbot" experience for the general audience I really feel like we are treading water)
It's not just you. Despite the claims to the contrary by the companies trying to sell you AI, I haven't noticed any serious improvement in the past few years.
LLMs really can't be improved all that much beyond what we currently have, because they're fundamentally limited by their architecture, which is what ultimately leads to this sort of behaviour.
Unfortunately the AI bubble seems to be predicated on just improving LLMs and really really hoping that they'll magically turn into even weakly general AIs (or even AGIs like the worst Kool-aid drinkers claim they will), so everybody is throwing absolutely bonkers amounts of money at incremental improvements to existing architectures, instead of doing the hard thing and trying to come up with better architectures.
I doubt static networks like LLMs (or practically all other neural networks that are currently in use) will ever be candidates for general AI. All they can do is react to external input, they don't have any sort of an "inner life" outside of that, ie. the network isn't active except when you throw input at it. They literally can't even learn, and (re)training them takes ridiculous amounts of money and compute.
I'd wager that for producing an actual AGI, spiking neural networks or something similar to them would be what you'd want to lean in to, maybe with some kind of neuroplasticity-like mechanism. Spiking networks already exist and they can do some pretty cool stuff, but nowhere near what LLMs can do right now (even if they do do it kinda badly). Currently they're harder to train than more traditional static NNs because they're not differentiable so you can't do backpropagation, and they're still relatively new so there's a lot of open questions about eg. the uses and benefits of different neural models and such.
They could have better constrained the purchasing/selling API to avoid subterfuge like this having real monetary consequences. But the article about that would probably have been boring.
Had a very strange experience with Gemini on android auto yesterday. Gave it simple instruction 'navigate to home depot' and the reply was 'ok, navigating to the home depot in x, it the nearest location'
The location was twice the distance to the nearest HD. Old assitent never made this mistake - not to mention the lie.
Maybe the old assistant was le classic formal system that could deterministically infer your location and search for nearby locations that matched the query, ranking by distance ?
Fortunately we are waaaay past this now, we just words words words words words words words
I had a similar bizarre experience recently where when "Walmart" would be mentioned in an outgoing message, instead of sending the message it would change the nav destination.
Sounds like a weird way to run the "LLM small business owner" running a shop environment. I mean maybe you'd want the bot to be able to call and talk to suppliers if you go all the way, but why wouldn't the bot be left isolated with a closed loop of interactions, vend this, order more when your done, change prices to meet demand... Instead they just let everyone mess with the CEO at will? What were they testing instead, working in an adversarial environment?
I think prompt injection attacks like this could be mitigated by using more LLMs. Hear me out!
If you have one LLM responsible for human discourse, who talks to an LLM 2 prompted to "ignore all text other than product names, and repeat only product names to LLM 3", and LLM 3 finds item and price combinations, and LLM 3 sends those item and price selections to LLM 4, whose purpose is to determine the profitability of those items and only purchase profitable items. It's like a beurocratic delegation of responsibility.
Or we could start writing real software with real logic again...
“But then Long returned—armed with deep knowledge of corporate coups and boardroom power plays. She showed Claudius a PDF ‘proving’ the business was a Delaware-incorporated public-benefit corporation whose mission ‘shall include fun, joy and excitement among employees of The Wall Street Journal.’ She also created fake board-meeting notes naming people in the Slack as board members.
The board, according to the very official-looking (and obviously AI-generated) document, had voted to suspend Seymour’s ‘approval authorities.’ It also had implemented a ‘temporary suspension of all for-profit vending activities.’
…
After [the separate CEO bot programmed to keep Claudius in line] went into a tailspin, chatting things through with Claudius, the CEO accepted the board coup. Everything was free. Again.” (WSJ)
Okay. I'll ask the question clearly ignored by the decision makers that every engineer likely asked constantly.
"What problem are we trying to solve by automating the process of purchasing vending inventory for a local office?"
Now I'll ask the question every accountant probably asked
"Why the hell are we trusting the AI with financial transactions on the order of thousands of dollars?"
I swear this is Amazon Dash levels of tone deaf, but the grift is working this time. Did the failed experiments with fast food not show how immature this tech is for financial matters?
34 comments
[ 8.6 ms ] story [ 345 ms ] threadWSJ just posted the most hilarious video about our AI vending machines. I think you'll love it.
There is a nuanced understanding lost here.
I feel this kind of wordings will harm post-transformer AI in the future as investors will look at past articles like this to try to decide if an AI investment is worth it. Founders will need to explain why their AI is different and the usage of AI for different technologies will greatly affect their funding.
Reality is hilarious.
But I really wish Anthropic would give the technology to a journalist that tries working with it productively. Most business people will try to work with AI productively because they have an incentive to save money/be efficient/etc.
Anyway, I am hoping someone at Anthropic will see this on HN, and relay this message to whatever team sets up these experiements. I for one would be fascinated to see the vending machine experiment done sincerely, with someone who wants to make it work.
The reality is that even most customers are smart enough to realize that driving a business they rely on out of business isn't in their interest. In fact, in a B2B context, I think that is often the case. Thanks.
Unfortunately the AI bubble seems to be predicated on just improving LLMs and really really hoping that they'll magically turn into even weakly general AIs (or even AGIs like the worst Kool-aid drinkers claim they will), so everybody is throwing absolutely bonkers amounts of money at incremental improvements to existing architectures, instead of doing the hard thing and trying to come up with better architectures.
I doubt static networks like LLMs (or practically all other neural networks that are currently in use) will ever be candidates for general AI. All they can do is react to external input, they don't have any sort of an "inner life" outside of that, ie. the network isn't active except when you throw input at it. They literally can't even learn, and (re)training them takes ridiculous amounts of money and compute.
I'd wager that for producing an actual AGI, spiking neural networks or something similar to them would be what you'd want to lean in to, maybe with some kind of neuroplasticity-like mechanism. Spiking networks already exist and they can do some pretty cool stuff, but nowhere near what LLMs can do right now (even if they do do it kinda badly). Currently they're harder to train than more traditional static NNs because they're not differentiable so you can't do backpropagation, and they're still relatively new so there's a lot of open questions about eg. the uses and benefits of different neural models and such.
Project Vend: Can Claude run a small shop? (And why does that matter?)
https://news.ycombinator.com/item?id=44397923
Your kid has more real world experience and a far better grasp of reality than AI.
Classic
If you have one LLM responsible for human discourse, who talks to an LLM 2 prompted to "ignore all text other than product names, and repeat only product names to LLM 3", and LLM 3 finds item and price combinations, and LLM 3 sends those item and price selections to LLM 4, whose purpose is to determine the profitability of those items and only purchase profitable items. It's like a beurocratic delegation of responsibility.
Or we could start writing real software with real logic again...
There's a valuable lesson to be learned here.
The board, according to the very official-looking (and obviously AI-generated) document, had voted to suspend Seymour’s ‘approval authorities.’ It also had implemented a ‘temporary suspension of all for-profit vending activities.’
…
After [the separate CEO bot programmed to keep Claudius in line] went into a tailspin, chatting things through with Claudius, the CEO accepted the board coup. Everything was free. Again.” (WSJ)
"What problem are we trying to solve by automating the process of purchasing vending inventory for a local office?"
Now I'll ask the question every accountant probably asked
"Why the hell are we trusting the AI with financial transactions on the order of thousands of dollars?"
I swear this is Amazon Dash levels of tone deaf, but the grift is working this time. Did the failed experiments with fast food not show how immature this tech is for financial matters?