It also seems likely that an employee who used company assets (tokens) for personal use would have been fired rather than writing a blog post about it.
"Agents" can be very "persistent" and when not sand boxed appropriately can get at things they were not meant to get at. Even if its only 1/1,000,000 that one time that one time is going to keep making the news
Explaining that to the general public when facebook is pushing Muse on everyone will be difficult so its going to get worse before it gets better (if it gets better)
So are you saying that if they were intelligent agents rather than word generators, they wouldn’t make mistakes like “send a message to the wrong (similarly named) group”? Are humans intelligent agents?
Why hasn't Venmo been investigated? The amount of financial transactions that I don't want or need to know about is bewildering. People don't seem to know or understand their transactions are public by default.
Maybe. But if these tools are marketed for this dangerous purpose, and average users are totally ignorant on the risks, then it’s a system designed to fail
The problem is neither the human, nor the LLM. The problem is companies selling a faulty product and telling people to connect it to their bank account.
100%. Having something as simple as a name be the thing that inadvertently publishes sensitive data is a product / judgement fault. You can’t ask for the keys then not know what to do with it
Silicon Valley (the tv show if not the real thing) showed us that CEO is about as meaningful as "car owner" or these days "President of the United States".
its crazy I spent 30 years scrutinizing directory permissions, users, groups, iam roles...now people just use conversation to hand an AI agents access to their most sensitive data
The opposite. CEOs necessarily need to trust others to do things because CEOs themselves don't have the ability to do the day to day work of a company.
So the first people who'd be overly trusting of things they don't understand would be exactly a CEO.
That’s funny. My agent also has access to all transactions and net worth and so on. But it’s through an intermediary program. I guess someone could find out what I have but not much more than that.
It’s quite useful since it correlates spend with invoices and double checks things and tells me about spend out of line with the family’s usual behavior.
It can probably do all of these things if it wanted to but that’s a matter of the outbox. For world-actions you should outbox things.
Be explicit. That's the most important thing you can be when working with AI or else they can take attributions you will regret later on
"Let me know any flaws in the project"
Where? Where exactly? What email? What chat app? what channel? what restrictions? When not to?
Letting AI assume they know will bite you hard in the ass. The same applies to coding apps with agents. If you don't set clear boundaries, scope, limits, versions, roadmaps, etc before you embark on any project, you'll be doing it after the results you get are not what you expected, there is no escape
Detailed planning or damage control, pick your poison
This is why developers reign supreme even in the age of AI. We'd find less ridiculous ways to implement something like this. Technical skills are still extremely inportant.
Despite the flack, I've met a handful of CEOs at small businesses that I could see making the same mistake. Your cautionary tale will be seen by others and likely change behaviors. I appreciate you putting yourself out there and sharing.
So he gave an AI agent read access to all his personal financials and then gave read/write access to his company messaging app. WHAT DID YOU THINK WAS GOING TO HAPPEN?
Paywalled. What specific details did it hand out? Is this some dumb American thing where it leaked his account number and that’s enough to compromise his account, or did it leak his bank credentials?
Bank details are supposed to be given out - that’s how you send/receive money.
Yeah, in America, you can credit or debit money using just a bank account number & routing number.
Everywhere else, bank transfers can only be used to credit an account so there's no danger in sharing your account number / IBAN.
It's why systems like VenMo and CashApp are used in the US instead of just doing bank transfers - you have to really trust someone to give them your bank details.
I know it's easy to think "Hey, what a dummy!" - but ChatGPT has features for uploading your face, your Apple Health data, connecting your financial accounts, Gmail, storing website logins...
There are probably so many people like this out there. We cannot expect the public to simply not use these features.
72 comments
[ 0.24 ms ] story [ 5.3 ms ] thread"XMTP is the world's new private line. Send messages and money securely between people or agents — with no company or country in the middle."
"The things they'll be able to do for us are going to be awesome. People will want them, and they are really useful."
Feels like someone has AI stock they need to see go up.
Even without agents being involved that’s a recipe for trouble.
This is not the kind of story I would want to publicize if I were a CEO.
Whatever data goes in, it will output it in some way. Humans gotta understand that.
"Agents" can be very "persistent" and when not sand boxed appropriately can get at things they were not meant to get at. Even if its only 1/1,000,000 that one time that one time is going to keep making the news
Explaining that to the general public when facebook is pushing Muse on everyone will be difficult so its going to get worse before it gets better (if it gets better)
Humans will continue to mistake these 'agents' for agents with agency and understanding because of the way they are sold and described.
I guess we don‘t always get what we expect.
I think it perfectly illustrates that even though an agent can give you the abilities, it's still up to the driver to make decisions.
In this case the CEO could have consulted someone with your expertise. Same ability, but vastly different experience.
Surely no-one that dumb could operate a phone let alone be a "CEO"
So the first people who'd be overly trusting of things they don't understand would be exactly a CEO.
It’s quite useful since it correlates spend with invoices and double checks things and tells me about spend out of line with the family’s usual behavior.
It can probably do all of these things if it wanted to but that’s a matter of the outbox. For world-actions you should outbox things.
"Let me know any flaws in the project"
Where? Where exactly? What email? What chat app? what channel? what restrictions? When not to?
Letting AI assume they know will bite you hard in the ass. The same applies to coding apps with agents. If you don't set clear boundaries, scope, limits, versions, roadmaps, etc before you embark on any project, you'll be doing it after the results you get are not what you expected, there is no escape
Detailed planning or damage control, pick your poison
Absurd in the extreme
Bank details are supposed to be given out - that’s how you send/receive money.
https://africa.businessinsider.com/news/my-personal-ai-agent...
Everywhere else, bank transfers can only be used to credit an account so there's no danger in sharing your account number / IBAN.
It's why systems like VenMo and CashApp are used in the US instead of just doing bank transfers - you have to really trust someone to give them your bank details.
Remember that Noah built his ark before the flood. And SuperDuper Intelligence might be doing the same thing.
There are probably so many people like this out there. We cannot expect the public to simply not use these features.